How to Serve Multiple Regions Using a Single Workload

Some time ago, I worked on a project where we used Rockset for real-time data analytics. We needed to send data through private connections, not over the internet. Rockset, like most SaaS providers, offered private connections in only three regions, none of which matched ours. On another occasion, I faced the same challenge when I tried to implement an integration with Materialize.

This is a challenge faced by many enterprise that need private connection to SaaS vendors. Often, they have no choice but to transfer sensitive data over the internet. When I spoke to some vendors, they explained that supporting every region is expensive because there aren't enough customers in those region to justify the cost.

To address this challenge, I want to share a solution where SaaS providers can offer their services to customers in multiple regions using a single workload.
For example, say you have customers in Europe but your applications and workloads are deployed in Canada, or if you want to expand to multiple regions without dealing with the complexity and cost of managing servers and databases across different regions, you can solve this using simple networking and load balancers.

How Does The Solution Work?

Suppose your applications are deployed in the Canada region, which is your main base of operations. We'll call this the ca-west-1 region. Now, let's assume you have enterprise partners and end users in the Europe region, which we'll call eu-central-1. Here's how you can serve your European customers:

• Establish a link between ca-west-1 and eu-central-1 regions through a VPC peering connection or transit gateway peering.

• Setup application and network load balancers in eu-central-1 that target your workloads in ca-west-1.

• Provision a VPC endpoint service connected to the network load balancer. This will provide PrivateLink connection to enterprise customers.

• Provision a domain for the application load balancer. This will provide service to general customers.

Take a look at the architecture below to see how it all comes together.

In the diagram above, there are two VPCs connected by a VPC peering connection between the Europe and Canada regions. Target groups are created to help load balancers find applications easily. These load balancers are exposed to customers through Route 53 and securely to partners via AWS PrivateLink.

When an instance is added or removed from the autoscaling fleet in the Canada region, an event is published. This event is enriched with the instance's IP and port by a Lambda function before being sent to other regions through EventBridge. Once the event reaches the destination region, a Lambda function picks it up and updates the target groups with the new target.

Implementation

To implement this setup, we'll use Terraform, a popular vendor-agnostic Infrastructure as Code (IaC) tool used by modern engineers.

The full project is available on my GitHub repo.

Requirements

• An AWS account

• AWS CLI installed and configured with credentials that have sufficient permissions

• Terraform v1.7 or later

Step 1: Setup Workload in Base Region (Canada)

Step 2: Setup VPC in Secondary Region (Europe)

Step 3: Peer Europe & Canada Regions

Step 4: Setup Load Balancers & Target Groups in Europe

Step 5: Lambda Publish Instance Launch Event

import { getInstanceIpAddress, publishEvent, completeLifecycleAction } from './utils.js';

async function handleInstanceLaunch(event) {
  const port = process.env.APP_PORT;
  const availabilityZone = process.env.AVAILABILITY_ZONE || 'all';
  const { detail } = event;

  if(!detail.EC2InstanceId) {
    throw new Error('Details contains no instance ID');
  }

  const ipAddress = await getInstanceIpAddress(detail.EC2InstanceId);

  await publishEvent('INSTANCE_IS_LAUNCHING', {
    instanceId: detail.EC2InstanceId,
    ipAddress,
    port,
    availabilityZone
  });

  await completeLifecycleAction({
    asgName: detail.AutoScalingGroupName,
    instanceId: detail.EC2InstanceId,
    lifecycleHookName: detail.LifecycleHookName,
    lifecycleActionToken: detail.LifecycleActionToken
  });
}

export { handleInstanceLaunch };

Step 6: Lambda Capture Event & Update Target Groups

import { ElasticLoadBalancingV2Client, RegisterTargetsCommand } from '@aws-sdk/client-elastic-load-balancing-v2';

const elasticLoadBalancingV2Client = new ElasticLoadBalancingV2Client();

async function registerTarget (event) {
  const targetGroupArns = JSON.parse(process.env.TARGET_GROUP_ARNS);
  const { detail } = event;

  for (const targetGroupArn of targetGroupArns) {
    const registerTargetsCommand = new RegisterTargetsCommand({
      TargetGroupArn: targetGroupArn,
      Targets: [{
        Id: detail.ipAddress,
        Port: detail.port,
        AvailabilityZone: detail.availabilityZone
      }]
    });

    await elasticLoadBalancingV2Client.send(registerTargetsCommand);
  }
}

export { registerTarget };

The full project is available on my GitHub repo.

Speed Analysis

We'll make requests to each region to see how fast the service responds. We'll monitor the speed using the network analyzer in Google Chrome.

Calgary to Calgary

The image below shows a request made by a user in Canada to the service endpoint in Calgary. As expected, the latency is very low. It takes 21 milliseconds to receive a response for users in Calgary.

Berlin, Germany to Calgary, Canada (~7,500km apart)

The image below shows a request made by a user in Berlin to the service endpoint in Germany. Excluding the time for Queueing and Stalled, the request took about 400 milliseconds. This is more than sufficient for most applications.

Cost Analysis

For cost analysis, since the cost of the primary region is fixed, we'll only consider the cost of running the secondary region.

For an additional $113 each month, you can have another region running and serving requests. This is much cheaper than setting up compute and database resources in another region.

These costs were calculated using the pricing provided in the AWS documentation.

Benefits

• Wider product reach: easily serve customers in multiple region

• Reduced cost: as seen from the cost analysis

• Reduced complexity: simplifies multi-region resource management

Limitations

• Reduced resiliency: failure of the primary region cripples all secondary region

• Increased latency: not suitable for applications needing very fast response times

• Data sovereignty: not suitable for customers who want their data to stay in their own country/region

• vpc-peering connection limitations (Can be resolved by replacing with transit gateway)

Conclusion

Sometimes, as service providers, we have customers in regions we don't currently support. This shouldn't be a reason to miss out on revenue or fail to offer them the best experience. With a bit of setup, you can provide them with something secure, private, and better than using the internet. Plus, when you eventually move workloads to those regions, your customers won't need to change anything on their end.

In this article, we explored a multi-region solution for serving customers using AWS services. By deploying a load balancer and leveraging AWS PrivateLink, you can securely and efficiently serve customers in different regions without managing servers across multiple locations. We covered the architecture, implementation using Terraform, speed analysis, cost analysis, and the benefits and limitations of this approach.

---

References

https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer

https://aws.amazon.com/elasticloadbalancing/pricing/

https://aws.amazon.com/lambda/pricing/