Multi-Region Replication in AWS

AWS supports Cross-Region Replication (CRR) to allow replication of contents in an S3 bucket into another bucket in a different region. This works perfectly well if you need to replicate contents of one bucket to another, but it does not allow replicating the contents of the source bucket to buckets in multiple regions.

Let’s take a sample scenario: we need to replicate contents of a source bucket “annex-test-replication-source” in us-east-1 region into “annex-test-replication-usw” which is in us-west-1 region and ““annex-test-replication-euc” which is in EU central region. AWS cross-region replication will only allow the replication from source bucket to one destination bucket (either US-WEST or EU region, but will not allow both).

The following solution uses SNS and lambda function to overcome the limitation. On the source S3 bucket configuration, event notifications are configured for “All Object Creation” and “All Object Destroy” events to SNS topic which invokes a lambda function to perform replication to destination buckets in several regions by assuming cross-region replication role.

The following is a high-level design for this solution.

Steps to perform multi-region replication in AWS

1. Create an SNS topic called “Multi Region Replication Topic”

2. Create an S3 “Source” bucket and configure it to notify events of “All Object Creation” and “All Object Deletion” events; provide the SNS topic created above to send the notifications.

3. Create the S3 destination bucket in each destination region and setup appropriate bucket policies to control permissions.

4. In each destination account, create a new role: “CrossRegionReplicationRole” and provide access control and trust policies to allow write access to the lambda execution role in the source account.

5. Create a lambda function with subscription to the SNS topic: “Multi Region Replication Topic”. The lambda function will receive object creation and deletion event notifications from S3 and replicate the events in the corresponding destination buckets by assuming CrossRegionReplicationRole.

Below is Lambda function Code Sample:

6. Use Cloud Watch to monitor and perform logging.

That it! if you still have questions around any aspect of Multi-Region Replication, please don’t hesitate to reach out to us. We’d love to help you get started.