AWS supports Cross-Region Replication (CRR) to allow replication of contents in an S3 bucket into another bucket in a different region. This works perfectly well if you need to replicate contents of one bucket to another, but it does not allow replicating the contents of the source bucket to buckets in multiple regions.
Let’s take a sample scenario: we need to replicate contents of a source bucket “annex-test-replication-source” in us-east-1 region into “annex-test-replication-usw” which is in us-west-1 region and ““annex-test-replication-euc” which is in EU central region. AWS cross-region replication will only allow the replication from source bucket to one destination bucket (either US-WEST or EU region, but will not allow both).
The following solution uses SNS and lambda function to overcome the limitation. On the source S3 bucket configuration, event notifications are configured for “All Object Creation” and “All Object Destroy” events to SNS topic which invokes a lambda function to perform replication to destination buckets in several regions by assuming cross-region replication role.
The following is a high-level design for this solution.
2. Create an S3 “Source” bucket and configure it to notify events of “All Object Creation” and “All Object Deletion” events; provide the SNS topic created above to send the notifications.
3. Create the S3 destination bucket in each destination region and setup appropriate bucket policies to control permissions.
4. In each destination account, create a new role: “CrossRegionReplicationRole” and provide access control and trust policies to allow write access to the lambda execution role in the source account.
5. Create a lambda function with subscription to the SNS topic: “Multi Region Replication Topic”. The lambda function will receive object creation and deletion event notifications from S3 and replicate the events in the corresponding destination buckets by assuming CrossRegionReplicationRole.
Below is Lambda function Code Sample:
6. Use Cloud Watch to monitor and perform logging.
That it! if you still have questions around any aspect of Multi-Region Replication, please don’t hesitate to reach out to us. We’d love to help you get started.
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
Cookie | Duration | Description |
---|---|---|
cookielawinfo-checbox-analytics | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics". |
cookielawinfo-checbox-functional | 11 months | The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional". |
cookielawinfo-checbox-others | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other. |
cookielawinfo-checkbox-necessary | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary". |
cookielawinfo-checkbox-performance | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance". |
viewed_cookie_policy | 11 months | The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data. |
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.