Architecture Overview
Deploying this solution builds the following environment in the AWS Cloud.
Figure 1: Cross-Region Replication Monitor architecture on AWS
Monitor Template
This solution includes a primary AWS CloudFormation template that deploys all solution components to enable cross-region replication and monitoring in a single account. This include AWS Identity and Access Management (IAM) roles, AWS Lambda functions, an AWS CloudTrail trail, an Amazon CloudWatch event, an Amazon Simple Notification Service (Amazon SNS) topic, and an Amazon DynamoDB table. Also, the solution turns on AWS CloudTrail and automatically enables the data events for the source and destination buckets that have CRR enabled. Note that if you create an Amazon S3 bucket after deploying the solution, you can manually add it to AWS CloudTrail.
When an object is added to the Amazon S3 source bucket, AWS CloudTrail logs the data event. This activity triggers an Amazon CloudWatch event rule that delivers the status information to the monitoring accounts CloudWatch Logs, and sends the event to Amazon SNS. An Amazon Simple Queue Service (Amazon SQS) queue subscribed to the Amazon SNS topic receives the message for processing. Once the object replication to the destination bucket is successful, the successful replication triggers a similar event, and sends the status information back to the Amazon SQS queue.
Once the AWS Lambda function verifies an object was successfully replicated, it stores the data in an Amazon DynamoDB table for immediate access. Status data in the DynamoDB table is deleted and replaced every 24 hours.
Agent Template
The solution also includes a secondary AWS CloudFormation template that installs an AWS CloudTrail trail and an Amazon CloudWatch event rule to enable cross-region replication and monitoring across multiple accounts. Note that the solution cannot determine Amazon S3 bucket replication across account boundaries so you must configure AWS CloudTrail Data Events to match the desired buckets.
Figure 2: Cross-Region Replication Agent architecture on AWS
When an object is added to the Amazon S3 source bucket, AWS CloudTrail logs the data event. This activity triggers an Amazon CloudWatch event rule that delivers the status information to the CloudWatch Logs in the Monitor account using an event bus. In the Monitor account, CloudWatch Logs sends the event to Amazon SNS. An Amazon SQS queue subscribed to the Amazon SNS topic receives the message for processing. Once the object replication to the destination bucket is successful, the successful replication triggers a similar event, and sends the status information back to the Amazon SQS queue in the Monitor account.
Once the AWS Lambda function verifies an object was successfully replicated, it stores the data in an Amazon DynamoDB table for immediate access. Status data in the DynamoDB table is deleted and replaced every 24 hours.
Note
Customers who deploy this solution in an AWS Region that offers Amazon Kinesis Data Firehose can choose to archive solution data to Amazon S3. If you enable this feature, the solution uses a Firehose delivery stream to upload data to one of your existing S3 buckets for later analysis. You can use Amazon Athena, a serverless, interactive query service, to easily analyze historical data in Amazon S3.
