Centralized logging - Landing Zone Accelerator on AWS

Centralized logging

AWS log archiving architecture with EventBridge, Lambda, Kinesis, and S3 components.

Landing Zone Accelerator on AWS architecture – centralized logging

  1. A CloudWatch log group update workflow runs during the Logging stage of the pipeline. A CloudFormation custom resource invokes a Lambda function that updates existing log groups to the increase log retention if it's less than the solution log retention period, CloudWatch AWS KMS key, and subscription filter. The destination for the subscription filter is an Amazon Kinesis Data Stream deployed to the Log Archive account.

  2. An EventBridge rule monitors for new CloudWatch log groups created in core and workload accounts.

  3. When new log groups are created, the EventBridge rule invokes a Lambda function that updates the log group with the configured log retention period, CloudWatch AWS KMS key, and subscription filter. The destination for the subscription filter is the Kinesis Data Stream deployed to the Log Archive account.

  4. Log groups stream their logs to the Kinesis Data Stream. The data stream is encrypted at rest with the replication AWS KMS key.

  5. A delivery stream is configured with the Kinesis Data Stream and Firehose, allowing the logs to be transformed and replicated to Amazon S3. 

  6. The destination of the Firehose delivery stream is the aws-accelerator-central-logs Amazon S3 bucket. This bucket is encrypted at rest with the central logging AWS KMS key. In addition, the aws-accelerator-s3-access-logs and aws-accelerator-elb-access-logs buckets are encrypted at rest with Amazon S3-managed server-side encryption (SSE-S3) because these services don't support customer-managed AWS KMS keys. Logs delivered to the aws-accelerator-elb-access-logs bucket replicate to the central logs bucket with Amazon S3 replication.