AWS Service Logs - Centralized Logging with OpenSearch

AWS Service Logs

Centralized Logging with OpenSearch supports ingesting AWS service logs into Amazon OpenSearch Service through log analytics pipelines, which you can build using the Centralized Logging with OpenSearch web console or via a standalone CloudFormation template.

Centralized Logging with OpenSearch reads the data source, parse, cleanup/enrich and ingest logs into Amazon OpenSearch Service domains for analysis. Moreover, the solution provides templated dashboards to facilitate log visualization.


AWS managed services must be in the same region as Centralized Logging with OpenSearch. To ingest logs from different AWS regions, we recommend using S3 cross-region replication. The solution will rotate the index on a daily basis, and cannot be adjusted.

Supported AWS Services

Most of AWS managed services output logs to Amazon CloudWatch Logs, Amazon S3, Amazon Kinesis Data Streams or Amazon Kinesis Firehose.

The following table lists the supported AWS services and the corresponding features.

AWS Service Log Type Log Location Automatic Ingestion Built-in Dashboard
Amazon CloudTrail N/A S3 Yes Yes
Amazon S3 Access logs S3 Yes Yes
Amazon RDS/Aurora MySQL Logs CloudWatch Logs Yes Yes
Amazon CloudFront Standard access logs S3 Yes Yes
Application Load Balancer Access logs S3 Yes Yes
AWS WAF Web ACL logs S3 Yes Yes
AWS Lambda N/A CloudWatch Logs Yes Yes
Amazon VPC Flow logs S3 Yes Yes
AWS Config N/A S3 Yes Yes
  • Automatic Ingestion: The solution detects the log location of the resource automatically and then reads the logs.

  • Built-in Dashboard: An out-of-box dashboard for the specified AWS service. The solution will automatically ingest a dashboard into the Amazon OpenSearch Service.

Most of supported AWS services in Centralized Logging with OpenSearch offers built-in dashboard when creating the log analytics pipelines. You go to the OpenSearch Dashboards to view the dashboards after the pipeline being provisioned.

In this chapter, you will learn how to create log ingestion and dashboards for the following AWS services:

Cross-Region Logging

When you deploy Centralized Logging with OpenSearch in one Region, the solution allows you to process service logs from another Region.


For Amazon RDS/Aurora and AWS Lambda service logs, this feature is not supported.

The Region where the service resides is referred to as the Source Region. The Region where the Centralized Logging with OpenSearch console is deployed is referred to as the Logging Region.

For Amazon CloudTrail, you can create a new trail which send logs into a S3 bucket in the Logging Region. To learn how to create a new trail, please refer to Creating a trail.

For other services with logs located in S3 buckets, you can manually transfer logs (for example, using S3 Cross-Region Replication feature) to the Logging Region S3 bucket.

Follow the steps below to implement Cross-Region Logging:

  1. Set the service log location in another Region to be the Logging Region (such as AWS WAF), or automatically copy logs from the Source Region to the Logging Region using CRR.

  2. In the solution console, choose AWS Service Log in the left navigation pane. Then choose Create a pipeline.

  3. In the Select an AWS Service area, choose a service in the list. Choose Next.

  4. In Creation Method, choose Manual, then enter the resource name and S3 log location parameter, and choose Next.

  5. Set OpenSearch domain and Log Lifecycle as needed, and choose Next.

  6. Add tags if you need, and choose Next to create the pipeline.

Then you can use the OpenSearch dashboard to discover logs and view dashboards.