Analyzing log data with CloudWatch Logs Insights

With CloudWatch Logs Insights, you can interactively search and analyze your log data in Amazon CloudWatch Logs. You can perform queries to help you more efficiently and effectively respond to operational issues. If an issue occurs, you can use CloudWatch Logs Insights to identify potential causes and validate deployed fixes.

CloudWatch Logs Insights includes a purpose-built query language with a few simple but powerful commands. CloudWatch Logs Insights provides sample queries, command descriptions, query autocompletion, and log field discovery to help you get started. Sample queries are included for several types of AWS service logs.

CloudWatch Logs Insights automatically discovers fields in logs from AWS services such as Amazon Route 53, AWS Lambda, AWS CloudTrail, and Amazon VPC, and any application or custom log that emits log events as JSON.

You can use CloudWatch Logs Insights to search log data that was sent to CloudWatch Logs on November 5, 2018 or later.

Important

CloudWatch Logs Insights can't access log events with timestamps that pre-date the creation time of the log group.

You can also use natural language to create CloudWatch Logs Insights queries. To do so, ask questions about or describe the data you're looking for. This AI-assisted capability generates a query based on your prompt and provides a line-by-line explanation of how the query works. For more information, see Use natural language to generate and update CloudWatch Logs Insights queries.

If you are signed in to an account set up as a monitoring account in CloudWatch cross-account observability, you can run CloudWatch Logs Insights queries on log groups in source accounts linked to this monitoring account. You can run a query that queries multiple log groups located in different accounts. For more information, see CloudWatch cross-account observability.

A single request can query up to 50 log groups. Queries time out after 60 minutes, if they have not completed. Query results are available for 7 days.

You can save queries that you have created. This can help you run complex queries when you need, without having to re-create them each time that you want to run them.

CloudWatch Logs Insights queries incur charges based on the amount of data that is queried. For more information, see Amazon CloudWatch Pricing.

Important

If your network security team doesn't allow the use of web sockets, you can't currently access the CloudWatch Logs Insights portion of the CloudWatch console. You can use the CloudWatch Logs Insights query capabilities using APIs. For more information, see StartQuery in the Amazon CloudWatch Logs API Reference.

This chapter describes how to analyze your log data with CloudWatch Logs Insights.

Contents

• Commands supported in log classes
• Get started: Query tutorials
• Supported logs and discovered fields
• CloudWatch Logs Insights query syntax
• Create field indexes to improve query performance and reduce scan volume
• Pattern analysis
• Compare (diff) with previous time ranges
• Sample queries
• Visualize log data in graphs
• Save and re-run CloudWatch Logs Insights queries
• Add query to dashboard or export query results
• View running queries or query history
• Encrypt query results with AWS Key Management Service
• Use natural language to generate and update CloudWatch Logs Insights queries

Commands supported in log classes

All CloudWatch Logs Insights query commands are supported on log groups in the Standard log class. Log groups in the Infrequent Access log class support all query commands except pattern, diff, unmask and filterIndex.