Working with AWS CloudTrail Lake - AWS CloudTrail

Working with AWS CloudTrail Lake

AWS CloudTrail Lake lets you run SQL-based queries on your events. CloudTrail Lake converts existing events in row-based JSON format to Apache ORC format. ORC is a columnar storage format that is optimized for fast retrieval of data. Events are aggregated into event data stores, which are immutable collections of events based on criteria that you select by applying advanced event selectors. You can keep the event data in an event data store for up to seven years, or 2557 days. By default, event data is retained for the maximum period, 2557 days. The selectors that you apply to an event data store control which events persist and are available for you to query. CloudTrail Lake is an auditing solution that can complement your compliance stack, and assist you with near real-time troubleshooting.

When you create an event data store, you choose the category of AWS events to include in your event data store. You can create an event data store to include CloudTrail events, or AWS Config configuration items. Each event data store can only contain a single event category (for example, AWS Config configuration items), because the event schema for each event category is unique. You can run SQL queries across multiple event data stores using the supported SQL JOIN keywords. For information about running queries across multiple event data stores, see Advanced, multi-table query support.

CloudTrail Lake queries offer a deeper and more customizable view of events than simple key and value lookups in Event history, or running LookupEvents. An Event history search is limited to a single AWS account, only returns events from a single AWS Region, and cannot query multiple attributes. In contrast, CloudTrail Lake users can run complex Standard Query Language (SQL) queries across multiple fields in a CloudTrail event or configuration item. For a full list of supported SQL operators, see CloudTrail Lake SQL constraints.

You can save CloudTrail Lake queries for future use, and view results of queries for up to seven days. When you run queries, you can save the query results to an Amazon S3 bucket. CloudTrail Lake can also store events from an organization in AWS Organizations in an event data store, including events from multiple Regions and accounts.

CloudTrail doesn't support authorization based on tags for trails. However, you can control access to actions on event data stores by using authorization based on tags. For more information and examples, see Examples: Denying access to create or delete event data stores based on tags in this guide.

By default, all events in an event data store are encrypted by CloudTrail. When you configure an event data store, you can choose to use your own AWS Key Management Service key. Using your own KMS key incurs AWS KMS costs for encryption and decryption. After you associate an event data store with a KMS key, the KMS key cannot be removed or changed.

CloudTrail Lake event data stores and queries incur CloudTrail charges. For more information about CloudTrail Lake pricing, see AWS CloudTrail Pricing.

CloudTrail Lake supports Amazon CloudWatch metrics, which you can use to view information about the amount of data ingested into your event data store during the last hour and over the course of its retention period. For more information about supported CloudWatch metrics, see Supported CloudWatch metrics.

Note

CloudTrail typically delivers logs within an average of about 15 minutes of an API call. This time is not guaranteed.

Topics