Create an event data store for CloudTrail events with the console
Event data stores for CloudTrail events can include CloudTrail management events, data events, and network activity events. You can keep the event data in an event data store for up to 3,653 days (about 10 years) if you choose the One-year extendable retention pricing option, or up to 2,557 days (about 7 years) if you choose the Seven-year retention pricing option..
Note
Network activity events is in preview release for CloudTrail and is subject to change.
CloudTrail Lake event data stores incur charges. When you create an event data store, you choose the pricing option you want
to use for the event data store. The pricing option determines the cost for ingesting and storing events, and
the default and maximum retention period for the event data store. For information
about CloudTrail pricing and managing Lake costs, see
AWS CloudTrail Pricing
To create an event data store for CloudTrail events
Use this procedure to create an event data store that logs CloudTrail management events, data events, or network activity events.
Sign in to the AWS Management Console and open the CloudTrail console at https://console.aws.amazon.com/cloudtrail/
. -
From the navigation pane, under Lake, choose Event data stores.
-
Choose Create event data store.
-
On the Configure event data store page, in General details, enter a name for the event data store. A name is required.
-
Choose the Pricing option that you want to use for your event data store. The pricing option determines the cost for ingesting and storing events, and the default and maximum retention periods for your event data store. For more information, see AWS CloudTrail Pricing
and Managing CloudTrail Lake costs. The following are the available options:
-
One-year extendable retention pricing - Generally recommended if you expect to ingest less than 25 TB of event data per month and want a flexible retention period of up to 10 years. For the first 366 days (the default retention period), storage is included at no additional charge with ingestion pricing. After 366 days, extended retention is available at pay-as-you-go pricing. This is the default option.
-
Default retention period: 366 days
-
Maximum retention period: 3,653 days
-
-
Seven-year retention pricing - Recommended if you expect to ingest more than 25 TB of event data per month and need a retention period of up to 7 years. Retention is included with ingestion pricing at no additional charge.
-
Default retention period: 2,557 days
-
Maximum retention period: 2,557 days
-
-
-
Specify a retention period for the event data store. Retention periods can be between 7 days and 3,653 days (about 10 years) for the One-year extendable retention pricing option, or between 7 days and 2,557 days (about seven years) for the Seven-year retention pricing option.
CloudTrail Lake determines whether to retain an event by checking if the
eventTime
of the event is within the specified retention period. For example, if you specify a retention period of 90 days, CloudTrail will remove events when theireventTime
is older than 90 days.Note
If you are copying trail events to this event data store, CloudTrail will not copy an event if its
eventTime
is older than the specified retention period. To determine the appropriate retention period, take the sum of the oldest event you want to copy in days and the number of days you want to retain the events in the event data store (retention period =oldest-event-in-days
+number-days-to-retain
). For example, if the oldest event you're copying is 45 days old and you want to keep the events in the event data store for a further 45 days, you would set the retention period to 90 days. -
(Optional) To enable encryption using AWS Key Management Service, choose Use my own AWS KMS key. Choose New to have an AWS KMS key created for you, or choose Existing to use an existing KMS key. In Enter KMS alias, specify an alias, in the format
alias/
MyAliasName
. Using your own KMS key requires that you edit your KMS key policy to allow CloudTrail logs to be encrypted and decrypted. For more information, see Configure AWS KMS key policies for CloudTrail. CloudTrail also supports AWS KMS multi-Region keys. For more information about multi-Region keys, see Using multi-Region keys in the AWS Key Management Service Developer Guide.Using your own KMS key incurs AWS KMS costs for encryption and decryption. After you associate an event data store with a KMS key, the KMS key cannot be removed or changed.
Note
To enable AWS Key Management Service encryption for an organization event data store, you must use an existing KMS key for the management account.
-
(Optional) If you want to query against your event data using Amazon Athena, choose Enable in Lake query federation. Federation lets you view the metadata associated with the event data store in the AWS Glue Data Catalog and run SQL queries against the event data in Athena. The table metadata stored in the AWS Glue Data Catalog lets the Athena query engine know how to find, read, and process the data that you want to query. For more information, see Federate an event data store.
To enable Lake query federation, choose Enable and then do the following:
-
Choose whether you want to create a new role or use an existing IAM role. AWS Lake Formation uses this role to manage permissions for the federated event data store. When you create a new role using the CloudTrail console, CloudTrail automatically creates a role with the required permissions. If you choose an existing role, be sure the policy for the role provides the required minimum permissions.
-
If you are creating a new role, enter a name to identify the role.
-
If you are using an existing role, choose the role you want to use. The role must exist in your account.
-
-
(Optional) In the Tags section, you can add up to 50 tag key pairs to help you identify, sort, and control access to your event data store. For more information about how to use IAM policies to authorize access to an event data store based on tags, see Examples: Denying access to create or delete event data stores based on tags. For more information about how you can use tags in AWS, see Tagging AWS resources in the Tagging AWS Resources User Guide.
-
Choose Next to configure the event data store.
-
On the Choose events page, choose AWS events, and then choose CloudTrail events.
-
For CloudTrail events, choose at least one event type. By default, Management events is selected. You can add management events, data events, and network activity events to your event data store.
-
(Optional) Choose Copy trail events if you want to copy events from an existing trail to run queries on past events. To copy trail events to an organization event data store, you must use the management account for the organization. The delegated administrator account cannot copy trail events to an organization event data store. For more information about considerations for copying trail events, see Considerations for copying trail events.
-
To have your event data store collect events from all accounts in an AWS Organizations organization, select Enable for all accounts in my organization. You must be signed in to the management account or delegated administrator account for the organization to create an event data store that collects events for an organization.
Note
To copy trail events or enable Insights events, you must be signed in to the management account for your organization.
-
Expand Additional settings to choose whether you want your event data store to collect events for all AWS Regions, or only the current AWS Region, and choose whether the event data store ingests events. By default, your event data store collects events from all Regions in your account and starts ingesting events when it's created.
-
Select Include only the current region in my event data store to include only events that are logged in the current Region. If you do not choose this option, your event data store includes events from all Regions.
-
Deselect Ingest events if you do not want the event data store to start ingesting events. For example, you may want to deselect Ingest events, if you are copying trail events and do not want the event data store to include any future events. By default, the event data store starts ingesting events when it's created.
-
-
If your event data store includes management events, you can choose from the following options. For more information about management events, see Logging management events.
-
Choose whether you want to include Read events, Write events, or both. At least one is required.
-
Choose whether to exclude AWS Key Management Service or Amazon RDS Data API events from your event data store.
-
Choose whether to enable Insights. To enable Insights, you need to set up a destination event data store to collect Insights events based upon the management event activity in this event data store.
If you choose to enable Insights, do the following.
-
In Enable Insights, choose the destination event store that will log Insights events. The destination event data store will collect Insights events based upon the management event activity in this event data store. For information about how to create the destination event data store, see To create a destination event data store that logs Insights events.
-
Choose the Insights types. You can choose API call rate, API error rate, or both. You must be logging Write management events to log Insights events for API call rate. You must be logging Read or Write management events to log Insights events for API error rate.
-
-
-
To include data events in your event data store, do the following.
-
Choose a data event type. This is the AWS service and resource on which data events are logged. To log data events for AWS Glue tables created by Lake Formation, choose Lake Formation for the data type.
-
In Log selector template, choose a template. You can choose to log all data events,
readOnly
events,writeOnly
events, or Custom to build a custom log selector. -
(Optional) In Selector name, enter a name to identify your selector. The selector name is a descriptive name for an advanced event selector, such as "Log data events for only two S3 buckets". The selector name is listed as
Name
in the advanced event selector and is viewable if you expand the JSON view. -
In Advanced event selectors, build an expression for the specific resources on which you want to log data events. You can skip this step if you are using a predefined log template.
-
Choose from the following fields.
-
readOnly
-readOnly
can be set to equals a value oftrue
orfalse
. Read-only data events are events that do not change the state of a resource, such asGet*
orDescribe*
events. Write events add, change, or delete resources, attributes, or artifacts, such asPut*
,Delete*
, orWrite*
events. To log bothread
andwrite
events, don't add areadOnly
selector. -
eventName
-eventName
can use any operator. You can use it to include or exclude any data event logged to CloudTrail, such asPutBucket
,GetItem
, orGetSnapshotBlock
. -
resources.ARN
- You can use any operator withresources.ARN
, but if you use equals or does not equal, the value must exactly match the ARN of a valid resource of the type you've specified in the template as the value ofresources.type
.The following table shows the valid ARN format for each
resources.type
.Note
You can't use the
resources.ARN
field to filter resource types that do not have ARNs.resources.type resources.ARN AWS::DynamoDB::Table1
arn:
partition
:dynamodb:region
:account_ID
:table/table_name
AWS::Lambda::Function
arn:
partition
:lambda:region
:account_ID
:function:function_name
AWS::S3::Object
2arn:
partition
:s3:::amzn-s3-demo-bucket
/ arn:partition
:s3:::amzn-s3-demo-bucket
/object_or_file_name
/AWS::AppConfig::Configuration
arn:
partition
:appconfig:region
:account_ID
:application/application_ID
/environment/environment_ID
/configuration/configuration_profile_ID
AWS::B2BI::Transformer
arn:
partition
:b2bi:region
:account_ID
:transformer/transformer_ID
AWS::Bedrock::AgentAlias
arn:
partition
:bedrock:region
:account_ID
:agent-alias/agent_ID
/alias_ID
AWS::Bedrock::FlowAlias
arn:
partition
:bedrock:region
:account_ID
:flow/flow_ID
/alias/alias_ID
AWS::Bedrock::Guardrail
arn:
partition
:bedrock:region
:account_ID
:guardrail/guardrail_ID
AWS::Bedrock::KnowledgeBase
arn:
partition
:bedrock:region
:account_ID
:knowledge-base/knowledge_base_ID
AWS::Cassandra::Table
arn:
partition
:cassandra:region
:account_ID
:keyspace/keyspace_name
/table/table_name
AWS::CloudFront::KeyValueStore
arn:
partition
:cloudfront:region
:account_ID
:key-value-store/KVS_name
AWS::CloudTrail::Channel
arn:
partition
:cloudtrail:region
:account_ID
:channel/channel_UUID
AWS::CodeWhisperer::Customization
arn:
partition
:codewhisperer:region
:account_ID
:customization/customization_ID
AWS::CodeWhisperer::Profile
arn:
partition
:codewhisperer:region
:account_ID
:profile/profile_ID
AWS::Cognito::IdentityPool
arn:
partition
:cognito-identity:region
:account_ID
:identitypool/identity_pool_ID
AWS::DynamoDB::Stream
arn:
partition
:dynamodb:region
:account_ID
:table/table_name
/stream/date_time
AWS::EC2::Snapshot
arn:
partition
:ec2:region
::snapshot/snapshot_ID
AWS::EMRWAL::Workspace
arn:
partition
:emrwal:region
:account_ID
:workspace/workspace_name
AWS::FinSpace::Environment
arn:
partition
:finspace:region
:account_ID
:environment/environment_ID
AWS::Glue::Table
arn:
partition
:glue:region
:account_ID
:table/database_name
/table_name
AWS::GreengrassV2::ComponentVersion
arn:
partition
:greengrass:region
:account_ID
:components/component_name
AWS::GreengrassV2::Deployment
arn:
partition
:greengrass:region
:account_ID
:deployments/deployment_ID
AWS::GuardDuty::Detector
arn:
partition
:guardduty:region
:account_ID
:detector/detector_ID
AWS::IoT::Certificate
arn:
partition
:iot:region
:account_ID
:cert/certificate_ID
AWS::IoT::Thing
arn:
partition
:iot:region
:account_ID
:thing/thing_ID
AWS::IoTSiteWise::Asset
arn:
partition
:iotsitewise:region
:account_ID
:asset/asset_ID
AWS::IoTSiteWise::TimeSeries
arn:
partition
:iotsitewise:region
:account_ID
:timeseries/timeseries_ID
AWS::IoTTwinMaker::Entity
arn:
partition
:iottwinmaker:region
:account_ID
:workspace/workspace_ID
/entity/entity_ID
AWS::IoTTwinMaker::Workspace
arn:
partition
:iottwinmaker:region
:account_ID
:workspace/workspace_ID
AWS::KendraRanking::ExecutionPlan
arn:
partition
:kendra-ranking:region
:account_ID
:rescore-execution-plan/rescore_execution_plan_ID
AWS::Kinesis::Stream
arn:
partition
:kinesis:region
:account_ID
:stream/stream_name
AWS::Kinesis::StreamConsumer
arn:
partition
:kinesis:region
:account_ID
:stream_type
/stream_name
/consumer/consumer_name
:consumer_creation_timestamp
AWS::KinesisVideo::Stream
arn:
partition
:kinesisvideo:region
:account_ID
:stream/stream_name
/creation_time
AWS::MachineLearning::MlModel
arn:
partition
:machinelearning:region
:account_ID
:mlmodel/model_ID
AWS::ManagedBlockchain::Network
arn:
partition
:managedblockchain:::networks/network_name
AWS::ManagedBlockchain::Node
arn:
partition
:managedblockchain:region
:account_ID
:nodes/node_ID
AWS::MedicalImaging::Datastore
arn:
partition
:medical-imaging:region
:account_ID
:datastore/data_store_ID
AWS::NeptuneGraph::Graph
arn:
partition
:neptune-graph:region
:account_ID
:graph/graph_ID
AWS::One::UKey
arn:
partition
:one:region
:account_ID
:user/user_ID
/u-key/u-key_ID
AWS::One::User
arn:
partition
:one:region
:account_ID
:user/user_ID
AWS::PaymentCryptography::Alias
arn:
partition
:payment-cryptography:region
:account_ID
:alias/alias
AWS::PaymentCryptography::Key
arn:
partition
:payment-cryptography:region
:account_ID
:key/key_ID
AWS::PCAConnectorAD::Connector
arn:
partition
:pca-connector-ad:region
:account_ID
:connector/connector_ID
AWS::PCAConnectorSCEP::Connector
arn:
partition
:pca-connector-scep:region
:account_ID
:connector/connector_ID
AWS::QApps:QApp
arn:
partition
:qapps:region
:account_ID
:application/application_UUID
/qapp/qapp_UUID
AWS::QBusiness::Application
arn:
partition
:qbusiness:region
:account_ID
:application/application_ID
AWS::QBusiness::DataSource
arn:
partition
:qbusiness:region
:account_ID
:application/application_ID
/index/index_ID
/data-source/datasource_ID
AWS::QBusiness::Index
arn:
partition
:qbusiness:region
:account_ID
:application/application_ID
/index/index_ID
AWS::QBusiness::WebExperience
arn:
partition
:qbusiness:region
:account_ID
:application/application_ID
/web-experience/web_experienc_ID
AWS::RDS::DBCluster
arn:
partition
:rds:region
:account_ID
:cluster/cluster_name
AWS::RUM::AppMonitor
arn:
partition
:rum:region
:account_ID
:appmonitor/app_monitor_name
AWS::S3::AccessPoint
3arn:
partition
:s3:region
:account_ID
:accesspoint/access_point_name
AWS::S3Express::Object
arn:
partition
:s3express:region
:account_ID
:bucket/bucket_name
AWS::S3ObjectLambda::AccessPoint
arn:
partition
:s3-object-lambda:region
:account_ID
:accesspoint/access_point_name
AWS::S3Outposts::Object
arn:
partition
:s3-outposts:region
:account_ID
:object_path
AWS::SageMaker::Endpoint
arn:
partition
:sagemaker:region
:account_ID
:endpoint/endpoint_name
AWS::SageMaker::ExperimentTrialComponent
arn:
partition
:sagemaker:region
:account_ID
:experiment-trial-component/experiment_trial_component_name
AWS::SageMaker::FeatureGroup
arn:
partition
:sagemaker:region
:account_ID
:feature-group/feature_group_name
AWS::SCN::Instance
arn:
partition
:scn:region
:account_ID
:instance/instance_ID
AWS::ServiceDiscovery::Namespace
arn:
partition
:servicediscovery:region
:account_ID
:namespace/namespace_ID
AWS::ServiceDiscovery::Service
arn:
partition
:servicediscovery:region
:account_ID
:service/service_ID
AWS::SNS::PlatformEndpoint
arn:
partition
:sns:region
:account_ID
:endpoint/endpoint_type
/endpoint_name
/endpoint_ID
AWS::SNS::Topic
arn:
partition
:sns:region
:account_ID
:topic_name
AWS::SQS::Queue
arn:
partition
:sqs:region
:account_ID
:queue_name
AWS::SSM::ManagedNode
The ARN must be in one of the following formats:
-
arn:
partition
:ssm:region
:account_ID
:managed-instance/instance_ID
-
arn:
partition
:ec2:region
:account_ID
:instance/instance_ID
AWS::SSMMessages::ControlChannel
arn:
partition
:ssmmessages:region
:account_ID
:control-channel/control_channel_ID
AWS::StepFunctions::StateMachine
The ARN must be in one of the following formats:
-
arn:
partition
:states:region
:account_ID
:stateMachine:stateMachine_name
-
arn:
partition
:states:region
:account_ID
:stateMachine:stateMachine_name
/label_name
AWS::SWF::Domain
arn:
partition
:swf:region
:account_ID
:/domain/domain_name
AWS::ThinClient::Device
arn:
partition
:thinclient:region
:account_ID
:device/device_ID
AWS::ThinClient::Environment
arn:
partition
:thinclient:region
:account_ID
:environment/environment_ID
AWS::Timestream::Database
arn:
partition
:timestream:region
:account_ID
:database/database_name
AWS::Timestream::Table
arn:
partition
:timestream:region
:account_ID
:database/database_name
/table/table_name
AWS::VerifiedPermissions::PolicyStore
arn:
partition
:verifiedpermissions:region
:account_ID
:policy-store/policy_store_ID
1 For tables with streams enabled, the
resources
field in the data event contains bothAWS::DynamoDB::Stream
andAWS::DynamoDB::Table
. If you specifyAWS::DynamoDB::Table
for theresources.type
, it will log both DynamoDB table and DynamoDB streams events by default. To exclude streams events, add a filter on theeventName
field.2 To log all data events for all objects in a specific S3 bucket, use the
StartsWith
operator, and include only the bucket ARN as the matching value. The trailing slash is intentional; do not exclude it.3 To log events on all objects in an S3 access point, we recommend that you use only the access point ARN, don’t include the object path, and use the
StartsWith
orNotStartsWith
operators. -
For more information about the ARN formats of data event resources, see Actions, resources, and condition keys in the AWS Identity and Access Management User Guide.
-
-
For each field, choose + Condition to add as many conditions as you need, up to a maximum of 500 specified values for all conditions. For example, to exclude data events for two S3 buckets from data events that are logged on your event data store, you can set the field to resources.ARN, set the operator for does not start with, and then either paste in an S3 bucket ARN, or browse for the S3 buckets for which you do not want to log events.
To add the second S3 bucket, choose + Condition, and then repeat the preceding instruction, pasting in the ARN for or browsing for a different bucket.
For information about how CloudTrail evaluates multiple conditions, see How CloudTrail evaluates multiple conditions for a field.
Note
You can have a maximum of 500 values for all selectors on an event data store. This includes arrays of multiple values for a selector such as
eventName
. If you have single values for all selectors, you can have a maximum of 500 conditions added to a selector. -
Choose + Field to add additional fields as required. To avoid errors, do not set conflicting or duplicate values for fields. For example, do not specify an ARN in one selector to be equal to a value, then specify that the ARN not equal the same value in another selector.
-
-
Optionally, expand JSON view to see your advanced event selectors as a JSON block.
-
To add another data type on which to log data events, choose Add data event type. Repeat steps a through this step to configure advanced event selectors for the data event type.
-
-
To include network activity events in your event data store, do the following.
-
From Network activity event source, choose the source for network activity events.
-
In Log selector template, choose a template. You can choose to log all network activity events, log all network activity access denied events, or choose Custom to build a custom log selector to filter on multiple fields, such as
eventName
andvpcEndpointId
. -
(Optional) Enter a name to identify the selector. The selector name is listed as Name in the advanced event selector and is viewable if you expand the JSON view.
-
In Advanced event selectors build expressions by choosing values for Field, Operator, and Value. You can skip this step if you are using a predefined log template.
-
For excluding or including network activity events, you can choose from the following fields in the console.
-
eventName
– You can use any operator witheventName
. You can use it to include or exclude any event, such asCreateKey
. -
errorCode
– You can use it to filter on an error code. Currently, the only supportederrorCode
isVpceAccessDenied
. -
vpcEndpointId
– Identifies the VPC endpoint that the operation passed through. You can use any operator withvpcEndpointId
.
-
-
For each field, choose + Condition to add as many conditions as you need, up to a maximum of 500 specified values for all conditions.
-
Choose + Field to add additional fields as required. To avoid errors, do not set conflicting or duplicate values for fields.
-
-
To add another event source for which you want to log network activity events, choose Add network activity event selector.
-
Optionally, expand JSON view to see your advanced event selectors as a JSON block.
-
-
To copy existing trail events to your event data store, do the following.
-
Choose the trail that you want to copy. By default, CloudTrail only copies CloudTrail events contained in the S3 bucket's
CloudTrail
prefix and the prefixes inside theCloudTrail
prefix, and does not check prefixes for other AWS services. If you want to copy CloudTrail events contained in another prefix, choose Enter S3 URI, and then choose Browse S3 to browse to the prefix. If the source S3 bucket for the trail uses a KMS key for data encryption, ensure that the KMS key policy allows CloudTrail to decrypt the data. If your source S3 bucket uses multiple KMS keys, you must update each key's policy to allow CloudTrail to decrypt the data in the bucket. For more information about updating the KMS key policy, see KMS key policy for decrypting data in the source S3 bucket. -
Choose the time range for copying the events. CloudTrail checks the prefix and log file name to verify the name contains a date between the chosen start and end date before attempting to copy trail events. You can choose a Relative range or an Absolute range. To avoid duplicating events between the source trail and destination event data store, choose a time range that is earlier than the creation of the event data store.
Note
CloudTrail only copies trail events that have an
eventTime
within the event data store’s retention period. For example, if an event data store’s retention period is 90 days, then CloudTrail will not copy any trail events with aneventTime
older than 90 days.If you choose Relative range, you can choose to copy events logged in the last 6 months, 1 year, 2 years, 7 years, or a custom range. CloudTrail copies the events logged within the chosen time period.
If you choose Absolute range, you can choose a specific start and end date. CloudTrail copies the events that occurred between the chosen start and end dates.
-
For Permissions, choose from the following IAM role options. If you choose an existing IAM role, verify that the IAM role policy provides the necessary permissions. For more information about updating the IAM role permissions, see IAM permissions for copying trail events.
-
Choose Create a new role (recommended) to create a new IAM role. For Enter IAM role name, enter a name for the role. CloudTrail automatically creates the necessary permissions for this new role.
-
Choose Use a custom IAM role ARN to use a custom IAM role that is not listed. For Enter IAM role ARN, enter the IAM ARN.
-
Choose an existing IAM role from the drop-down list.
-
-
-
Choose Next to review your choices.
-
On the Review and create page, review your choices. Choose Edit to make changes to a section. When you're ready to create the event data store, choose Create event data store.
-
The new event data store is visible in the Event data stores table on the Event data stores page.
From this point forward, the event data store captures events that match its advanced event selectors (if you kept the Ingest events option selected). Events that occurred before you created the event data store are not in the event data store, unless you opted to copy existing trail events.
You can now run queries on your new event data store. The Sample queries tab provides example queries to get you started. For more information about creating and editing queries, see Create or edit a query with the CloudTrail console.
You can also view the CloudTrail Lake dashboard to visualize the management and S3 data events in your event data store. For more information about Lake dashboards, see View CloudTrail Lake dashboards with the CloudTrail console.