Log Amazon SageMaker API Calls with AWS CloudTrail
Amazon SageMaker is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or an AWS service in SageMaker. CloudTrail captures all API calls for SageMaker, with the exception of InvokeEndpoint and InvokeEndpointAsync, as events. The calls captured include calls from the SageMaker console and code calls to the SageMaker API operations. If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for SageMaker. If you don't configure a trail, you can still view the most recent events in the CloudTrail console in Event history. Using the information collected by CloudTrail, you can determine the request that was made to SageMaker, the IP address from which the request was made, who made the request, when it was made, and additional details.
To learn more about CloudTrail, see the AWS CloudTrail User Guide.
By default, log data is stored in CloudWatch Logs indefinitely. However, you can configure how long to store log data in a log group. For information, see Change Log Data Retention in CloudWatch Logs in the Amazon CloudWatch Logs User Guide.
For security purposes, you can monitor AWS CloudTrail logs to identify abnormal user activity. For more information about monitoring logs, see Logging and Monitoring.
SageMaker Information in CloudTrail
CloudTrail is enabled on your AWS account when you create the account. When activity occurs in Amazon SageMaker, that activity is recorded in a CloudTrail event along with other AWS service events in Event history. You can view, search, and download recent events in your AWS account. For more information, see Viewing Events with CloudTrail Event History.
For an ongoing record of events in your AWS account, including events for Amazon SageMaker, create a trail. A trail enables CloudTrail to deliver log files to an Amazon S3 bucket. By default, when you create a trail in the console, the trail applies to all AWS Regions. The trail logs events from all Regions in the AWS partition and delivers the log files to the Amazon S3 bucket that you specify. Additionally, you can configure other AWS services to further analyze and act upon the event data collected in CloudTrail logs. For more information, see the following:
All SageMaker actions, with the exception of
InvokeEndpoint and
InvokeEndpointAsync, are logged by CloudTrail and are documented in the Operations. For example, calls to the CreateTrainingJob,
CreateEndpoint and CreateNotebookInstance actions generate
entries in the CloudTrail log files.
Every event or log entry contains information about who generated the request. The identity information helps you determine the following:
-
Whether the request was made with root or IAM user credentials.
-
Whether the request was made with temporary security credentials for a role or federated user.
-
Whether the request was made by another AWS service.
For more information, see the CloudTrail userIdentity Element.
Operations Performed by Automatic Model Tuning
SageMaker supports logging non-API service events to your CloudTrail log files for automatic model
tuning jobs. These events are related to your tuning jobs but, are not the direct result of
a customer request to the public AWS API. For example, when you create a hyperparameter
tuning job by calling CreateHyperParameterTuningJob, SageMaker creates training jobs to evaluate
various combinations of hyperparameters to find the best result. Similarly, when you call
StopHyperParameterTuningJob to stop a hyperparameter tuning job, SageMaker
might stop any of the associated running training jobs. Non-API events for your tuning jobs
are logged to CloudTrail to help you improve governance, compliance, and operational and risk
auditing of your AWS account.
Log entries that result from non-API service events have an eventType of
AwsServiceEvent instead of AwsApiCall.
Understanding SageMaker Log File Entries
A trail is a configuration that enables delivery of events as log files to an S3 bucket that you specify. CloudTrail log files contain one or more log entries. An event represents a single request from any source and includes information about the requested action, the date and time of the action, request parameters, and so on. CloudTrail log files are not an ordered stack trace of the public API calls, so they do not appear in any specific order.
The following examples a log entry for the CreateEndpoint action, which
creates an endpoint to deploy a trained model.
{ "eventVersion":"1.05", "userIdentity": { "type":"IAMUser", "principalId":"AIXDAYQEXAMPLEUMLYNGL", "arn":"arn:aws:iam::123456789012:user/intern", "accountId":"123456789012", "accessKeyId":"ASXIAGXEXAMPLEQULKNXV", "userName":"intern" }, "eventTime":"2018-01-02T13:39:06Z", "eventSource":"sagemaker.amazonaws.com", "eventName":"CreateEndpoint", "awsRegion":"us-west-2", "sourceIPAddress":"127.0.0.1", "userAgent":"USER_AGENT", "requestParameters": { "endpointName":"ExampleEndpoint", "endpointConfigName":"ExampleEndpointConfig" }, "responseElements": { "endpointArn":"arn:aws:sagemaker:us-west-2:123456789012:endpoint/exampleendpoint" }, "requestID":"6b1b42b9-EXAMPLE", "eventID":"a6f85b21-EXAMPLE", "eventType":"AwsApiCall", "recipientAccountId":"444455556666" }
The following example is a log entry for the CreateModel action, which
creates one or more containers to host a previously trained model.
{ "eventVersion":"1.05", "userIdentity": { "type":"IAMUser", "principalId":"AIXDAYQEXAMPLEUMLYNGL", "arn":"arn:aws:iam::123456789012:user/intern", "accountId":"123456789012", "accessKeyId":"ASXIAGXEXAMPLEQULKNXV", "userName":"intern" }, "eventTime":"2018-01-02T15:23:46Z", "eventSource":"sagemaker.amazonaws.com", "eventName":"CreateModel", "awsRegion":"us-west-2", "sourceIPAddress":"127.0.0.1", "userAgent":"USER_AGENT", "requestParameters": { "modelName":"ExampleModel", "primaryContainer": { "image":"174872318107.dkr.ecr.us-west-2.amazonaws.com/kmeans:latest" }, "executionRoleArn":"arn:aws:iam::123456789012:role/EXAMPLEARN" }, "responseElements": { "modelArn":"arn:aws:sagemaker:us-west-2:123456789012:model/barkinghappy2018-01-02t15-23-32-275z-ivrdog" }, "requestID":"417b8dab-EXAMPLE", "eventID":"0f2b3e81-EXAMPLE", "eventType":"AwsApiCall", "recipientAccountId":"444455556666" }
