Amazon Machine Learning
Developer Guide (Version Latest)

Logging Amazon ML API Calls by Using AWS CloudTrail

Amazon Machine Learning (Amazon ML) is integrated with AWS CloudTrail (CloudTrail), a service that captures specific API calls made by or on behalf of Amazon ML in your AWS account and delivers the log files to an S3 bucket that you specify. CloudTrail captures API calls made from the Amazon ML console or from the Amazon ML API. Using the information collected by CloudTrail, you can determine which request was made to Amazon ML, the IP address from which the request was made, who made the request, when it was made, and so on. To learn more about CloudTrail, including how to configure and enable it, see the AWS CloudTrail User Guide.

Amazon ML Information in CloudTrail

When CloudTrail logging is enabled in your AWS account, API calls made to certain Amazon ML operations are tracked in CloudTrail log files, where they are written with other AWS service records. CloudTrail determines when to create and write to a new file based on a time period and file size.

The following Amazon ML operations are supported for logging by CloudTrail :

The following Amazon ML operations use request parameters that contain credentials. Before these requests are sent to CloudTrail, the credentials are replaced with three asterisks ("***"):

When the following Amazon ML operations are performed with the Amazon ML console, the attribute ComputeStatistics is not included in the RequestParameters component of the CloudTrail log:

Every log entry contains information about who generated the request. The user identity information in the log helps you determine whether the request was made with root or AWS Identity and Access Management (IAM) user credentials, with temporary security credentials for a role or federated user, or by another AWS service. For more information, see the userIdentity field in the CloudTrail Event Reference.

You can store your log files in your S3 bucket for as long as you want, but you can also define Amazon Simple Storage Service (Amazon S3) lifecycle rules to archive or delete log files automatically. By default, your log files are encrypted by using Amazon S3 server-side encryption (SSE).

If you want to take quick action upon log file delivery, you can have CloudTrail publish Amazon Simple Notification Service (Amazon SNS) notifications when new log files are delivered. For more information, see Configuring Amazon SNS Notifications.

You can also aggregate Amazon ML log files from multiple AWS regions and multiple AWS accounts into a single S3 bucket. For more information, see Aggregating CloudTrail Log Files to a Single Amazon S3 Bucket.

Understanding Amazon ML Log File Entries

CloudTrail log files contain one or more log entries, where each entry lists multiple JSON-formatted events. A log entry represents a single request from any source and includes information about the requested operation, including the date and time of the operation, request parameters, and so on. Log entries are not an ordered stack trace of the public API calls, so they do not appear in any particular order.

The following example log file shows log entries for two API calls. The first log entry shows an API call to create a datasource, and the second shows an API call to create a batch prediction using the datasource from the previous call.

{ "Records": [ { "eventVersion": "1.03", "userIdentity": { "type": "IAMUser", "principalId": "EX_PRINCIPAL_ID", "arn": "arn:aws:iam::012345678910:user/Alice", "accountId": "012345678910", "accessKeyId": "EXAMPLE_KEY_ID", "userName": "Alice" }, "eventTime": "2015-11-12T15:04:02Z", "eventSource": "", "eventName": "CreateDataSourceFromS3", "awsRegion": "us-east-1", "sourceIPAddress": "", "userAgent": "", "requestParameters": { "data": { "dataLocationS3": "s3://aml-sample-data/banking-batch.csv", "dataSchema": "{\"version\":\"1.0\",\"rowId\":null,\"rowWeight\":null, \"targetAttributeName\":null,\"dataFormat\":\"CSV\", \"dataFileContainsHeader\":false,\"attributes\":[ {\"attributeName\":\"age\",\"attributeType\":\"NUMERIC\"}, {\"attributeName\":\"job\",\"attributeType\":\"CATEGORICAL\"}, {\"attributeName\":\"marital\",\"attributeType\":\"CATEGORICAL\"}, {\"attributeName\":\"education\",\"attributeType\":\"CATEGORICAL\"}, {\"attributeName\":\"default\",\"attributeType\":\"CATEGORICAL\"}, {\"attributeName\":\"housing\",\"attributeType\":\"CATEGORICAL\"}, {\"attributeName\":\"loan\",\"attributeType\":\"CATEGORICAL\"}, {\"attributeName\":\"contact\",\"attributeType\":\"CATEGORICAL\"}, {\"attributeName\":\"month\",\"attributeType\":\"CATEGORICAL\"}, {\"attributeName\":\"day_of_week\",\"attributeType\":\"CATEGORICAL\"}, {\"attributeName\":\"duration\",\"attributeType\":\"NUMERIC\"}, {\"attributeName\":\"campaign\",\"attributeType\":\"NUMERIC\"}, {\"attributeName\":\"pdays\",\"attributeType\":\"NUMERIC\"}, {\"attributeName\":\"previous\",\"attributeType\":\"NUMERIC\"}, {\"attributeName\":\"poutcome\",\"attributeType\":\"CATEGORICAL\"}, {\"attributeName\":\"emp_var_rate\",\"attributeType\":\"NUMERIC\"}, {\"attributeName\":\"cons_price_idx\",\"attributeType\":\"NUMERIC\"}, {\"attributeName\":\"cons_conf_idx\",\"attributeType\":\"NUMERIC\"}, {\"attributeName\":\"euribor3m\",\"attributeType\":\"NUMERIC\"}, {\"attributeName\":\"nr_employed\",\"attributeType\":\"NUMERIC\"} ],\"excludedAttributeNames\":[]}" }, "dataSourceId": "exampleDataSourceId", "dataSourceName": "Banking sample for batch prediction" }, "responseElements": { "dataSourceId": "exampleDataSourceId" }, "requestID": "9b14bc94-894e-11e5-a84d-2d2deb28fdec", "eventID": "f1d47f93-c708-495b-bff1-cb935a6064b2", "eventType": "AwsApiCall", "recipientAccountId": "012345678910" }, { "eventVersion": "1.03", "userIdentity": { "type": "IAMUser", "principalId": "EX_PRINCIPAL_ID", "arn": "arn:aws:iam::012345678910:user/Alice", "accountId": "012345678910", "accessKeyId": "EXAMPLE_KEY_ID", "userName": "Alice" }, "eventTime": "2015-11-11T15:24:05Z", "eventSource": "", "eventName": "CreateBatchPrediction", "awsRegion": "us-east-1", "sourceIPAddress": "", "userAgent": "", "requestParameters": { "batchPredictionName": "Batch prediction: ML model: Banking sample", "batchPredictionId": "exampleBatchPredictionId", "batchPredictionDataSourceId": "exampleDataSourceId", "outputUri": "s3://EXAMPLE_BUCKET/BatchPredictionOutput/", "mLModelId": "exampleModelId" }, "responseElements": { "batchPredictionId": "exampleBatchPredictionId" }, "requestID": "3e18f252-8888-11e5-b6ca-c9da3c0f3955", "eventID": "db27a771-7a2e-4e9d-bfa0-59deee9d936d", "eventType": "AwsApiCall", "recipientAccountId": "012345678910" } ] }