Using CloudWatch Logs with AWS Batch - AWS Batch

Using CloudWatch Logs with AWS Batch

You can configure your jobs to send log information to CloudWatch Logs. This enables you to view different logs from your jobs in one convenient location. This topic helps you get started using CloudWatch Logs on your jobs that were launched with an Amazon ECS-optimized Amazon Linux AMI.

For information about sending logs from your jobs to CloudWatch Logs, see Using the awslogs log driver. For more information about CloudWatch Logs, see Monitoring Log Files in the Amazon CloudWatch User Guide.

CloudWatch Logs IAM Policy

Before your jobs can send log data to CloudWatch Logs, you must create an IAM policy to allow your container instances to use the CloudWatch Logs APIs, and then you must attach that policy to ecsInstanceRole.

To create the ECS-CloudWatchLogs IAM policy

  1. Open the IAM console at https://console.aws.amazon.com/iam/.

  2. In the navigation pane, choose Policies.

  3. Choose Create policy, JSON.

  4. Enter the following policy:

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams" ], "Resource": [ "arn:aws:logs:*:*:*" ] } ] }
  5. Choose Review policy.

  6. On the Review policy page, enter ECS-CloudWatchLogs for the Name and choose Create policy.

To attach the ECS-CloudWatchLogs policy to ecsInstanceRole

  1. Open the IAM console at https://console.aws.amazon.com/iam/.

  2. In the navigation pane, choose Roles.

  3. Choose ecsInstanceRole. If the role does not exist, follow the procedures in Amazon ECS instance role to create the role.

  4. Choose Permissions, Attach policies.

  5. To narrow the available policies to attach, for Filter, type ECS-CloudWatchLogs.

  6. Select the ECS-CloudWatchLogs policy and choose Attach policy.

Installing and configuring the CloudWatch agent

After you have added the ECS-CloudWatchLogs policy to your ecsInstanceRole, you can install the CloudWatch agent on your container instances.

For more information, see Download and configure the CloudWatch agent using the command line in the Amazon CloudWatch User Guide.

Viewing CloudWatch Logs

After you have given your container instance role the proper permissions to send logs to CloudWatch Logs, and you have configured and started the agent, your container instance should be sending its log data to CloudWatch Logs. You can view and search these logs in the AWS Management Console.

Note

New instance launches may take a few minutes to send data to CloudWatch Logs.

To view your CloudWatch Logs data

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. In the left navigation pane, choose Logs, Log groups.

    
      CloudWatch console log groups
  3. Choose a log group to view.

    
      CloudWatch console log streams
  4. Choose a log stream to view. By default, the streams are identified by the first 200 characters of the job name and the Amazon ECS task ID.

    
      CloudWatch console log events