Menu
Amazon EC2 Container Service
Developer Guide (API Version 2014-11-13)

Using the awslogs Log Driver

You can configure the containers in your tasks to send log information to CloudWatch Logs. This enables you to view different logs from your containers in one convenient location, and it prevents your container logs from taking up disk space on your container instances. This topic helps you get started using the awslogs log driver in your task definitions.

To send system logs from your Amazon ECS container instances to CloudWatch Logs, see Using CloudWatch Logs with Container Instances. For more information about CloudWatch Logs, see Monitoring Log Files in the Amazon CloudWatch User Guide.

Enabling the awslogs Log Driver on your Container Instances

Your Amazon ECS container instances require at least version 1.9.0 of the container agent to enable the awslogs log driver. For information about checking your agent version and updating to the latest version, see Updating the Amazon ECS Container Agent.

Note

If you are not using the Amazon ECS-optimized AMI (with at least version 1.9.0-1 of the ecs-init package) for your container instances, you also need to specify that the awslogs logging driver is available on the container instance when you start the agent by using the following environment variable in your docker run statement or environment variable file. For more information, see Installing the Amazon ECS Container Agent.

Copy
ECS_AVAILABLE_LOGGING_DRIVERS='["json-file","awslogs"]'

Your Amazon ECS container instances also require logs:CreateLogStream and logs:PutLogEvents permission on the IAM role with which you launch your container instances. If you created your Amazon ECS container instance role before awslogs log driver support was enabled in Amazon ECS, then you might need to add this permission. If your container instances use the managed IAM policy for container instances, then your container instances should have the correct permissions. For information about checking your Amazon ECS container instance role and attaching the managed IAM policy for container instances, see To check for the ecsInstanceRole in the IAM console.

Creating Your Log Groups

The awslogs log driver can send log streams to existing log groups in CloudWatch Logs, but it cannot create log groups. Before you launch any tasks that use the awslogs log driver, you must create the log groups that you intend your containers to use.

As an example, you could have a task with a WordPress container (which uses the awslogs-wordpress log group) that is linked to a MySQL container (which uses the awslogs-mysql log group). The sections below show how to create these log groups with the AWS CLI and with the CloudWatch console.

Creating a Log Group with the AWS CLI

The AWS Command Line Interface (AWS CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts. For more information, see the AWS Command Line Interface User Guide.

If you have a working installation of the AWS CLI, you can use it to create your log groups. The command below creates a log group called awslogs-wordpress in the ap-northeast-1 region. Run this command for each log group to create, replacing the log group name with your value and region name to the desired log destination.

Copy
aws logs create-log-group --log-group-name awslogs-wordpress --region ap-northeast-1

Creating a Log Group with the CloudWatch Console

The following procedure creates a log group in the CloudWatch console.

To create a log group in the CloudWatch console

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. In the left navigation pane, choose Logs.

  3. Choose Actions, Create log group.

  4. For Log Group Name, enter the name of the log group to create.

  5. Choose Create log group to finish.

Available awslogs Log Driver Options

The awslogs log driver supports the following options in Amazon ECS task definitions:

awslogs-region

Required: Yes

Specify the region to which the awslogs log driver should send your Docker logs. You can choose to send all of your logs from clusters in different regions to a single region in CloudWatch Logs so that they are all visible in one location, or you can separate them by region for more granularity. Be sure that the specified log group exists in the region that you specify with this option.

awslogs-group

Required: Yes

You must specify a log group to which the awslogs log driver will send its log streams. For more information, see Creating Your Log Groups.

awslogs-stream-prefix

Required: No

The awslogs-stream-prefix option allows you to associate a log stream with the specified prefix, the container name, and the ID of the Amazon ECS task to which the container belongs. If you specify a prefix with this option, then the log stream takes the following format:

prefix-name/container-name/ecs-task-id

If you do not specify a prefix with this option, then the log stream is named after the container ID that is assigned by the Docker daemon on the container instance. Because it is difficult to trace logs back to the container that sent them with just the Docker container ID (which is only available on the container instance), we recommend that you specify a prefix with this option.

For Amazon ECS services, you could use the service name as the prefix, which would allow you to trace log streams to the service that the container belongs to, the name of the container that sent them, and the ID of the task to which the container belongs.

Specifying a Log Configuration in your Task Definition

Before your containers can send logs to CloudWatch, you must specify the awslogs log driver for containers in your task definition. This section describes the log configuration for a container to use the awslogs log driver. For more information, see Creating a Task Definition.

The task definition JSON shown below has a logConfiguration object specified for each container; one for the WordPress container that sends logs to a log group called awslogs-wordpress, and one for a MySQL container that sends logs to a log group called awslogs-mysql. Both containers use the awslogs-example log stream prefix.

Copy
{ "containerDefinitions": [ { "name": "wordpress", "links": [ "mysql" ], "image": "wordpress", "essential": true, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "awslogs-wordpress", "awslogs-region": "ap-northeast-1", "awslogs-stream-prefix": "awslogs-example" } }, "memory": 500, "cpu": 10 }, { "environment": [ { "name": "MYSQL_ROOT_PASSWORD", "value": "password" } ], "name": "mysql", "image": "mysql", "cpu": 10, "memory": 500, "essential": true, "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "awslogs-mysql", "awslogs-region": "ap-northeast-1", "awslogs-stream-prefix": "awslogs-example" } } } ], "family": "awslogs-example" }

In the Amazon ECS console, the log configuration for the wordpress container is specified as shown in the image below.


                    Console log configuration
After you have registered a task definition with the awslogs log driver in a container definition log configuration, you can run a task or create a service with that task definition to start sending logs to CloudWatch Logs. For more information, see Running Tasks and Creating a Service.

Viewing awslogs Container Logs in CloudWatch Logs

After your container instance role has the proper permissions to send logs to CloudWatch Logs, your container agents are updated to at least version 1.9.0, and you have configured and started a task with containers that use the awslogs log driver, your configured containers should be sending their log data to CloudWatch Logs. You can view and search these logs in the console.

To view your CloudWatch Logs data for a container from the Amazon ECS console

  1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

  2. On the Clusters page, select the cluster that contains the task to view.

  3. On the Cluster: cluster_name page, choose Tasks and select the task to view.

  4. On the Task: task_id page, expand the container view by choosing the arrow to the left of the container name.

  5. In the Log Configuration section, choose View logs in CloudWatch, which opens the associated log stream in the CloudWatch console.

    
                            Task definition view of log configuration

To view your CloudWatch Logs data in the CloudWatch console

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. In the left navigation pane, choose Logs.

  3. Select a log group to view. You should see the log groups that you created in Creating Your Log Groups.

    
                            awslogs console metrics view

  4. Choose a log stream to view.

    
                            awslogs console metrics view