Sign In to the Console
Amazon ECS
User Guide for AWS Fargate (API Version 2014-11-13)

AWS Documentation » Amazon ECS » User Guide for AWS Fargate » Amazon ECS Task Definitions » Using the awslogs Log Driver

Using the awslogs Log Driver

You can configure the containers in your tasks to send log information to CloudWatch Logs. This allows you to view the logs from the containers in your Fargate tasks.This topic helps you get started using the awslogs log driver in your task definitions.

Note

The type of information that is logged by your task's containers depends mostly on their ENTRYPOINT command. By default, the logs that are captured show the command output that you would normally see in an interactive terminal if you ran the container locally, which are the STDOUT and STDERR I/O streams. The awslogs log driver simply passes these logs from Docker to CloudWatch. For more information on how Docker logs are processed, including alternative ways to capture different file data or streams, see View logs for a container or service in the Docker documentation.

Enabling the awslogs Log Driver for Your Containers

If you are using the Fargate launch type for your tasks, all you need to do to enable the awslogs log driver is add the required logConfiguration parameters to your task definition. This configures your Fargate tasks to send log information to Amazon CloudWatch Logs. For more information, see Specifying a Log Configuration in your Task Definition.

Creating Your Log Groups

The awslogs log driver can send log streams to existing log groups in CloudWatch Logs, but it cannot create log groups. Before you launch any tasks that use the awslogs log driver, you should ensure the log groups that you intend your containers to use are created. The console provides an auto-configure option. If you register your task definitions in the console and choose the Auto-configure CloudWatch Logs option, your log groups are created for you. Alternatively, you can manually create your log groups using the following steps.

As an example, you could have a task with a WordPress container (which uses the awslogs-wordpress log group) that is linked to a MySQL container (which uses the awslogs-mysql log group). The sections below show how to create these log groups with the AWS CLI and with the CloudWatch console.

Creating a Log Group with the AWS CLI

The AWS Command Line Interface (AWS CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts. For more information, see the AWS Command Line Interface User Guide.

If you have a working installation of the AWS CLI, you can use it to create your log groups. The command below creates a log group called awslogs-wordpress in the us-west-2 region. Run this command for each log group to create, replacing the log group name with your value and region name to the desired log destination. 

aws logs create-log-group --log-group-name awslogs-wordpress --region us-west-2

Using the Auto-configuration Feature to Create a Log Group

When registering a task definition in the Amazon ECS console, you have the option to allow Amazon ECS to auto-configure your CloudWatch logs. The option also creates the specified log groups for you. To make it easy, the auto-configuration option sets up the CloudWatch logs and log groups with the specified prefix.

To create a log group in the Amazon ECS console

  1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

  2. In the left navigation pane, choose Task Definitions, Create new Task Definition.

  3. Select your compatibility option and choose Next Step.

  4. Choose Add container.

  5. In the Storage and Logging section, for Log configuration, choose Auto-configure CloudWatch Logs.

  6. Enter your awslogs log driver options. For more information, see Specifying a Log Configuration in your Task Definition.

  7. Complete the rest of the task definition wizard.

Creating a Log Group with the CloudWatch Console

The following procedure creates a log group in the CloudWatch console.

To create a log group in the CloudWatch console

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. In the left navigation pane, choose Logs.

  3. Choose Actions, Create log group.

  4. For Log Group Name, enter the name of the log group to create.

  5. Choose Create log group to finish.

Available awslogs Log Driver Options

The awslogs log driver supports the following options in Amazon ECS task definitions. For more information, see CloudWatch Logs logging driver.

awslogs-create-group

Required: No

Specify whether you want the log group automatically created. If this option is not specified, it defaults to false.

Note

Your IAM policy must include the logs:CreateLogGroup permission before you attempt to use awslogs-create-group.

awslogs-datetime-format

Required: No

This option defines a multiline start pattern in Python strftime format. A log message consists of a line that matches the pattern and any following lines that don’t match the pattern. Thus the matched line is the delimiter between log messages.

One example of a use case for using this format is for parsing output such as a stack dump, which might otherwise be logged in multiple entries. The correct pattern allows it to be captured in a single entry.

This option always takes precedence if both awslogs-datetime-format and awslogs-multiline-pattern are configured.

Note

Multiline logging performs regular expression parsing and matching of all log messages, which may have a negative impact on logging performance.

awslogs-region

Required: Yes

Specify the region to which the awslogs log driver should send your Docker logs. You can choose to send all of your logs from clusters in different regions to a single region in CloudWatch Logs so that they are all visible in one location, or you can separate them by region for more granularity. Be sure that the specified log group exists in the region that you specify with this option.

awslogs-group

Required: Yes

You must specify a log group to which the awslogs log driver sends its log streams. For more information, see Creating Your Log Groups.

awslogs-multiline-pattern

Required: No

This option defines a multiline start pattern using a regular expression. A log message consists of a line that matches the pattern and any following lines that don’t match the pattern. Thus the matched line is the delimiter between log messages.

This option is ignored if awslogs-datetime-format is also configured.

Note

Multiline logging performs regular expression parsing and matching of all log messages. This may have a negative impact on logging performance.

awslogs-stream-prefix

Required: Yes, when using the Fargate launch type.

The awslogs-stream-prefix option allows you to associate a log stream with the specified prefix, the container name, and the ID of the Amazon ECS task to which the container belongs. If you specify a prefix with this option, then the log stream takes the following format: 

prefix-name/container-name/ecs-task-id

For Amazon ECS services, you could use the service name as the prefix, which would allow you to trace log streams to the service that the container belongs to, the name of the container that sent them, and the ID of the task to which the container belongs.

Specifying a Log Configuration in your Task Definition

Before your containers can send logs to CloudWatch, you must specify the awslogs log driver for containers in your task definition. This section describes the log configuration for a container to use the awslogs log driver. For more information, see Creating a Task Definition.

The task definition JSON shown below has a logConfiguration object specified for each container; one for the WordPress container that sends logs to a log group called awslogs-wordpress, and one for a MySQL container that sends logs to a log group called awslogs-mysql. Both containers use the awslogs-example log stream prefix. 

{
    "containerDefinitions": [
        {
            "name": "wordpress",
            "links": [
                "mysql"
            ],
            "image": "wordpress",
            "essential": true,
            "portMappings": [
                {
                    "containerPort": 80,
                    "hostPort": 80
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "awslogs-wordpress",
                    "awslogs-region": "us-west-2",
                    "awslogs-stream-prefix": "awslogs-example"
                }
            },
            "memory": 500,
            "cpu": 10
        },
        {
            "environment": [
                {
                    "name": "MYSQL_ROOT_PASSWORD",
                    "value": "password"
                }
            ],
            "name": "mysql",
            "image": "mysql",
            "cpu": 10,
            "memory": 500,
            "essential": true,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "awslogs-mysql",
                    "awslogs-region": "us-west-2",
                    "awslogs-stream-prefix": "awslogs-example"
                }
            }
        }
    ],
    "family": "awslogs-example"
}

In the Amazon ECS console, the log configuration for the wordpress container is specified as shown in the image below.

Console log configuration

Viewing awslogs Container Logs in CloudWatch Logs

After your Fargate tasks that use the awslogs log driver have launched, your configured containers should be sending their log data to CloudWatch Logs. You can view and search these logs in the console.

To view your CloudWatch Logs data for a container from the Amazon ECS console

  1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

  2. On the Clusters page, select the cluster that contains the task to view.

  3. On the Cluster: cluster_name page, choose Tasks and select the task to view.

  4. On the Task: task_id page, expand the container view by choosing the arrow to the left of the container name.

  5. In the Log Configuration section, choose View logs in CloudWatch, which opens the associated log stream in the CloudWatch console.

    Task definition view of log configuration

To view your CloudWatch Logs data in the CloudWatch console

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. In the left navigation pane, choose Logs.

  3. Select a log group to view. You should see the log groups that you created in Creating Your Log Groups.

    awslogs console metrics view

  4. Choose a log stream to view.

    awslogs console metrics view