Menu
Amazon CloudWatch Logs
User Guide

Quick Start: Install and Configure the CloudWatch Logs Agent on an EC2 Instance at Launch

You can use Amazon EC2 user data, a feature of Amazon EC2 that allows parametric information to be passed to the instance on launch, to install and configure the CloudWatch Logs agent on that instance. To pass the CloudWatch Logs agent installation and configuration information to Amazon EC2, you can provide the configuration file in a network location such as an Amazon S3 bucket. You can launch a new EC2 instance and enable logs by performing the following steps.

To launch a new instance and enable CloudWatch Logs

  1. Create an agent configuration file that describes all your log groups and log streams.

    Sample agent configuration file for Amazon Linux

    [general]
    state_file = /var/awslogs/state/agent-state  
     
    [/var/log/messages]
    file = /var/log/messages
    log_group_name = /var/log/messages
    log_stream_name = {instance_id}
    datetime_format = %b %d %H:%M:%S

    Sample agent configuration file for Ubuntu

    [general]
    state_file = /var/awslogs/state/agent-state
     
    [/var/log/syslog]
    file = /var/log/syslog
    log_group_name = /var/log/syslog
    log_stream_name = {instance_id}
    datetime_format = %b %d %H:%M:%S

    The agent configuration file describes the log files to monitor and the target log groups and log streams to upload it to. The agent consumes this configuration file and starts monitoring and uploading all the log files described in it. For more information about the settings in the agent configuration file, see CloudWatch Logs Agent Reference.

    Save it as a text file (for example, awslogs.cfg) either on the AMI's filesystem, in a publicly accessible http/https location, or an Amazon S3 location (for example, s3://myawsbucket/my-config-file). For more information about assigning permissions to an Amazon S3 bucket, see Specifying Resources in a Policy in the Amazon Simple Storage Service Developer Guide.

    Note

    Configuring multiple log sources to send data to a single log stream is not supported.

  2. Open the Identity and Access Management (IAM) console at https://console.aws.amazon.com/iam/.

  3. In the navigation pane, choose Policies, Create Policy.

  4. On the Create Policy page, under Create Your Own Policy, choose Select. For more information about creating custom policies, see IAM Policies for Amazon EC2 in the Amazon EC2 User Guide for Linux Instances.

  5. On the Review Policy page, for Policy Name, type a name for the policy.

  6. For Policy Document, paste in the following policy:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "logs:CreateLogGroup",
                    "logs:CreateLogStream",
                    "logs:PutLogEvents",
                    "logs:DescribeLogStreams"
                ],
                "Resource": [
                    "arn:aws:logs:*:*:*"
                ]
            },
            {
                "Effect": "Allow",
                "Action": [
                    "s3:GetObject"
                ],
                "Resource": [
                    "arn:aws:s3:::myawsbucket/*"
                ]
            }
        ]
    }
  7. Choose Create Policy.

  8. In the navigation pane, choose Roles, Create New Role.

  9. On the Set Role Name page, enter a name for the role and choose Next Step.

  10. On the Select Role Type page, choose Select next to Amazon EC2.

  11. On the Attach Policy page, in the table header (next to Filter and Search), choose Policy Type, Customer Managed Policies.

  12. For Customer Managed Policies, select the IAM policy that you created above and choose Next Step.

  13. If you're satisfied with the role, choose Create Role.

    For more information about IAM users and policies, see IAM Users and Groups and Managing IAM Policies in the IAM User Guide.

  14. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

  15. If necessary, change the region. From the navigation bar, select the region that meets your needs. For more information, see Regions and Endpoints in the Amazon Web Services General Reference.

  16. Choose Launch Instance.

    For more information, see Launching an Instance in Amazon EC2 User Guide for Linux Instances.

  17. On the Step 1: Choose an Amazon Machine Image (AMI) page, select the Linux instance type to launch, and then on the Step 2: Choose an Instance Type page, choose Next: Configure Instance Details.

    Note

    Make sure that cloud-init (http://cloudinit.readthedocs.org/en/latest/index.html) is installed on your Amazon Machine Image (AMI). Amazon Linux, Ubuntu, and RHEL instances already include cloud-init, but CentOS and other AMIs in the AWS Marketplace may not.

  18. On the Step 3: Configure Instance Details page, for IAM role, select the IAM role that you created above.

  19. Under Advanced Details, for User data, paste in the script and update the -c option with the location of the configuration file:

    #!/bin/bash
    curl https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O
    chmod +x ./awslogs-agent-setup.py
    ./awslogs-agent-setup.py -n -r us-east-1 -c s3://myawsbucket/my-config-file

    Note

    You can install the CloudWatch Logs agent by specifying the us-east-1, us-west-1, us-west-2, ap-south-1, ap-northeast-2, ap-southeast-1, ap-southeast-2, ap-northeast-1, eu-central-1, eu-west-1, or sa-east-1 regions.

  20. Make any other changes to the instance, review your launch settings, and then choose Launch.

  21. You should see the newly-created log group and log stream in the CloudWatch console after the agent has been running for a few moments.

    To view your logs, see View Log Data Sent to CloudWatch Logs.