Menu
Amazon CloudWatch
Developer Guide (API Version 2010-08-01)

Quick Start: Install and Configure the CloudWatch Logs Agent on an Existing EC2 Instance

You can use the Amazon CloudWatch Logs agent installer on an existing EC2 instance to install and configure the CloudWatch Logs agent. After installation is complete, the agent confirms that it has started and it stays running until you disable it.

In addition to the agent, you can also publish log data using the AWS CLI, CloudWatch Logs SDK, or the CloudWatch Logs API. The AWS CLI is best suited for publishing data at the command line or through scripts. The CloudWatch Logs SDK is best suited for publishing log data directly from applications or building your own log publishing application.

Step 1: Configure your IAM role or user for CloudWatch Logs

The CloudWatch Logs agent supports IAM roles and users. If your instance already has an IAM role associated with it, make sure that you include the IAM policy below. If you don't already have an IAM role assigned to your instance, you'll need to use your IAM credentials for the next steps because you cannot assign an IAM role to an existing instance; you can only specify a role when you launch a new instance.

For more information about IAM users and policies, see IAM Users and Groups and Managing IAM Policies in IAM User Guide.

To configure your IAM role or user for CloudWatch Logs

  1. Open the Identity and Access Management (IAM) console at https://console.aws.amazon.com/iam/.

  2. In the navigation pane, click Roles, and then in the Role Name column, click an IAM role.

  3. On the Permissions tab, under Inline Policies, click Create Role Policy.

  4. On the Set Permissions page, click Custom Policy, and then click Select.

    For more information about creating custom policies, see IAM Policies for Amazon EC2 in the Amazon EC2 User Guide for Linux Instances.

  5. On the Review Policy page, in the Policy Name field, type a name for the policy.

  6. In the Policy Document field, paste in the following policy:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "logs:CreateLogGroup",
            "logs:CreateLogStream",
            "logs:PutLogEvents",
            "logs:DescribeLogStreams"
        ],
          "Resource": [
            "arn:aws:logs:*:*:*"
        ]
      }
     ]
    }
  7. Click Apply Policy.

Step 2: Install and configure CloudWatch Logs on an existing Amazon EC2 instance

The process for installing the CloudWatch Logs agent differs depending on whether your Amazon EC2 instance is running Amazon Linux, Ubuntu, CentOS, or Red Hat. Use the steps appropriate for the version of Linux on your instance.

To install and configure CloudWatch Logs on an existing Amazon Linux instance

Starting with Amazon Linux AMI 2014.09, the CloudWatch Logs agent is available as an RPM installation with the awslogs package. Earlier versions of Amazon Linux can access the awslogs package by updating their instance with the sudo yum update -y command. By installing the awslogs package as an RPM instead of the using the CloudWatch Logs installer, your instance will receive regular package updates and patches from Amazon without having to manually reinstall the CloudWatch Logs agent.

Caution

Do not update the CloudWatch Logs agent using the RPM installation method if you previously used the Python script to install the agent. Doing so may cause configuration issues that prevent the CloudWatch Logs agent from sending your logs to CloudWatch.

  1. Connect to your Amazon Linux instance. For more information, see Connect to Your Instance in the Amazon EC2 User Guide for Linux Instances.

    If you have trouble connecting, see Troubleshooting Connecting to Your Instance in the Amazon EC2 User Guide for Linux Instances.

  2. Update your Amazon Linux instance to pick up the latest changes in the package repositories.

    [ec2-user ~]$ sudo yum update -y
  3. Install the awslogs package.

    [ec2-user ~]$ sudo yum install -y awslogs
  4. Edit the /etc/awslogs/awscli.conf file and in the [default] section, specify the region where you want to view log data and add your credentials.

    region = us-east-1
    aws_access_key_id = <YOUR ACCESS KEY>
    aws_secret_access_key = <YOUR SECRET KEY>

    Note

    Adding your credentials here is optional if your instance was launched using an IAM role or user with the appropriate permissions to use CloudWatch Logs.

  5. Edit the /etc/awslogs/awslogs.conf file to configure the logs you would like to track. For more information on editing this file, see CloudWatch Logs Agent Reference.

  6. Start the awslogs service.

    [ec2-user ~]$ sudo service awslogs start
    Starting awslogs:                                          [  OK  ]
  7. (Optional) Check the /var/log/awslogs.log file for errors logged when starting the service.

  8. (Optional) Run the following command to start the awslogs service at each system boot.

    [ec2-user ~]$ sudo chkconfig awslogs on
  9. You should see the newly created log group and log stream in the CloudWatch console after the agent has been running for a few moments.

    To view your logs, see Viewing Log Data.

To install and configure CloudWatch Logs on an existing Ubuntu Server, CentOS, or Red Hat instance

If you're using an AMI running Ubuntu Server, CentOS, or Red Hat, use the following procedure to manually install the CloudWatch Logs agent on your instance.

  1. Connect to your EC2 instance. For more information, see Connect to Your Instance in the Amazon EC2 User Guide for Linux Instances.

    If you have trouble connecting, see Troubleshooting Connecting to Your Instance in the Amazon EC2 User Guide for Linux Instances.

  2. Run the CloudWatch Logs agent installer. On the instance, open a command prompt, type the following commands, and then follow the prompts.

    Note

    On Ubuntu, run apt-get update before running the commands below.

    curl https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O

    sudo python ./awslogs-agent-setup.py --region us-east-1

    Note

    You can install the CloudWatch Logs agent by specifying the us-east-1, us-west-1, us-west-2, eu-west-1, eu-central-1, ap-southeast-1, ap-southeast-2, ap-northeast-1, or sa-east-1 regions.

    The CloudWatch Logs agent installer requires certain information during set up. Before you start, you will need to know what log file you want to monitor and its timestamp format. You should also have the following information ready:

    ItemDescription

    AWS Access Key ID

    Press enter if using an IAM role. Otherwise, enter your AWS access key ID.

    AWS Secret Access Key

    Press enter if using an IAM role. Otherwise, enter your AWS secret access key.

    Default region name

    Press enter. The default is us-east-1. You can set this to us-east-1, us-west-1, us-west-2, eu-west-1, eu-central-1, ap-southeast-1, ap-southeast-2, ap-northeast-1, or sa-east-1.

    Default output format

    Leave blank and press enter.

    Path of log file to upload

    The location of the file that contains the log data you want to send. The installer will suggest a path for you.

    Destination Log Group name

    The name for your log group. The installer will suggest a log group name for you.

    Destination Log Stream name

    By default, this is the name of the host. The installer will suggest a host name for you.

    Timestamp format

    Specify the format of the timestamp within the specified log file. Choose custom to specify your own format.

    Initial position

    How data will be uploaded. Set this to start_of_file to upload everything in the data file. Set to end_of_file to upload only newly appended data.

    After you have completed these steps, the installer asks if you want to configure another log file. You can run the process as many times as you like for each log file. If you have no more log files to monitor, choose N when prompted by the installer to set up another log. For more information about the settings in the agent configuration file, see CloudWatch Logs Agent Reference.

    Note

    Configuring multiple log sources to send data to a single log stream is not supported.

  3. You should see the newly created log group and log stream in the CloudWatch console after the agent has been running for a few moments.

    To view your logs, see Viewing Log Data.