Installing the CloudWatch agent on new instances using AWS CloudFormation - Amazon CloudWatch

Installing the CloudWatch agent on new instances using AWS CloudFormation

Amazon has uploaded several AWS CloudFormation templates to GitHub to help you install and update the CloudWatch agent on new Amazon EC2 instances. For more information about using AWS CloudFormation, see What is AWS CloudFormation?.

The template location is Deploy the Amazon CloudWatch agent to EC2 instances using AWS CloudFormation. This location includes both inline and ssm directories. Each of these directories contains templates for both Linux and Windows instances.

  • The templates in the inline directory have the CloudWatch agent configuration embedded into the AWS CloudFormation template. By default, the Linux templates collect the metrics mem_used_percent and swap_used_percent, and the Windows templates collect Memory % Committed Bytes In Use and Paging File % Usage.

    To modify these templates to collect different metrics, modify the following section of the template. The following example is from the template for Linux servers. Follow the format and syntax of the agent configuration file to make these changes. For more information, see Manually create or edit the CloudWatch agent configuration file.

    { "metrics":{ "append_dimensions":{ "AutoScalingGroupName":"${!aws:AutoScalingGroupName}", "ImageId":"${!aws:ImageId}", "InstanceId":"${!aws:InstanceId}", "InstanceType":"${!aws:InstanceType}" }, "metrics_collected":{ "mem":{ "measurement":[ "mem_used_percent" ] }, "swap":{ "measurement":[ "swap_used_percent" ] } } } }
    Note

    In the inline templates, all placeholder variables must have an exclamation mark (!) before them as an escape character. You can see this in the example template. If you add other placeholder variables, be sure to add an exclamation mark before the name.

  • The templates in the ssm directory load an agent configuration file from Parameter Store. To use these templates, you must first create a configuration file and upload it to Parameter Store. You then provide the Parameter Store name of the file in the template. You can create the configuration file manually or by using the wizard. For more information, see Create the CloudWatch agent configuration file.

You can use both types of templates for installing the CloudWatch agent and for updating the agent configuration.

Tutorial: Install and configure the CloudWatch agent using an AWS CloudFormation inline template

This tutorial walks you through using AWS CloudFormation to install the CloudWatch agent on a new Amazon EC2 instance. This tutorial installs on a new instance running Amazon Linux 2 using the inline templates, which don't require the use of the JSON configuration file or Parameter Store. The inline template includes the agent configuration in the template. In this tutorial, you use the default agent configuration contained in the template.

After the procedure for installing the agent, the tutorial continues with how to update the agent.

To use AWS CloudFormation to install the CloudWatch agent on a new instance
  1. Download the template from GitHub. In this tutorial, download the inline template for Amazon Linux 2 as follows:

    curl -O https://raw.githubusercontent.com/awslabs/aws-cloudformation-templates/master/aws/solutions/AmazonCloudWatchAgent/inline/amazon_linux.template
  2. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.

  3. Choose Create stack.

  4. For Choose a template, select Upload a template to Amazon S3, choose the downloaded template, and choose Next.

  5. On the Specify Details page, fill out the following parameters and choose Next:

    • Stack name: Choose a stack name for your AWS CloudFormation stack.

    • IAMRole: Choose an IAM role that has permissions to write CloudWatch metrics, logs, and traces. For more information, see Create IAM roles to use with the CloudWatch agent on Amazon EC2 instances.

    • InstanceAMI: Choose an AMI that is valid in the Region where you're going to launch your stack.

    • InstanceType: Choose a valid instance type.

    • KeyName: To enable SSH access to the new instance, choose an existing Amazon EC2 key pair. If you don't already have an Amazon EC2 key pair, you can create one in the AWS Management Console. For more information, see Amazon EC2 Key Pairs in the Amazon EC2 User Guide for Linux Instances.

    • SSHLocation: Specifies the IP address range that can be used to connect to the instance using SSH. The default allows access from any IP address.

  6. On the Options page, you can choose to tag your stack resources. Choose Next.

  7. On the Review page, review your information, acknowledge that the stack might create IAM resources, and then choose Create.

    If you refresh the console, you see that the new stack has the CREATE_IN_PROGRESS status.

  8. When the instance is created, you can see it in the Amazon EC2 console. Optionally, you can connect to the host and check the progress.

    Use the following command to confirm that the agent is installed:

    rpm -qa amazon-cloudwatch-agent

    Use the following command to confirm that the agent is running:

    ps aux | grep amazon-cloudwatch-agent

The next procedure demonstrates using AWS CloudFormation to update the CloudWatch agent using an inline template. The default inline template collects the mem_used_percent metric. In this tutorial, you change the agent configuration to stop collecting that metric.

To use AWS CloudFormation to update the CloudWatch agent
  1. In the template that you downloaded in the previous procedure, remove the following lines and then save the template:

    "mem": { "measurement": [ "mem_used_percent" ] },
  2. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.

  3. On the AWS CloudFormation dashboard, select the stack that you created and choose Update Stack.

  4. For Select Template, select Upload a template to Amazon S3, choose the template that you modified, and choose Next.

  5. On the Options page, choose Next and then Next.

  6. On the Review page, review your information and choose Update.

    After some time, you see UPDATE_COMPLETE.

Tutorial: Install the CloudWatch agent using AWS CloudFormation and Parameter Store

This tutorial walks you through using AWS CloudFormation to install the CloudWatch agent on a new Amazon EC2 instance. This tutorial installs on a new instance running Amazon Linux 2 using an agent configuration file that you create and save in Parameter Store.

After the procedure for installing the agent, the tutorial continues with how to update the agent.

To use AWS CloudFormation to install the CloudWatch agent on a new instance using a configuration from Parameter Store
  1. If you haven't done so already, download the CloudWatch agent package to one of your computers so that you can create the agent configuration file. For more information and downloading the agent using Parameter Store, see Download and configure the CloudWatch agent. For more information on downloading the package using the command line, see Download and configure the CloudWatch agent using the command line.

  2. Create the agent configuration file and save it in Parameter Store. For more information, see Create the CloudWatch agent configuration file.

  3. Download the template from GitHub as follows:

    curl -O https://raw.githubusercontent.com/awslabs/aws-cloudformation-templates/master/aws/solutions/AmazonCloudWatchAgent/ssm/amazon_linux.template
  4. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.

  5. Choose Create stack.

  6. For Choose a template, select Upload a template to Amazon S3, choose the template that you downloaded, and choose Next.

  7. On the Specify Details page, fill out the following parameters accordingly and choose Next:

    • Stack name: Choose a stack name for your AWS CloudFormation stack.

    • IAMRole: Choose an IAM role that has permissions to write CloudWatch metrics, logs, and traces. For more information, see Create IAM roles to use with the CloudWatch agent on Amazon EC2 instances.

    • InstanceAMI: Choose an AMI that is valid in the Region where you're going to launch your stack.

    • InstanceType: Choose a valid instance type.

    • KeyName: To enable SSH access to the new instance, choose an existing Amazon EC2 key pair. If you don't already have an Amazon EC2 key pair, you can create one in the AWS Management Console. For more information, see Amazon EC2 Key Pairs in the Amazon EC2 User Guide for Linux Instances.

    • SSHLocation: Specifies the IP address range that can be used to connect to the instance using SSH. The default allows access from any IP address.

    • SSMKey: Specifies the agent configuration file that you created and saved in Parameter Store.

  8. On the Options page, you can choose to tag your stack resources. Choose Next.

  9. On the Review page, review your information, acknowledge that the stack might create IAM resources, and then choose Create.

    If you refresh the console, you see that the new stack has the CREATE_IN_PROGRESS status.

  10. When the instance is created, you can see it in the Amazon EC2 console. Optionally, you can connect to the host and check the progress.

    Use the following command to confirm that the agent is installed:

    rpm -qa amazon-cloudwatch-agent

    Use the following command to confirm that the agent is running:

    ps aux | grep amazon-cloudwatch-agent

The next procedure demonstrates using AWS CloudFormation to update the CloudWatch agent, using an agent configuration that you saved in Parameter Store.

To use AWS CloudFormation to update the CloudWatch agent using a configuration in Parameter Store
  1. Change the agent configuration file stored in Parameter Store to the new configuration that you want.

  2. In the AWS CloudFormation template that you downloaded in the Tutorial: Install the CloudWatch agent using AWS CloudFormation and Parameter Store topic, change the version number. For example, you might change VERSION=1.0 to VERSION=2.0.

  3. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.

  4. On the AWS CloudFormation dashboard, select the stack that you created and choose Update Stack.

  5. For Select Template, select Upload a template to Amazon S3, select the template that you just modified, and choose Next.

  6. On the Options page, choose Next and then Next.

  7. On the Review page, review your information and choose Update.

    After some time, you see UPDATE_COMPLETE.

Troubleshooting installation of the CloudWatch agent with AWS CloudFormation

This section helps you troubleshoot issues with installing and updating the CloudWatch agent using AWS CloudFormation.

Detecting when an update fails

If you use AWS CloudFormation to update your CloudWatch agent configuration, and use an invalid configuration, the agent stops sending any metrics to CloudWatch. A quick way to check whether an agent configuration update succeeded is to look at the cfn-init-cmd.log file. On a Linux server, the file is located at /var/log/cfn-init-cmd.log. On a Windows instance, the file is located at C:\cfn\log\cfn-init-cmd.log.

Metrics are missing

If you don't see metrics that you expect to see after installing or updating the agent, confirm that the agent is configured to collect that metric. To do this, check the amazon-cloudwatch-agent.json file to make sure that the metric is listed, and check that you are looking in the correct metric namespace. For more information, see CloudWatch agent files and locations.