Amazon CloudWatch
User Guide

Common Scenarios with the CloudWatch Agent

The following sections outline how to complete some common configuration and customization tasks when using the CloudWatch agent.

Adding Custom Dimensions to Metrics Collected by the CloudWatch Agent

To add custom dimensions such as tags to metrics collected by the agent, add the append_dimensions field to the section of the agent configuration file that lists those metrics.

For example, the following example section of the configuration file adds a custom dimension named stackName with a value of Prod to the cpu and disk metrics collected by the agent.

"cpu":{ "resources":[ "*" ], "measurement":[ "cpu_usage_guest", "cpu_usage_nice", "cpu_usage_idle" ], "totalcpu":false, "append_dimensions":{ "stackName":"Prod" } }, "disk":{ "resources":[ "/", "/tmp" ], "measurement":[ "total", "used" ], "append_dimensions":{ "stackName":"Prod" } }

Remember that any time you change the agent configuration file, you must restart the agent to have the changes take effect.

Multiple CloudWatch Agent Configuration Files

You can set up the CloudWatch agent to use multiple configuration files. For example, you can use a common configuration file that collects a set of metrics and logs that you always want to collect from all servers in your infrastructure. You can then use additional configuration files that collect metrics from certain applications or in certain situations.

To set this up, first create the configuration files that you want to use. Any configuration files that will be used together on the same server must have different file names. You can store the configuration files on servers or in Parameter Store.

Start the CloudWatch agent using the fetch-config option and specify the first configuration file. To append the second configuration file to the running agent, use the same command but with the append-config option. All metrics and logs listed in either configuration file are collected. The following example Linux commands illustrate this scenario using configurations stores as files. The first line starts the agent using the infrastructure.json configuration file, and the second line appends the app.json configuration file.

/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/tmp/infrastructure.json -s
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a append-config -m ec2 -c file:/tmp/app.json -s

The following example configuration files illustrate a use for this feature. The first configuration file is used for all servers in the infrastructure, and the second collects only logs from a certain application and is appended to servers running that application.

infrastructure.json

{ "metrics": { "metrics_collected": { "cpu": { "resources": [ "*" ], "measurement": [ "usage_active" ], "totalcpu": true }, "mem": { "measurement": [ "used_percent" ] } } }, "logs": { "logs_collected": { "files": { "collect_list": [ { "file_path": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log", "log_group_name": "amazon-cloudwatch-agent.log" }, { "file_path": "/var/log/messages", "log_group_name": "/var/log/messages" } ] } } } }

app.json

{ "logs": { "logs_collected": { "files": { "collect_list": [ { "file_path": "/app/app.log*", "log_group_name": "/app/app.log" } ] } } }

Any configuration files appended to the configuration must have different file names from each other and from the initial configuration file. If you use append-config with a configuration file with the same file name as a configuration file that the agent is already using, the append command overwrites the information from the first configuration file instead of appending to it. This is true even if the two configuration files with the same file name are on different file paths.

The preceding example shows the use of two configuration files, but there is no limit to the number of configuration files that you can append to the agent configuration. You can also mix the use of configuration files located on servers and configurations located in Parameter Store.

Aggregating or Rolling Up Metrics Collected by the CloudWatch Agent

To aggregate or roll up metrics collected by the agent, add an aggregation_dimensions field to the section for that metric in the agent configuration file.

For example, the following configuration file snippet rolls up metrics on the AutoScalingGroupName dimension. The metrics from all instances in each Auto Scaling group are aggregated and can be viewed as a whole.

"metrics": { "cpu":{...} "disk":{...} "aggregation_dimensions" : [["AutoScalingGroupName"]] }

To roll up along the combination of each InstanceId and InstanceType dimensions in addition to rolling up on the Auto Scaling group name, add the following.

"metrics": { "cpu":{...} "disk":{...} "aggregation_dimensions" : [["AutoScalingGroupName"], ["InstanceId", "InstanceType"]] }

To roll up metrics into one collection instead, use [].

"metrics": { "cpu":{...} "disk":{...} "aggregation_dimensions" : [[]] }

Remember that any time you change the agent configuration file, you must restart the agent to have the changes take effect.

Collecting High-Resolution Metrics With the CloudWatch agent

The metrics_collection_interval field specifies the time interval for the metrics collected, in seconds. By specifying a value of less than 60 for this field, the metrics are collected as high-resolution metrics.

For example, if your metrics should all be high-resolution and collected every 10 seconds, specify 10 as the value for metrics_collection_interval under the agent section as a global metrics collection interval.

"agent": { "metrics_collection_interval": 10 }

Alternatively, the following example sets the cpu metrics to be collected every second, and all other metrics are collected every minute.

"agent":{ "metrics_collection_interval": 60 }, "metrics":{ "metrics_collected":{ "cpu":{ "resources":[ "*" ], "measurement":[ "cpu_usage_guest" ], "totalcpu":false, "metrics_collection_interval": 1 }, "disk":{ "resources":[ "/", "/tmp" ], "measurement":[ "total", "used" ] } } }

Remember that any time you change the agent configuration file, you must restart the agent to have the changes take effect.

Sending Metrics and Logs to a Different Account

To have the CloudWatch agent send the metrics, logs, or both to a different account, specify a role_arn parameter in the agent configuration file on the sending server. The role_arn value specifies an IAM role in the target account that the agent uses when sending data to the target account. This role enables the sending account to assume a corresponding role in the target account when delivering the metrics or logs to the target account.

You can also specify two separate role_arn strings in the agent configuration file: one to use when sending metrics and another for sending logs.

The following example of part of the agent section of the configuration file sets the agent to use CrossAccountAgentRole when sending metrics and logs to a different account.

{ "agent": { "credentials": { "role_arn": "CrossAccountAgentRole" } }, ..... }

Alternatively, the following example sets different roles for the sending account to use for sending metrics and logs:

"metrics": { "credentials": { "role_arn": "RoleToSendMetrics" }, "metrics_collected": {....
"logs": { "credentials": { "role_arn": "RoleToSendLogs" }, ....

Policies Needed

When you specify a role_arn in the agent configuration file, you must also make sure the IAM roles of the sending and target accounts have certain policies. The roles in both the sending and target accounts should have CloudWatchAgentServerPolicy. For more information about assigning this policy to a role, see Create IAM Roles to Use with the CloudWatch Agent on Amazon EC2 Instances.

The role in the sending account also must include the following policy. You add this policy on the Permissions tab in the IAM console when you edit the role.

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "sts:AssumeRole" ], "Resource": [ "arn:aws:iam::target-account-ID:role/agent-role-in-target-account" ] } ] }

The role in the target account must include the following policy so that it recognizes the IAM role used by the sending account. You add this policy on the Trust relationships tab in the IAM console when you edit the role. The role in the target account where you add this policy is the role you created in Create IAM Roles to Use with the CloudWatch Agent on Amazon EC2 Instances. This role is the role specified in agent-role-in-target-account in the policy used by the sending account.

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::sending-account-ID:role/role-specified-in-role_arn" ] }, "Action": "sts:AssumeRole" } ] }

Timestamp Differences Between the Unified CloudWatch Agent and the Older CloudWatch Logs Agent

The CloudWatch agent supports a different set of symbols for timestamp formats, compared to the older CloudWatch Logs agent. These differences are shown in the following table.

Symbols Supported by Both Agents Symbols Supported Only by Unified CloudWatch Agent Symbols Supported Only by Older CloudWatch Logs Agent

%A, %a, %b, %B, %d, %H, %l, %m, %M, %p, %S, %y, %Y, %Z, %z

%-d, %-l, %-m, %-M, %-S

%c, %f, %j, %U, %W, %w

For more information about the meanings of the symbols supported by the new CloudWatch agent, see CloudWatch Agent Configuration File: Logs Section in the Amazon CloudWatch User Guide. For information about symbols supported by the CloudWatch Logs agent, see Agent Configuration File in the Amazon CloudWatch Logs User Guide.