Creating and storing standard CloudWatch agent configuration files - AWS Prescriptive Guidance

Creating and storing standard CloudWatch agent configuration files

We recommend that you create a standard CloudWatch agent configuration file that includes the system logs and metrics that you want to capture across all your EC2 instances and on-premises servers. You can use the CloudWatch agent configuration file wizard to help you create the configuration file. You can run the configuration wizard multiple times to generate unique configurations for different systems and environments. You can also modify the configuration file or create variations by using the configuration file schema. The CloudWatch agent configuration file can be stored in AWS Systems Manager Parameter Store. However, this approach is harder to scale across multiple accounts and Regions because parameter values must be created and synchronized in each account and Region.

The amazon-cloudwatch-agent-ctl script included with the CloudWatch agent allows you to specify a configuration file, Parameter Store parameter, or the agent's default configuration. The default configuration aligns to the basic, predefined metric set and configures the agent to report memory and disk space metrics to CloudWatch. However, it doesn’t include any log file configurations. The default configuration is also applied if you use Systems Manager Quick Setup for the CloudWatch agent.

Because the default configuration doesn’t include logging and isn’t customized for your requirements, we recommend that you create and load your configuration files by including them in the CloudWatch configuration file directory. For Linux, the CloudWatch configuration directory is found at /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d. For Windows, the configuration directory is found at C:\ProgramData\Amazon\AmazonCloudWatchAgent\Configs.

When you start the CloudWatch agent, the agent automatically appends each file found in these directories to create a CloudWatch composite configuration file. The configuration files should be stored in a central location (for example, an S3 bucket) that can be accessed by your required accounts and Regions. They should also be copied to the CloudWatch configuration file directory. For more information about this approach, see the Set up State Manager and Distributor for CloudWatch agent deployment and configuration section of this guide.

Storing CloudWatch configuration files in an S3 bucket

We recommend that you use CloudWatch configuration files instead of Parameter Store parameters for the following reasons:

  • If you use multiple Regions, you must synchronize CloudWatch configuration updates in each Region’s Parameter Store. Parameter Store is a Regional service and the same parameter must be updated in each Region that uses the CloudWatch agent.

  • If you have multiple CloudWatch configurations, you must initiate the retrieval and application of each Parameter Store configuration. You must individually retrieve each CloudWatch configuration from the Parameter Store and also update the retrieval method whenever you add a new configuration. In contrast, CloudWatch provides a configuration directory for storing configuration files and applies each configuration in the directory, without requiring them to be individually specified.

  • If you use multiple accounts, you must ensure that each new account has the required CloudWatch configurations in its Parameter Store. You also need to make sure that any configuration changes are applied to these accounts and their Regions in the future.

You can store CloudWatch configurations in an S3 bucket that is accessible from all your accounts and Regions. You can then copy these configurations from the S3 bucket to the CloudWatch configuration directory by using Systems Manager Automation runbooks and Systems Manager State Manager. You can use the cloudwatch-config-s3-bucket.yaml AWS CloudFormation template to create an S3 bucket that is accessible from multiple accounts within an organization in AWS Organizations. The template includes an OrganizationID parameter that grants read access to all accounts within your organization.

After you create the S3 bucket, you can create a key or folder prefix structure to store your CloudWatch configuration files. The following table outlines a sample folder structure.

/config/standard/windows/ec2 Store standard Windows-specific CloudWatch configuration files for Amazon EC2. You can also further categorize your standard OS configurations for different Windows versions, EC2 instance types, and environments under this folder.
/config/standard/windows/onpremises Store standard Windows-specific CloudWatch configuration files for on-premises servers. You can also further categorize your standard OS configurations for different Windows versions, server types, and environments under this folder.
/config/standard/linux/ec2 Store your standard Linux-specific CloudWatch configuration files for Amazon EC2. You can also further categorize your standard OS configuration for different Linux distributions, EC2 instance types, and environments under this folder.
/config/standard/linux/onpremises Store your standard Linux-specific CloudWatch configuration files for on-premises servers. You can also further categorize your standard OS configuration for different Linux distributions, server types, and environments under this folder.
/config/ecs Store CloudWatch configuration files that are specific to Amazon ECS if you use Amazon ECS container instances. These configurations can be appended to the standard Amazon EC2 configurations for Amazon ECS specific systems-level logging and monitoring.
/config/<application_name> Store your application-specific CloudWatch configuration files. You can also further categorize your applications with additional folders and prefixes for environments and versions.

The augmented sample Systems Manager runbook, provided in the Set up State Manager and Distributor for CloudWatch agent deployment and configuration section of this guide, is configured to retrieve files using the S3 bucket created by the cloudwatch-config-s3-bucket.yaml AWS CloudFormation template.

Alternatively, you can use a version control system (for example, GitHub or AWS CodeCommit) to store your configuration files. If you want to automatically retrieve configuration files stored in a version control system, you have to manage or centralize the credential storage and update the Systems Manager Automation runbook that is used to retrieve the credentials across your accounts and Regions