Centralize monitoring by using Amazon CloudWatch Observability Access Manager

Created by Anand Krishna Varanasi (AWS), Jimmy Morgan (AWS), Ashish Kumar (AWS), Balaji Vedagiri (AWS), JAGDISH KOMAKULA (AWS), Sarat Chandra Pothula (AWS), and Vivek Thangamuthu (AWS)

Code repository: cloudwatch-obervability-access-manager-terraform	Environment: Production	Technologies: Infrastructure; Multi account strategy; Operations
AWS services: Amazon CloudWatch; Amazon CloudWatch Logs

Summary

Observability is crucial to monitoring, understanding, and troubleshooting applications. Applications that span multiple accounts, as with AWS Control Tower or landing zone implementations, generate a large number of logs and trace data. To quickly troubleshoot problems or understand user analytics or business analytics, you need a common observability platform across all accounts. The Amazon CloudWatch Observability Access Manager gives you access to, and control over, multiple account logs from a central location.

You can use the Observability Access Manager to view and manage observability data logs generated by source accounts. Source accounts are individual AWS accounts that generate observability data for their resources. Observability data is shared between source accounts and monitoring accounts. The shared observability data can include metrics in Amazon CloudWatch, logs in Amazon CloudWatch Logs, and traces in AWS X-Ray. For more information, see the Observability Access Manager documentation.

This pattern is for users who have applications or infrastructure that run in multiple AWS accounts and need a common place to view logs. It explains how you can set up Observability Access Manager by using Terraform, to monitor the status and health of these applications or infrastructure. You can install this solution in multiple ways:

• As a standalone Terraform module that you set up manually

• By using a continuous integration and continuous delivery (CI/CD) pipeline

• By integrating with other solutions such as AWS Control Tower Account Factory for Terraform (AFT)

The instructions in the Epics section cover the manual implementation. For AFT installation steps, see the readme file for the GitHub Observability Access Manager repository.

Prerequisites and limitations

Prerequisites

• Terraform installed or referenced in your system or in automated pipelines. (We recommend that you use the latest version.)

• An account that you can use as a central monitoring account. Other accounts create links to the central monitoring account in order to view logs.

• (Optional) A source code repository such as GitHub, AWS CodeCommit, Atlassian Bitbucket, or similar system. A source code repository isn’t necessary if you’re using automated CI/CD pipelines.

• (Optional) Permissions to create pull requests (PRs) for code review and code collaboration in GitHub.

Limitations

Observability Access Manager has the following service quotas, which cannot be changed. Consider these quotas before you deploy this feature. For more information, see CloudWatch service quotas in the CloudWatch documentation.

• Source account links: You can link each source account to a maximum of five monitoring accounts.

• Sinks: You can use only one sink per account.

In addition:

• Sinks and links must be created in the same AWS Region; they cannot be cross-Region.

• For cross-Region, cross-account monitoring, you can create cross-account and cross-region CloudWatch dashboards for alarms and metrics, except for logs and traces. Another option is to create centralized logging by using Amazon OpenSearch Service.

Architecture

Components

Amazon CloudWatch Observability Access Manager consists of two major components that enable cross-account observability:

• A sink provides the ability for source accounts to send observability data to the central monitoring account. A sink basically provides a gateway junction for source accounts to connect to. There can be only one sink gateway or connection, and multiple accounts can connect to it.

• Each source account has a link to the sink gateway junction, and observability data is sent through this link. You must create a sink before you create links from each source account.

Architecture

The following diagram illustrates Observability Access Manager and its components.

Tools

AWS services

• Amazon CloudWatch helps you monitor the metrics of your AWS resources and the applications you run on AWS in real time.

• AWS Organizations is an account management service that helps you consolidate multiple AWS accounts into an organization that you create and centrally manage.

• AWS Identity and Access Management (IAM) helps you securely manage access to your AWS resources by controlling who is authenticated and authorized to use them.

Tools

• Terraform is an infrastructure as code (IaC) tool from HashiCorp that helps you create and manage cloud and on-premises resources.

• AWS Control Tower Account Factory for Terraform (AFT) sets up a Terraform pipeline to help you provision and customize accounts in AWS Control Tower. You can optionally use AFT to set up Observability Access Manager at scale across multiple accounts.

Code repository

The code for this pattern is available in the GitHub Observability Access Manager repository.

Best practices

• In AWS Control Tower environments, mark the logging account as the central monitoring account (sink).

• If you have multiple organizations with multiple accounts in AWS Organizations, we recommend that you include the organizations instead of individual accounts in the configuration policy. If you have a small number of accounts or if the accounts aren’t part of an organization in the sink configuration policy, you might decide to include individual accounts instead.

Epics

Set up the sink module

Task	Description	Skills required
Clone the repository.	Clone the GitHub Observability Access Manager repository: git clone https://github.com/aws-samples/cloudwatch-obervability-access-manager-terraform	AWS DevOps, Cloud administrator, AWS administrator
Specify property values for the sink module.	In the main.tf file (in the deployments/aft-account-customizations/LOGGING/terraform/ folder of the repository), specify values for the following properties: sink_name: The name of the Amazon CloudWatch sink. allowed_oam_resource_types: Observability Access Manager currently supports CloudWatch metrics, log groups, and AWS X-Ray traces. allowed_source_accounts: The source accounts that are allowed to send logs to the central CloudWatch sink account. allowed_source_organizations: The source Control Tower organizations that are allowed to send logs to the central CloudWatch sink account. For more information, see AWS::Oam::Sink in the AWS CloudFormation documentation.	AWS DevOps, Cloud administrator, AWS administrator
Install the sink module.	Export the credentials of the AWS account that you have selected as the monitoring account, and install the Observability Access Manager sink module: Terraform Init Terrafom Plan Terraform Apply	AWS DevOps, Cloud administrator, AWS administrator

Set up the link module

Task	Description	Skills required
Specify property values for the link module.	In the main.tf file (in the deployments/aft-account-customizations/LOGGING/terraform/ folder of the repository), specify values for the following properties: account_label: Use one of the following values: $AccountName: The name of the account. $AccountEmail: A globally unique email address, which includes the email domain (for example, [e-mail]) $AccountEmailNoDomain: An email address without the domain name. allowed_oam_resource_types: Observability Access Manager currently supports CloudWatch metrics, log groups, and AWS X-Ray traces. For more information, see AWS::Oam::Link in the AWS CloudFormation documentation.	AWS DevOps, Cloud administrator, Cloud architect
Install the link module for individual accounts.	Export the credentials of individual accounts and install the Observability Access Manager link module: Terraform Plan Terraform Apply You can set up the link module individually for each account, or use AFT to automatically install this module across a large number of accounts.	AWS DevOps, Cloud administrator, Cloud architect

Approve sink-to-link connections

Task	Description	Skills required
Check the status message.	Sign in to the monitoring account. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/. In the left navigation pane, choose Settings. On the right, you should see the status message Monitoring account enabled with a green checkmark. This means that the monitoring account has an Observability Access Manager sink that the links of other accounts will connect to.
Approve the link-to-sink connections.	Choose the Resources to link accounts option below the status message. The information confirms that this is the monitoring account, lists the data that is shared from the tenant source accounts (Logs, Metrics, Traces), and shows the account label as $AccountName. This screen provides two options for linking tenant accounts to the monitoring account: organization-level approval or account-level approval. For each option, you can choose to download an AWS CloudFormation template for the approval or approve each account individually. For simplicity, choose Any Account to approve at each account level. This option provides an approval link for the account. Choose Copy URL to copy the link. Sign in to each source account. In a browser window, paste the link, and choose Approve link connect to sink. Repeat for additional source accounts. For more information, see Link monitoring accounts with source accounts in the Amazon CloudWatch documentation.	AWS DevOps, Cloud administrator, Cloud architect

Verify cross-account observability data

Task	Description	Skills required
View cross-account data.	Sign in to the central monitoring account. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/. In the left navigation pane, choose options to view cross-account logs, metrics, and traces.	AWS DevOps, Cloud administrator, Cloud architect

(Optional) Enable source accounts to trust monitoring account

Task	Description	Skills required
View metrics, dashboards, logs, widgets, and alarms from other accounts.	As an additional feature, you can share the CloudWatch metrics, dashboards, logs, widgets, and alarms with other accounts. Each account uses an IAM role called CloudWatch-CrossAccountSharingRole to gain access to this data. Source accounts that have a trust relationship with the central monitoring account can assume this role and view data from the monitoring account. CloudWatch provides a sample CloudFormation script to create the role. Choose Manage role in IAM and run this script in the accounts where you want to view data. { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::XXXXXXXXX:root", "arn:aws:iam::XXXXXXXXX:root", "arn:aws:iam::XXXXXXXXX:root", "arn:aws:iam::XXXXXXXXX:root" ] }, "Action": "sts:AssumeRole" } ] } For more information, see Enabling cross-account functionality in CloudWatch in the CloudWatch documentation	AWS DevOps, Cloud administrator, Cloud architect

(Optional) View cross-account cross-Region from the monitoring account

Task	Description	Skills required
Set up cross-account, cross-Region access.	In the central monitoring account, you can optionally add an account selector to easily switch between accounts and view their data without having to authenticate. Sign in to the central monitoring account. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/. In the left navigation pane, choose Setttings. In the View cross-account cross-region section, choose Configure. Choose Enable, and then select the Show selector in the console check box. Choose one of these options: Account Id Input: This option prompts you to manually input the account ID whenever you want to change accounts to view cross-account data. AWS Organization account selector: If you have integrated CloudWatch with AWS Organizations, this option provides a dropdown selector with a complete list of accounts in the organization. Custom account selector: This option lets you manually input a list of account IDs to populate the selector. Choose Save changes. For more information, see Cross-account cross-Region CloudWatch console in the CloudWatch documentation.	AWS DevOps, Cloud administrator, Cloud architect

Related resources

• CloudWatch cross-account observability (Amazon CloudWatch documentation)

• Amazon CloudWatch Observability Access Manager API Reference (Amazon CloudWatch documentation)

• Resource: aws_oam_sink (Terraform documentation)

• Data Source: aws_oam_link (Terraform documentation)

• CloudWatchObservabilityAccessManager (AWS Boto3 documentation)