Set up AWS CloudFormation drift detection in a multi-Region, multi-account organization

Created by Ram Kandaswamy (AWS)

Environment: Production	Technologies: Multi account strategy; Infrastructure; Modernization	Workload: All other workloads
AWS services: Amazon SNS; AWS Config; AWS Lambda; AWS CloudFormation

Summary

Customers on Amazon Web Services (AWS) are often looking for an efficient way to detect resource configuration mismatches, including drift in AWS CloudFormation stacks, and fix them as soon as possible. This is especially the case when AWS Control Tower is used.

This pattern provides a prescriptive solution that efficiently solves the problem by using consolidated resource configuration changes and acting on those changes to generate results. The solution is designed for scenarios where there are several AWS CloudFormation stacks created in more than one AWS Region, or in more than one account, or a combination of both. The goals of the solution are the following:

• Simplify the drift detection process

• Set up notification and alerting

• Set up consolidated reporting

Prerequisites and limitations

Prerequisites 

• AWS Config enabled in all the Regions and accounts that must be monitored

Limitations 

• The report generated supports only the comma-separated values (CSV) and JSON output formats.

Architecture

The following diagram shows AWS Organizations set up with multiple accounts. AWS Config rules communicate between the accounts.  

The workflow includes the following steps:

1. The AWS Config rule detects drift.

2. Drift detection results that are found in other accounts are sent to the management account.

3. The Amazon CloudWatch rule calls an AWS Lambda function.

4. The Lambda function queries the AWS Config rule for aggregated results.

5. The Lambda function notifies Amazon Simple Notification Service (Amazon SNS), which sends email notification of the drift.

Automation and scale

The solution presented here can scale for both additional Regions and accounts.

Tools

AWS services

AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time.

Amazon CloudWatch helps you monitor the metrics of your AWS resources and the applications you run on AWS in real time.

AWS Lambda is a compute service that helps you run code without needing to provision or manage servers. It runs your code only when needed and scales automatically, so you pay only for the compute time that you use.

Amazon Simple Notification Service (Amazon SNS) helps you coordinate and manage the exchange of messages between publishers and clients, including web servers and email addresses.

Epics

Automate drift detection for AWS CloudFormation

Task	Description	Skills required
Create the aggregator.	Sign in to the AWS Management Console and open the AWS Config console at https://console.aws.amazon.com/config. Create an aggregator in the management account. Ensure that data replication is turned on so that AWS Config can fetch data from the source accounts. Select all applicable Regions and accounts. You can select accounts based on AWS Organizations. We recommend this approach because new accounts in the organization are automatically part of the aggregator.	Cloud architect
Create an AWS managed rule.	Add the cloudformation-stack-drift-detection-check AWS managed rule. The rule needs one parameter value: cloudformationArn. Enter the IAM role Amazon Resource Name (ARN) that has permissions to detect stack drift. The role must have a trust policy that enables AWS Config to assume the role.	Cloud architect
Create the advanced query section of the aggregator.	To fetch drifted stacks from multiple sources, create the following query: SELECT resourceId, configuration.driftInformation.stackDriftStatus WHERE resourceType = 'AWS::CloudFormation::Stack'  AND configuration.driftInformation.stackDriftStatus IN ('DRIFTED')	Cloud architect, Developer
Automate running the query and publish.	Create a Lambda function by using the code that is attached. Lambda will publish the results to an SNS topic that is provided as an environment variable in the Lambda function. To receive alerts, create an email subscription to the SNS topic.	Cloud architect, Developer
Create a CloudWatch rule.	Create a schedule-based CloudWatch rule to call the Lambda function, which is responsible for alerting.	Cloud architect

Related resources

Resources

• What Is AWS Config?

• Multi-account multi-Region data aggregation

• Detecting unmanaged configuration changes to stacks and resources

• IAM: Pass an IAM role to a specific AWS service

• What is Amazon SNS?

Additional information

Considerations

We recommend using the solution presented in this pattern instead of using custom solutions that involve API calls at specific intervals to initiate drift detection on each CloudFormation stack or stack set. Custom solutions that use API calls at specific internals can lead to a large number API calls and affect performance. Because of the number of API calls, throttling can occur. Another potential issue is a delay in detection if resource changes are identified based on schedule only.

Because stack sets are made of stacks, you can use this solution. Stack instance details are also available as part of the solution.

Attachments

To access additional content that is associated with this document, unzip the following file: attachment.zip