Overview - Amazon CloudWatch Monitoring Framework

Overview

The Amazon CloudWatch Monitoring Framework solution automates the process of setting up an Amazon CloudWatch dashboard so that you can instantly monitor key performance metrics and logs for your Apache web servers running on AWS. This solution provides a pre-configured dashboard so you can focus on analyzing web traffic patterns, determining to scale servers up or out, and detect bottlenecks and other performance problems for your Apache workloads.

This solution is built on Amazon CloudWatch and uses several CloudWatch features, such as metrics, metrics explorer, logs insights and dashboards to automate the dashboard set up process, and provide you with a centralized view of your workload performance. This solution also provides demo resources, such as a demo EC2 instance, so that you can preview how this solution works.

This solution provides the following key features:

  • Pre-configured dashboard for Apache workloads

  • Automated process to set up an Amazon CloudWatch dashboard

  • Tagging mechanism to add or remove instances from the dashboard

  • Amazon CloudWatch agent configuration files for specific workloads

This guide provides detailed instructions for planning and deploying the Amazon CloudWatch Monitoring Framework in the AWS Cloud.

Cost

You are responsible for the cost of the AWS services used while running this solution. As of April 2021, the cost for running this solution with the default settings in the N. Virginia is approximately $3.16 per month.

This cost includes estimated charges for Amazon CloudWatch, AWS Lambda and AWS Systems Manager Parameter Store.

AWS Service Dimension Total Cost
Amazon CloudWatch 1 Dashboard $3.00
AWS Lambda 15,000 requests with average billed duration of 500ms and memory allocated 512MB $0.06
AWS SSM 1 Advanced parameter storage cost + 10,000 API interactions cost $0.10
Total $3.16

If you choose to deploy the demo resources, this solution launches a sample Apache web server that runs on an Amazon EC2 instance. The Amazon EC2 instance sends metrics and logs to Amazon CloudWatch. If the instance runs for one month and generates 10 GB of log data with 1GB of data analyzed by CloudWatch Logs Insight queries, the estimated cost would be as shown in the table below.

AWS Service Dimension Total Cost
Amazon CloudWatch 10GB Logs data with 1GB analyzed with CloudWatch Logs Insights queries $5.10
Amazon EC2 t3.micro instance with 8GB EBS (gp2) storage $8.40
Total $13.50

Prices are subject to change. For full details, refer to the pricing webpage for each AWS service you will be using in this solution.

Architecture overview

This solution is a reference architecture that automates the set up an Amazon CloudWatch dashboard to monitor key performance metrics and logs from Apache web servers. This solution deploys a pre-configured dashboard for Apache workloads.

This solution uses tagging mechanisms to add or remove instances from the monitoring dashboard. When you deploy the solution, you provide the tag (key, value pair) as a parameter to identify your instances. An AWS Lambda function runs on cron schedule to search for instances with the provided tag. The Lambda function maintains the list of instances in an AWS Systems Manager Parameter Store. Whenever the instance list on SSM Parameter store gets updated, it invokes an Amazon CloudWatch Events rule. This rule uses Lambda to update the deployed dashboard with metrics and logs for the new instance list.

Note

Before you add designated tags to the instance, ensure that you configure your instances correctly. For more information, refer to Configuring your Apache EC2 instance. It is important to refer to the CloudWatch agent and httpd config files when configuring your instances. Failing to do so will result in missing metrics and logs from dashboard.

As illustrated in Figure 1, the deployed solution provides an Apache server demo instance and Amazon CloudWatch dashboard to monitor your Apache workload. As of the date of publication, this solution only supports Apache. However, you can customize this solution to work with other workloads. For more information on how to extend this solution to other workloads, refer to the README.md file in the GitHub repository.

Figure 1: Deployment diagram

Figure 1: Deployment diagram

Deploying this solution with the default parameters builds the following environment in the AWS Cloud. The architecture can be broken down into two workflows: User Interaction and Dashboard Management

Amazon CloudWatch Monitoring Framework architecture

Figure 2: Amazon CloudWatch Monitoring Framework architecture

User Interaction Workflow:

The AWS CloudFormation template deploys the Amazon CloudWatch Events rule, AWS Lambda function and AWS Systems Manager Parameter Store necessary to capture Apache instances in your account.

  1. Users update Tag on their instances with the tagging schema provided as input during the stack deployment.

  2. The CloudWatch Events rule invokes the tagHandler Lambda function on cron schedule. The Lambda function primarily performs two tasks:

    • It uses the ec2 describe-tags API call to get instances with the desired tag value (same as step 1).

    • It compares the fetched instance list with the SSM Parameter Store instance list and updates the parameter as needed.

Dashboard Management Workflow:

The AWS CloudFormation template deploys the Amazon CloudWatch Events rule and AWS Lambda function necessary to configure workload specific Amazon CloudWatch dashboard in your account.

  1. When the SSM Parameter Store gets updated, the event is captured using CloudWatch Events rule to invoke the dashboardHandler Lambda function.

  2. The dashboardHandler Lambda function updates the deployed dashboard widgets with logs and metrics for the updated instance list.

Note

AWS CloudFormation resources are created from AWS Cloud Development Kit (CDK) constructs.