Architecture overview - DevOps Monitoring Dashboard on AWS

Architecture overview

Deploying this solution with the default parameters builds the following environment in the AWS Cloud.

Architecture Diagram

Figure 1: DevOps Monitoring Dashboard on AWS architecture

This solution runs the following workflow:

  1. A developer initiates an activity in an AWS CI/CD pipeline, such as pushing a code change to AWS CodeCommit or deploying an application using AWS CodeDeploy. These activities create events. For development using GitHub repository, git push events are generated.

  2. An Amazon EventBridge events rule detects the events based on predefined event patterns and then sends the event data to an Amazon Kinesis Data Firehose delivery stream. One event rule is created per event source. For activities in AWS CodeBuild, a CloudWatch metric stream is set up to capture CloudWatch metrics and deliver them to a Kinesis Data Firehose delivery stream. For GitHub push events, an Amazon API endpoint is created to post these events and deliver them to a Kinesis Data Firehose delivery stream.

  3. An Amazon EventBridge events rule is also created to capture events from an Amazon CloudWatch alarm that monitors the status of an Amazon CloudWatch synthetics canary, if you have set up the canary and alarm in your account. This alarm is needed to gather data for calculating Mean Time to Recover (MTTR) metrics.

  4. Amazon Kinesis Data Firehose uses an AWS Lambda function for data transformation. The Lambda function extracts relevant data to each metric and sends it to an Amazon S3 bucket for downstream processing.

  5. The data in Amazon S3 is linked to an Amazon Athena database, which runs queries against this data and returns query results to Amazon QuickSight.

  6. Amazon QuickSight obtains the query results and builds dashboard visualizations for your management team.