Architecture details

This section describes the components and AWS services that make up this solution and the architecture details on how these components work together.

AWS services in this solution

AWS service	Description
Amazon EventBridge	Core. Creates one EventBridge events rule for each data source to capture events.
Amazon Data Firehose	Core. Delivers data to Amazon S3 buckets.
AWS Lambda	Core. Deploys multiple Lambda functions to transform raw data, run queries and create QuickSight dashboard.
Amazon S3	Core. Object storage service offering industry-leading scalability, data availability, security, and performance.
Amazon CloudWatch	Core. Stores transformed data.
AWS Glue	Core. Creates catalog for Athena database.
Amazon Athena	Core. Creates Athena views, queries and work groups to support QuickSight dashboard visualization.
Amazon QuickSight	Core. Creates dashboard to visualized data.
Amazon API Gateway	Optional. Creates an API endpoint that receives GitHub events.
AWS Secrets Manager	Optional. Creates a secret token that is used to authenticate GitHub requests.
AWS Identity and Access Management	Supporting. Creates IAM roles and permissions used by AWS services to perform operations.
AWS Systems Manager	Supporting. Provides application-level resource monitoring and visualization of resource operations and cost data.

Amazon EventBridge events rule

This solution creates one Amazon EventBridge events rule for each data source as follows:

• CodeCommit events rule - This rule is invoked by AWS CodeCommit events that match a predefined event pattern for code pushes. It routes the events to a target Amazon Kinesis Data Firehose delivery stream for processing.

• CodeDeploy events rule - This rule is invoked by AWS CodeDeploy events that match a predefined event pattern for code deployment state changes. It routes the events to a target Amazon Kinesis Data Firehose delivery stream for processing.

• CodePipeline events rule - This rule is invoked by AWS CodePipeline events that match a predefined event pattern for changes in CodePipeline action run states. This rule routes the events to a target Kinesis Data Firehose delivery stream for processing.

• Canary events rule - This rule is invoked by Amazon CloudWatch Alarm events that match a predefined event pattern for an alarm linked to an Amazon CloudWatch Synthetics Canary, which monitors your endpoints and APIs. It routes the events to a target Amazon Kinesis Data Firehose delivery stream for processing.

• Athena partitions events rule - This rule runs on a daily schedule to invoke an AWS Lambda function to add a new daily partition to an Amazon Athena table.

• CodePipeline alarm events rule - This rule is invoked by Amazon CloudWatch Alarm events that match a predefined event pattern for an alarm monitoring the state (FAILED or SUCCEEDED) of an AWS CodePipeline. It routes the events to a target Kinesis Data Firehose delivery stream for processing.