Solution components - AWS Streaming Data Solution for Amazon MSK

Solution components

Component details for all templates.

Components for option 1: Amazon MSK

CloudWatch dashboards and alerts

Option 1 deploys an Amazon CloudWatch dashboard that monitors the health of the Amazon MSK cluster. You can customize the dashboards and alerts using Amazon CloudWatch or the source code from the solution’s GitHub repository.


        Amazon MSK health metrics on the CloudWatch dashboard, part 1

Figure 5: Amazon MSK health metrics on the CloudWatch dashboard (upper)


        Amazon MSK health metrics on the CloudWatch dashboard, part 2

Figure 6: Amazon MSK health metrics on the CloudWatch dashboard (lower)

Components for option 4: Amazon MSK, Amazon Kinesis Data Analytics, and Amazon S3

Demo consumer application

A consumer is an application that processes data from an Apache Kafka topic. Option 4 includes a demo consumer application, which is a Java application for Amazon Kinesis Data Analytics for Apache Flink that demonstrates how to use several Apache Flink features (such as sources and sinks). For information about customizing the demo consumer application, or replacing it with your own application, refer to the README.md file in the solution’s GitHub repository.

Note

The demo consumer application uses the schema provided in Getting Started with Amazon Kinesis Data Analytics for Apache Flink (Table API) in the Amazon Kinesis Data Analytics Developer Guide.

CloudWatch dashbards and alerts

Option 4 deploys an Amazon CloudWatch dashboard that monitors the health of the Apache Flink application. You can customize the dashboards and alerts using either Amazon CloudWatch, or the source code from the solution’s GitHub repository.


        Application Health on the CloudWatch dashboard

Figure 7: Figure 7: Application Health on the CloudWatch dashboard


        Kafka Source Metrics on the CloudWatch dashboard

Figure 8: Figure 8: Kafka Source Metrics on the CloudWatch dashboard