Option 2: Deploy the AWS CloudFormation template using Amazon EC2, KPL, Kinesis Data Streams, Managed Service for Apache Flink, and CloudWatch

Deploying the aws-streaming-data-solution-for-kinesis-using-kpl-and-kinesis-data-analytics AWS CloudFormation template builds the following environment in the AWS Cloud.

AWS CloudFormation template using Amazon EC2, KPL, Kinesis Data Streams, Managed Service for Apache Flink, and CloudWatch reference architecture

This AWS CloudFormation template deploys a reference architecture that includes the following:

1. An Amazon Elastic Compute Cloud (Amazon EC2) instance that uses the Amazon Kinesis Producer Library (KPL) to generate data.

2. Kinesis Data Streams to store the incoming streaming data.

3. Managed Service for Apache Flink Studio processes the incoming records and saves the processed data in an Amazon Simple Storage Service (Amazon S3) bucket.

4. An Amazon CloudWatch dashboard monitors application health, progress, resource utilization, events, and errors. For information on essential metrics including recommended alarms, refer to Using CloudWatch Alarms with Amazon Managed Service for Apache Flink for Apache Flink in the Managed Service for Apache Flink Developer Guide.

5. Kinesis Data Analytics Studio uses an AWS Glue Data Catalog to store metadata tables representing data stores.

Note

You can configure the solution to use an existing Amazon Virtual Private Cloud (Amazon VPC) to allow access to private resources (for example, private databases). By default, Managed Service for Apache Flink does not launch into your VPC, but you can allow your applications to access Amazon Relational Database Service (Amazon RDS) using this solution. For information about connecting to a virtual private cloud in your account, refer to Configuring Managed Service for Apache Flink in the Managed Service for Apache Flink Developer Guide.