Option 2 - AWS Streaming Data Solution for Amazon MSK

Option 2

Option 2: Deploy the aws-streaming-data-solution-for-msk-using-aws-lambda CloudFormation template

Before you launch this template, review the architecture and other considerations in this guide. Follow the step-by-step instructions in this section to configure and deploy the solution into your account.

Time to deploy: Approximately five minutes

Step 1. Launch the Stack


You are responsible for the cost of the AWS services used while running this solution. Refer to the Cost section for more details. For full details, refer to the pricing webpage for each AWS service used in this solution.

  1. Sign in to the AWS Management Console and use the button below to launch the aws-streaming-data-solution-for-msk-using-aws-lambda.template AWS CloudFormation template.

                                    AWS Streaming Data Solution for Amazon MSK using AWS Lambda
                                        launch button

    Alternatively, you can download the template as a starting point for your own implementation.

  2. The template launches in the US East (N. Virginia) Region by default. To launch this solution in a different AWS Region, use the Region selector in the console navigation bar.

  3. On the Create stack page, verify that the correct template URL shows in the Amazon S3 URL text box and choose Next.

  4. On the Specify stack details page, assign a name to your solution stack. For information about naming character limitations, refer to IAM and STS Limits in the AWS Identity and Access Management User Guide.

  5. Under Parameters, review the parameters for the template and modify them as necessary. This solution uses the following default values.

    Parameter Default Description
    AWS Lambda consumer configuration
    ARN of the MSK cluster


    <Requires input>

    ARN of the Amazon MSK cluster.

    Maximum number of items to retrieve in a single batch


    100 The maximum number of records to retrieve in a single batch. The allowed range is from 1 to 10000.
    Name of a Kafka topic to consume


    <Requires input>

    The name of the Apache Kafka topic to consume.

  6. Choose Next.

  7. On the Configure stack options page, choose Next.

  8. On the Review page, review and confirm the settings. Check the box acknowledging that the template will create AWS Identity and Access Management (IAM) resources.

  9. Choose Create stack to deploy the stack.

    You can view the status of the stack in the AWS CloudFormation console in the Status column. You should receive a CREATE_COMPLETE status in approximately five minutes.