Getting started with Amazon Managed Service for Apache Flink for Python

This section introduces you to the fundamental concepts of a Managed Service for Apache Flink using Python and the Table API. It describes the available options for creating and testing your applications. It also provides instructions for installing the necessary tools to complete the tutorials in this guide and to create your first application.

Topics

• Getting started with Pyflink - The Python Interpreter for Apache | Amazon Web Services
• Components of a Managed Service for Apache Flink application
• Prerequisites
• Create and run a Managed Service for Apache Flink for Python application
• Clean up AWS resources

Getting started with Pyflink - The Python Interpreter for Apache | Amazon Web Services

Before you begin, we encourage you to watch the following video:

Components of a Managed Service for Apache Flink application

To process data, your Managed Service for Apache Flink application uses a Python application that processes input and produces output using the Apache Flink runtime.

Managed Service for Apache Flink application has the following components:

• Runtime properties: You can use runtime properties to configure your application without recompiling your application code.

• Table Source: The application consumes data by using a source. A source connector reads data from a Kinesis data stream, an Amazon MSK topic, or similar. For more information, see Table API sources.

• Functions: The application processes data by using one or more functions. A function can transform, enrich, or aggregate data.

• Sink: The application produces data to external sources by using sinks. A sink connector writes data to a Kinesis data stream, a Firehose Firehose stream, an Amazon MSK topic, an Amazon S3 bucket, and so on. For more information, see Table API sinks.

After you create and package your application code, you upload the code package to an Amazon S3 bucket. You then create a Managed Service for Apache Flink application. You pass in the code package location, a streaming data source, and typically a streaming or file location that receives the application's processed data.

Prerequisites

Before starting this tutorial, complete the first two steps of the Getting started with Amazon Managed Service for Apache Flink (DataStream API):

• Step 1: Set up an AWS account and create an administrator user

• Step 2: Set up the AWS Command Line Interface (AWS CLI)

To get started, see Create an Application.