Creating Managed Service for Apache Flink applications with Apache Beam

Note

Apache Beam is not supported in Apache Flink version 1.18. For more information, see Flink Version Compatibility in the Apache Beam Documentation.>

You can use the Apache Beam framework with your Managed Service for Apache Flink application to process streaming data. Managed Service for Apache Flink applications that use Apache Beam use Apache Flink runner to execute Beam pipelines.

For a tutorial about how to use Apache Beam in a Managed Service for Apache Flink application, see Using CloudFormation with Managed Service for Apache Flink.

This topic contains the following sections:

• Using Apache Beam with Managed Service for Apache Flink
• Beam capabilities
• Creating an application using Apache Beam

Using Apache Beam with Managed Service for Apache Flink

Note the following about using the Apache Flink runner with Managed Service for Apache Flink:

• Apache Beam metrics are not viewable in the Managed Service for Apache Flink console.

• Apache Beam is only supported with Managed Service for Apache Flink applications that use Apache Flink version 1.8 and above. Apache Beam is not supported with Managed Service for Apache Flink applications that use Apache Flink version 1.6.

Beam capabilities

Managed Service for Apache Flink supports the same Apache Beam capabilties as the Apache Flink runner. For information about what features are supported with the Apache Flink runner, see the Beam Compatibility Matrix.

We recommend that you test your Apache Flink application in the Managed Service for Apache Flink service to verify that we support all the features that your application needs.