Amazon Managed Service for Apache Flink 之前称为 Amazon Kinesis Data Analytics for Apache Flink。
本文属于机器翻译版本。若本译文内容与英语原文存在差异,则一律以英文原文为准。
更新 Java 应用程序
按照以下步骤更新 Java 应用程序:
flink-connector-kinesis
如果应用程序使用 flink-connector-kinesis
。
Kinesis 连接器使用阴影将一些依赖项(包括)打包到连接器 jar AWS SDK 中。要更新 AWS SDK版本,请使用以下步骤替换这些阴影类:
- Maven
-
将 Kinesis 连接器和所需 AWS SDK模块添加为项目依赖项。
配置
maven-shade-plugin
:在复制 Kinesis 连接器 jar 的内容时,添加过滤器以排除阴影 AWS SDK类。
按照 Kinesis 连接器的预期,添加重新定位规则以将更新的 AWS SDK类移到包中。
pom.xml
<project> ... <dependencies> ... <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-connector-kinesis</artifactId> <version>1.15.4</version> </dependency> <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>kinesis</artifactId> <version>2.20.144</version> </dependency> <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>netty-nio-client</artifactId> <version>2.20.144</version> </dependency> <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>sts</artifactId> <version>2.20.144</version> </dependency> ... </dependencies> ... <build> ... <plugins> ... <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>3.1.1</version> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> ... <filters> ... <filter> <artifact>org.apache.flink:flink-connector-kinesis</artifact> <excludes> <exclude>org/apache/flink/kinesis/shaded/software/amazon/awssdk/**</exclude> <exclude>org/apache/flink/kinesis/shaded/org/reactivestreams/**</exclude> <exclude>org/apache/flink/kinesis/shaded/io/netty/**</exclude> <exclude>org/apache/flink/kinesis/shaded/com/typesafe/netty/**</exclude> </excludes> </filter> ... </filters> <relocations> ... <relocation> <pattern>software.amazon.awssdk</pattern> <shadedPattern>org.apache.flink.kinesis.shaded.software.amazon.awssdk</shadedPattern> </relocation> <relocation> <pattern>org.reactivestreams</pattern> <shadedPattern>org.apache.flink.kinesis.shaded.org.reactivestreams</shadedPattern> </relocation> <relocation> <pattern>io.netty</pattern> <shadedPattern>org.apache.flink.kinesis.shaded.io.netty</shadedPattern> </relocation> <relocation> <pattern>com.typesafe.netty</pattern> <shadedPattern>org.apache.flink.kinesis.shaded.com.typesafe.netty</shadedPattern> </relocation> ... </relocations> ... </configuration> </execution> </executions> </plugin> ... </plugins> ... </build> </project>
- Gradle
-
将 Kinesis 连接器和所需 AWS SDK模块添加为项目依赖项。
调整 shadowJar 配置:
复制 Kinesis 连接器 jar 的内容时排除阴影 AWS SDK类。
将更新的 AWS SDK类重新定位到 Kinesis 连接器预期的包中。
build.gradle
... dependencies { ... flinkShadowJar("org.apache.flink:flink-connector-kinesis:1.15.4") flinkShadowJar("software.amazon.awssdk:kinesis:2.20.144") flinkShadowJar("software.amazon.awssdk:sts:2.20.144") flinkShadowJar("software.amazon.awssdk:netty-nio-client:2.20.144") ... } ... shadowJar { configurations = [project.configurations.flinkShadowJar] exclude("software/amazon/kinesis/shaded/software/amazon/awssdk/**/*") exclude("org/apache/flink/kinesis/shaded/org/reactivestreams/**/*.class") exclude("org/apache/flink/kinesis/shaded/io/netty/**/*.class") exclude("org/apache/flink/kinesis/shaded/com/typesafe/netty/**/*.class") relocate("software.amazon.awssdk", "org.apache.flink.kinesis.shaded.software.amazon.awssdk") relocate("org.reactivestreams", "org.apache.flink.kinesis.shaded.org.reactivestreams") relocate("io.netty", "org.apache.flink.kinesis.shaded.io.netty") relocate("com.typesafe.netty", "org.apache.flink.kinesis.shaded.com.typesafe.netty") } ...
其他受影响的连接器
如果应用程序使用其他受影响的连接器:
要更新 AWS SDK版本,应在项目生成配置中强制使用该SDK版本。
- Maven
-
将物料 AWS SDK清单 (BOM) 添加到
pom.xml
文件的依赖关系管理部分,以强制执行项目的SDK版本。pom.xml
<project> ... <dependencyManagement> <dependencies> ... <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>bom</artifactId> <version>2.20.144</version> <scope>import</scope> <type>pom</type> </dependency> ... </dependencies> </dependencyManagement> ... </project>
- Gradle
-
添加平台对物料 AWS SDK清单 (BOM) 的依赖关系,以强制执行项目的SDK版本。这需要 Gradle 5.0 或更高版本:
build.gradle
... dependencies { ... flinkShadowJar(platform("software.amazon.awssdk:bom:2.20.144")) ... } ...
Flink 1.15 异步接收器死锁
更新 Python 应用程序