Apache Airflow versions on Amazon Managed Workflows for Apache Airflow - Amazon Managed Workflows for Apache Airflow

Apache Airflow versions on Amazon Managed Workflows for Apache Airflow

This page describes the Apache Airflow versions Amazon Managed Workflows for Apache Airflow supports and the strategies we recommend to upgrade to the latest version.

About Amazon MWAA versions

Amazon MWAA builds container images that bundle Apache Airflow releases with other common binaries and Python libraries. The image uses the Apache Airflow base install for the version you specify. When you create an environment, you specify an image version to use. Once an environment is created, it keeps using the specified image version until you upgrade it to a later version.

Latest version

Amazon MWAA supports more than one Apache Airflow version. If you do not specify an image version when you create an environment, Amazon MWAA creates an environment using the latest supported version of Apache Airflow.

Apache Airflow versions

The following Apache Airflow versions are supported on Amazon Managed Workflows for Apache Airflow.

Note
  • Beginning with Apache Airflow v2.2.2, Amazon MWAA supports installing Python requirements, provider packages, and custom plugins directly on the Apache Airflow web server.

  • Beginning with Apache Airflow v2.7.2, your requirements file must include a --constraint statement. If you do not provide a constraint, Amazon MWAA will specify one for you to ensure the packages listed in your requirements are compatible with the version of Apache Airflow you are using.

    For more information on setting up constraints in your requirements file, see Installing Python dependencies.

For more information about migrating your self-managed Apache Airflow deployments, or migrating an existing Amazon MWAA environment, including instructions for backing up your metadata database, see the Amazon MWAA Migration Guide.

Apache Airflow components

This section describes the number of Apache Airflow schedulers and workers available for each Apache Airflow version on Amazon MWAA, and provides a list of key Apache Airflow features, indicating the version that supports each feature.

Schedulers

Apache Airflow version Scheduler (default) Scheduler (min) Scheduler (max)

Apache Airflow v2 and above

2

2

5

Workers

Airflow version Workers (min) Workers (max) Workers (default)

Apache Airflow v2

1

25

10

Upgrading the Apache Airflow version

Amazon MWAA supports minor version upgrades. This means you can upgrade your environment from version x.1.z to x.2.z, but no to a new major version, for example, from 1.y.z to 2.y.z.

Note

You cannot downgrade the Apache Airflow version for your environment.

For more information, and detailed instructions on updating your workflow resources, and upgrading the environment to a new version, see Upgrading the Apache Airflow version.

Apache Airflow deprecated versions

The following table lists the deprecated versions of Apache Airflow in Amazon MWAA, along with initial release and end of support dates for each version. For more information about migrating to a newer version, see the Amazon MWAA Migration Guide.

Apache Airflow version Apache Airflow release date Amazon MWAA availability date Amazon MWAA limited support date Amazon MWAA end of support date
v1.10.12 August 25, 2020 November 24, 2020 August 21, 2023 February 21, 2024
v2.0.2 April 19, 2021 May 25, 2021 November 23, 2023 April 29, 2024
v2.2.2 November 15, 2021 January 27,2022 January 25, 2024 June 27, 2024

Apache Airflow version support and FAQ

In accordance with the Apache Airflow community release process and version policy, Amazon MWAA is committed to supporting at least three minor versions of Apache Airflow at any given time. We will announce the end of support date of a given Apache Airflow minor version at least 90 days before the end of support date.

Frequently asked questions

Q: How long does Amazon MWAA support an Apache Airflow version?

A: Amazon MWAA supports an Apache Airflow minor version for a minimum of 12 months after first being available.

Q: Am I notified when support is ending for an Apache Airflow version on Amazon MWAA?

A: Yes. If any Amazon MWAA environments in your account run the version nearing the end of support, Amazon MWAA sends out a notice through the AWS Health Dashboard with the end of support date.

Q: What happens on the limited support date?

A: On the limited support date, you can no longer create new Amazon MWAA environments with the associated version. Your existing environments will continue to be available until the end of support date.

Q: What happens on the end of support date?

A: On the end of support date, you will no longer be able to access existing Amazon MWAA environments that run the associated, deprecated version of Apache Airflow. For instructions on upgrading to a newer version of Apache Airflow on Amazon MWAA, see the Amazon MWAA Migration Guide.