Apache Airflow provider packages installed on Amazon MWAA environments - Amazon Managed Workflows for Apache Airflow

Apache Airflow provider packages installed on Amazon MWAA environments

Amazon MWAA installs provider extras for Apache Airflow v2 and above connection types when you create a new environment. Installing provider packages allows you to view a connection type in the Apache Airflow UI. It also means you don't need to specify these packages as a Python dependency in your requirements.txt file. This page lists the Apache Airflow provider packages installed by Amazon MWAA for all Apache Airflow v2 environments.

Note

For Apache Airflow v2 and above, Amazon MWAA installs Watchtower version 2.0.1 after performing pip3 install -r requirements.txt, to ensure compatibility with CloudWatch logging is not overridden by other Python library installations.

Provider packages for Apache Airflow v2.9.2 connections

When you create an Amazon MWAA environment in Apache Airflow v2.9.2, Amazon MWAA installs the following provider packages used for Apache Airflow connections.

Note

You can specify the latest supported version of apache-airflow-providers-amazon to upgrade this provider. For more information on specifying newer versions, see Specifying newer provider packages.

Provider packages for Apache Airflow v2.8.1 connections

When you create an Amazon MWAA environment in Apache Airflow v2.8.1, Amazon MWAA installs the following provider packages used for Apache Airflow connections.

Note

You can specify the latest supported version of apache-airflow-providers-amazon to upgrade this provider. For more information on specifying newer versions, see Specifying newer provider packages.

Provider packages for Apache Airflow v2.7.2 connections

When you create an Amazon MWAA environment in Apache Airflow v2.7.2, Amazon MWAA installs the following provider packages used for Apache Airflow connections.

Note

You can specify the latest supported version of apache-airflow-providers-amazon to upgrade this provider. For more information on specifying newer versions, see Specifying newer provider packages.

Provider packages for Apache Airflow v2.6.3 connections

When you create an Amazon MWAA environment in Apache Airflow v2.6.3, Amazon MWAA installs the following provider packages used for Apache Airflow connections.

Note

You can specify the latest supported version of apache-airflow-providers-amazon to upgrade this provider. For more information on specifying newer versions, see Specifying newer provider packages.

Provider packages for Apache Airflow v2.5.1 connections

When you create an Amazon MWAA environment in Apache Airflow v2.5.1, Amazon MWAA installs the following provider packages used for Apache Airflow connections.

Note

You can specify the latest supported version of apache-airflow-providers-amazon to upgrade this provider. For more information on specifying newer versions, see Specifying newer provider packages.

Provider packages for Apache Airflow v2.4.3 connections

When you create an Amazon MWAA environment in Apache Airflow v2.4.3, Amazon MWAA installs the following provider packages used for Apache Airflow connections.

Provider packages for Apache Airflow v2.2.2 connections

When you create an Amazon MWAA environment in Apache Airflow v2.2.2, Amazon MWAA installs the following provider packages used for Apache Airflow connections.

Provider packages for Apache Airflow v2.0.2 connections

When you create an Amazon MWAA environment in Apache Airflow v2.0.2, Amazon MWAA installs the following provider packages used for Apache Airflow connections.

Specifying newer provider packages

Beginning with Apache Airflow v2.7.2, your requirements file must include a --constraint statement. If you do not provide a constraint, Amazon MWAA will specify one for you to ensure the packages listed in your requirements are compatible with the version of Apache Airflow you are using.

Apache Airflow constraints files specify the provider versions available at the time of a Apache Airflow release. In many cases, however, newer providers are compatible with that version of Apache Airflow. Because you must use constraints, to specify a newer version of a provider package, you can modify the constraints file for a specific provider version:

  1. Download the version-specific constraints file from https://raw.githubusercontent.com/apache/airflow/constraints-2.7.2/constraints-3.11.txt"

  2. Modify the apache-airflow-providers-amazon version in the constraints file to the version you want to use.

  3. Save the modified constraints file to the Amazon S3 dags folder of your Amazon MWAA environment, for example, as constraints-3.11-updated.txt

  4. Specify your requirements as shown in the following.

    --constraint "/usr/local/airflow/dags/constraints-3.11-updated.txt" apache-airflow-providers-amazon==version-number
    Note

    If you are using a private web server, we recommend you package the required libraries as WHL files by using the Amazon MWAA local-runner.