Managing connections to Apache Airflow

This section describes the different ways to configure an Apache Airflow connection for an Amazon Managed Workflows for Apache Airflow environment.

Topics

• Overview of Apache Airflow variables and connections
• Apache Airflow provider packages installed on Amazon MWAA environments
• Overview of connection types
• Configuring an Apache Airflow connection using a AWS Secrets Manager secret

Overview of Apache Airflow variables and connections

In some cases, you may want to specify additional connections or variables for an environment, such as an AWS profile, or to add your execution role in a connection object in the Apache Airflow metastore, then refer to the connection from within a DAG.

• Self-managed Apache Airflow. On a self-managed Apache Airflow installation, you set Apache Airflow configuration options in airflow.cfg.

  [secrets]
  backend = airflow.providers.amazon.aws.secrets.secrets_manager.SecretsManagerBackend
  backend_kwargs = {"connections_prefix" : "airflow/connections", "variables_prefix" : "airflow/variables"}
  

• Apache Airflow on Amazon MWAA. On Amazon MWAA, you need to add these configuration settings as Apache Airflow configuration options on the Amazon MWAA console. Apache Airflow configuration options are written as environment variables to your environment and override all other existing configurations for the same setting.