Supported Apache Airflow configurations
Note
This feature is in public preview. Workflow Orchestration Manager is powered by Apache Airflow.
In Workflow Orchestration Manager, Apache Airflow configurations can be integrated with the platform's runtime as key-value pairs. While the airflow.cfg
isn't directly accessible in the UI, users can override these configurations via the UI's "Airflow Configuration overrides" section, retaining access to other airflow.cfg
settings. Developers have the flexibility to override most Apache Airflow configurations within Workflow Orchestration Manager, except for those explicitly outlined in a provided table
.
APPLIES TO: Azure Data Factory Azure Synapse Analytics
Tip
Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!
For more information on Apache Airflow configurations, see Configuration Reference.
The following table contains the list of configurations that don't support overrides.
Configuration | Description | Default value |
---|---|---|
AIRFLOW__CELERY__FLOWER_URL_PREFIX | The root URL for Flower. | "" |
AIRFLOW__CORE__DAGS_FOLDER | The path of the folder where Airflow pipelines live. | AIRFLOW_DAGS_FOLDER |
AIRFLOW__CORE__DONOT_PICKLE | Whether to disable pickling DAGs. | False |
AIRFLOW__CORE__ENABLE_XCOM_PICKLING | Whether to enable pickling for xcom. | False |
AIRFLOW__CORE__EXECUTOR | The executor class that Airflow should use. | CeleryExecutor |
AIRFLOW__CORE__FERNET_KEY | Secret key to save connection passwords in the database. | AIRFLOW_FERNET_KEY |
AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION | Are DAGs paused by default at creation? | False |
AIRFLOW__CORE__PLUGINS_FOLDER | Path to the folder that contains Airflow plugins. | AIRFLOW_PLUGINS_FOLDER |
AIRFLOW__LOGGING__BASE_LOG_FOLDER | The folder where Airflow should store its log files. | /opt/airflow/logs |
AIRFLOW__LOGGING__LOG_FILENAME_TEMPLATE | Formatting for how Airflow generates file names or paths for each task run. | {{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ try_number }}.log |
AIRFLOW__LOGGING__DAG_PROCESSOR_MANAGER_LOG_LOCATION | Full path of the dag_processor_manager log file. |
/opt/airflow/logs/dag_processor_manager/dag_processor_manager.log |
AIRFLOW__LOGGING__LOGGING_CONFIG_CLASS | Logging config class specifies the logging configuration. This class has to be on the Python class path. | log_config.LOGGING_CONFIG |
AIRFLOW__LOGGING__COLORED_LOG_FORMAT | Log format for when Colored logs is enabled. | [%(asctime)s] {{%(filename)s:%(lineno)d}} %(levelname)s - %(message)s |
AIRFLOW__LOGGING__LOGGING_LEVEL | Logging level. | INFO |
AIRFLOW__METRICS__STATSD_ON | Enables sending metrics to StatsD. | True |
AIRFLOW__METRICS__STATSD_HOST | Hostname of the StatsD server. | geneva-services |
AIRFLOW__METRICS__STATSD_PORT | Port number of the StatsD server. | 8125 |
AIRFLOW__METRICS__STATSD_PREFIX | Prefix for all Airflow metrics sent to StatsD. | AirflowMetrics |
AIRFLOW__SCHEDULER__CHILD_PROCESS_LOG_DIRECTORY | Path of the directory where the Airflow scheduler writes its child process logs. | /opt/airflow/logs/scheduler |
AIRFLOW__SCHEDULER__DAG_DIR_LIST_INTERVAL | How often (in seconds) to scan the DAGs' directory for new files. Default to 5 minutes. | 5 |
AIRFLOW__WEBSERVER__BASE_URL | The base URL of your website because Airflow can't guess what domain or cname you're using. This URL is used in automated emails that Airflow sends to point links to the right web server. | https://localhost:8080 |
AIRFLOW__WEBSERVER__COOKIE_SAMESITE | Set samesite policy on session cookie. | None |
AIRFLOW__WEBSERVER__COOKIE_SECURE | Set secure flag on session cookie. | True |
AIRFLOW__WEBSERVER__EXPOSE_CONFIG | Expose the configuration file in the web server. | False |
AIRFLOW__WEBSERVER__AUTHENTICATE | Authenticate user to sign in to the Airflow UI. | True |
AIRFLOW__WEBSERVER__AUTH_BACKEND | airflow.api.auth.backend.basic_auth | |
AIRFLOW__WEBSERVER__RELOAD_ON_PLUGIN_CHANGE | If set to True, Airflow tracks files in the plugins_folder directory. When it detects changes, then reload the gunicorn. |
True |
AIRFLOW__WEBSERVER__SECRET_KEY | Secret key used to run your flask app. | AIRFLOW_FERNET_KEY |
AIRFLOW__API__AUTH_BACKEND | Comma-separated list of auth backends to authenticate users of the API. | airflow.api.auth.backend.basic_auth |
AIRFLOW__API__ENABLE_EXPERIMENTAL_API | True |