Supported Apache Airflow configurations

Note

This feature is in public preview. Workflow Orchestration Manager is powered by Apache Airflow.

In Workflow Orchestration Manager, Apache Airflow configurations can be integrated with the platform's runtime as key-value pairs. While the airflow.cfg isn't directly accessible in the UI, users can override these configurations via the UI's "Airflow Configuration overrides" section, retaining access to other airflow.cfg settings. Developers have the flexibility to override most Apache Airflow configurations within Workflow Orchestration Manager, except for those explicitly outlined in a provided table.

APPLIES TO: Azure Data Factory Azure Synapse Analytics

Tip

Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!

For more information on Apache Airflow configurations, see Configuration Reference.

The following table contains the list of configurations that don't support overrides.

Configuration Description Default value
AIRFLOW__CELERY__FLOWER_URL_PREFIX The root URL for Flower. ""
AIRFLOW__CORE__DAGS_FOLDER The path of the folder where Airflow pipelines live. AIRFLOW_DAGS_FOLDER
AIRFLOW__CORE__DONOT_PICKLE Whether to disable pickling DAGs. False
AIRFLOW__CORE__ENABLE_XCOM_PICKLING Whether to enable pickling for xcom. False
AIRFLOW__CORE__EXECUTOR The executor class that Airflow should use. CeleryExecutor
AIRFLOW__CORE__FERNET_KEY Secret key to save connection passwords in the database. AIRFLOW_FERNET_KEY
AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION Are DAGs paused by default at creation? False
AIRFLOW__CORE__PLUGINS_FOLDER Path to the folder that contains Airflow plugins. AIRFLOW_PLUGINS_FOLDER
AIRFLOW__LOGGING__BASE_LOG_FOLDER The folder where Airflow should store its log files. /opt/airflow/logs
AIRFLOW__LOGGING__LOG_FILENAME_TEMPLATE Formatting for how Airflow generates file names or paths for each task run. {{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ try_number }}.log
AIRFLOW__LOGGING__DAG_PROCESSOR_MANAGER_LOG_LOCATION Full path of the dag_processor_manager log file. /opt/airflow/logs/dag_processor_manager/dag_processor_manager.log
AIRFLOW__LOGGING__LOGGING_CONFIG_CLASS Logging config class specifies the logging configuration. This class has to be on the Python class path. log_config.LOGGING_CONFIG
AIRFLOW__LOGGING__COLORED_LOG_FORMAT Log format for when Colored logs is enabled. [%(asctime)s] {{%(filename)s:%(lineno)d}} %(levelname)s - %(message)s
AIRFLOW__LOGGING__LOGGING_LEVEL Logging level. INFO
AIRFLOW__METRICS__STATSD_ON Enables sending metrics to StatsD. True
AIRFLOW__METRICS__STATSD_HOST Hostname of the StatsD server. geneva-services
AIRFLOW__METRICS__STATSD_PORT Port number of the StatsD server. 8125
AIRFLOW__METRICS__STATSD_PREFIX Prefix for all Airflow metrics sent to StatsD. AirflowMetrics
AIRFLOW__SCHEDULER__CHILD_PROCESS_LOG_DIRECTORY Path of the directory where the Airflow scheduler writes its child process logs. /opt/airflow/logs/scheduler
AIRFLOW__SCHEDULER__DAG_DIR_LIST_INTERVAL How often (in seconds) to scan the DAGs' directory for new files. Default to 5 minutes. 5
AIRFLOW__WEBSERVER__BASE_URL The base URL of your website because Airflow can't guess what domain or cname you're using. This URL is used in automated emails that Airflow sends to point links to the right web server. https://localhost:8080
AIRFLOW__WEBSERVER__COOKIE_SAMESITE Set samesite policy on session cookie. None
AIRFLOW__WEBSERVER__COOKIE_SECURE Set secure flag on session cookie. True
AIRFLOW__WEBSERVER__EXPOSE_CONFIG Expose the configuration file in the web server. False
AIRFLOW__WEBSERVER__AUTHENTICATE Authenticate user to sign in to the Airflow UI. True
AIRFLOW__WEBSERVER__AUTH_BACKEND airflow.api.auth.backend.basic_auth
AIRFLOW__WEBSERVER__RELOAD_ON_PLUGIN_CHANGE If set to True, Airflow tracks files in the plugins_folder directory. When it detects changes, then reload the gunicorn. True
AIRFLOW__WEBSERVER__SECRET_KEY Secret key used to run your flask app. AIRFLOW_FERNET_KEY
AIRFLOW__API__AUTH_BACKEND Comma-separated list of auth backends to authenticate users of the API. airflow.api.auth.backend.basic_auth
AIRFLOW__API__ENABLE_EXPERIMENTAL_API True