Hi Mohammed,
Thank you for posting on Microsoft Learn.
Managed Airflow in ADF or Microsoft Fabric is a PaaS offering. That means it runs in Microsoft-managed infrastructure, but it can be integrated with a VNet using Managed VNet integration.
The main pros :
- Fully managed, scalable, and monitored.
- Integrated into the Azure ecosystem (ADF pipelines, triggers, logging).
- Lower operational overhead.
Can Managed Airflow run tasks on Azure VMs ?
- Direct execution on VMs: Not out-of-the-box.
- Remote execution via SSH, APIs, or agents: Yes, but you must configure DAGs to connect (for example you can use BashOperator or SSHOperator).
ADF Managed Airflow can run in a managed VNet, which can be linked to your own VNet using Private Endpoints or VNet peering.
Once integrated, your DAGs can securely connect to Azure VMs, databases, or other resources inside the VNet.
You must ensure DNS resolution and routing is set up properly, and the VMs must allow access from the ADF Managed VNet.
If you want maximum control, Airflow on AKS gives:
- Full VNet integration
- Control over how and where the tasks run (you can use KubernetesPodOperator or SSH)
- Easier to use workers inside your VNet, including Azure VMs.
But, don't forget that it requires managing and maintaining AKS, Airflow Helm chart, scaling...