Integrating Azure Data Factory with VNET and Azure Firewall

Question

We're currently working on a use case involving the integration of Azure Data Factory Integration Runtime with a Virtual Network, as well as integration with Azure Firewall to analyze all inbound and outbound packets routed through the firewall.

While ADF utilizes the managed virtual network where the VNET is administered by Microsoft, we've encountered a challenge in controlling routing within it to implement a custom route table for directing outbound calls to Azure Firewall or a virtual appliance. What recommendations can be provided to achieve this?

We've come across an article indicating that all ports are open for outbound connectivity when utilizing a managed VNET, which poses a security risk for some organizations (Reference: https://learn.microsoft.com/en-us/azure/data-factory/managed-virtual-network-private-endpoint#outbound-communications-through-public-endpoint-from-a-data-factory-managed-virtual-network).

One potential workaround is to utilize Self-Hosted Integration Runtime to achieve VNET and firewall integration. However, this approach introduces additional architectural complexity and technical debt for the operations teams, which may not align with our goals.

Answer 1

@Bhushan Gawale

Thanks for reaching out to Microsoft Q&A

You're right, there are limitations with controlling outbound traffic in a managed virtual network for Azure Data Factory (ADF). Here's how to approach this while considering your concerns:

1. Leverage Managed Private Endpoints with Azure Firewall Policies:

• While you can't control routing within the managed VNET, you can still enforce security using Azure Firewall policies.
• Implement Managed Private Endpoints for your data stores within the managed VNET.
• Configure Azure Firewall policies to restrict outbound traffic to only the specific ports and destinations required by ADF to access these data stores via private endpoints.
• This provides granular control over outbound communication while leveraging the benefits of a managed VNET.
• Refer:Managed Virtual Network in Azure Data Factory (ADF

2. Explore Azure Synapse Workspace with Managed VNET:

• Consider using an Azure Synapse Workspace with a managed VNET instead of just ADF.
• Synapse Workspaces offer the option to configure outbound network traffic filters for the managed VNET.
• This allows you to define allowed destinations and ports for ADF activities, achieving some level of control within the managed environment.

3. Hybrid Approach with Self-Hosted Integration Runtime (SHIR):

• If the limitations of managed VNET and the complexity of a full SHIR deployment are drawbacks, explore a hybrid approach.
• Set up a minimal SHIR environment within your VNET. This SHIR instance would primarily act as a jump box for ADF activities.
• Configure Azure Firewall policies to allow outbound traffic only from the SHIR instance to the required data stores and ports.
• This approach introduces some additional management overhead but provides more control compared to a fully managed VNET

refer :Self-Hosted Integration Runtime (SHIR

Custom Route Table: You can create a custom route table to define the route of the traffic through the NVA virtual machine. The route table is associated with the subnet where the virtual machine is deployed..

Azure Firewall Custom Routing: You can create a custom route table to activate VNet-to-Internet and VNet-to-Branch: 0.0.0.0/0 with the next hop pointing to Azure Firewall.

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.