VTAP not working when NAT Gateway is enabled

Question

We are doing a POC with various NDR solutions on Azure, by using VTAP to forward a copy of network packets to the NDR sensors from a VM NIC. Everything works fine until the NAT Gateway is enabled. We stopped seeing the expected traffic coming into the various NDR solutions. Once we disabled the NAT Gateway, everything works fine again. Not sure what am I missing here.

Answer 1

Hello Nickson,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that your VTAP not working when NAT Gateway is enabled.

Your test show that this is not an NDR-specific issue: the same result occurs with multiple NDR products and an independent Windows collector, an any-to-any NSG rule does not change the result, and removing NAT Gateway restores bidirectional mirrored traffic. The evidence therefore points to a vTAP and NAT Gateway datapath-interoperability issue, but the exact Azure platform root cause has not yet been proven. vTAP is intended to stream traffic from a VM NIC to a collector, while NAT Gateway provides subnet-level SNAT for outbound connections and their return traffic. - https://learn.microsoft.com/en-us/azure/virtual-network/virtual-network-tap-overview, https://learn.microsoft.com/en-us/azure/nat-gateway/nat-overview

The best practice method is to:

• Disassociate NAT Gateway from the monitored VM’s subnet, or move the monitored VM to a dedicated subnet without NAT Gateway.
• Keep the existing vTAP source NIC and collector configuration unchanged and confirm that bidirectional capture is restored.
• If NAT Gateway must remain, use Azure Network Watcher packet capture on the source VM or a supported host-based NDR sensor as the temporary source of complete packet visibility. - https://learn.microsoft.com/en-us/azure/network-watcher/packet-capture-overview
• Perform synchronized packet captures on the source VM and vTAP collector, first without NAT Gateway, then with NAT Gateway, and finally after removing it again.
• Escalate the synchronized captures and exact UTC test times to Azure Support for vTAP backend investigation and Product Group confirmation. Azure Support can collect advanced resource diagnostics that are not accessible to the customer. - https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request

Do not add Azure Firewall, UDRs, broad NSG rules, or additional NAT rules as a speculative fix. NAT Gateway does not require a UDR, and the customer has already demonstrated that relaxing the NSG does not restore the missing return copies. - https://learn.microsoft.com/en-us/azure/nat-gateway/nat-overview

After removing NAT Gateway from the monitored subnet, bidirectional vTAP visibility should return based on your existing A/B test. If NAT Gateway and vTAP must coexist, the permanent resolution requires Microsoft Internal Support to confirm and correct the Azure-managed replication behavior. Until that confirmation is received, the affected design should not be approved for production NDR monitoring.

I hope this is helpful. Please! Do not hesitate to let me know if you have any other questions, steps or clarifications.

---

Please do not close the thread by upvoting and accepting the answer if any part of it is helpful.

Answer 2

Hi @Jose Benjamin Solis Nolasco , thanks for the information. I have a Windows 11 Virtual Machine with its NIC as the vTAP source. And the destination is being sent to my NDR's IP address. All the component reside in a single VNet and Subnet. On the same VNet and subnet, I then enabled the NAT Gateway.

This is all I have in my lab setup. There is no Azure firewall, UDR or NSG blocking the mirror traffic. All resources are running in Azure Singapore.

I have open a Any-to-Any NSG rule and validated it does not make a difference. To rule out that it is not a NDR issue, I also spun up a Windows machine to collect the mirrored traffic from VTAP, and the same behaviour is observed. Below are the screenshots of the PCAP:

Without NAT Gateway:

All traffic is captured as expected.

WIth NAT Gateway:

The response packets are missing from the capture. Since the VTAP source is the Windows 11 VM's NIC, all traffic should be captured even if the return path is altered by NAT Gateway.

Not sure if anyone else is getting the same behaviour from VTAP + NAT Gateway.

Answer 3

Welcome to Microsoft Q&A

Hello @Nickson I hope you are doing well.

Since VTAP works correctly until the NAT Gateway is associated with the subnet, the issue is likely related to how traffic is handled after SNAT is introduced.

A few things to check:

• Verify whether the traffic you expect VTAP to mirror is still traversing the VM NIC. NAT Gateway only provides outbound SNAT, but changes to the outbound path may affect what traffic is available for packet mirroring.
• Review the VTAP configuration to ensure it is attached to the correct NIC and that the filter configuration still matches the traffic after enabling NAT Gateway.
• Check for NSGs, UDRs, or Azure Firewall that could alter the traffic path when the NAT Gateway is enabled.
• Validate the NDR sensor to confirm it is not filtering or expecting pre-NAT traffic only.

Limitations:

• Virtual Network TAP supports only virtual machine (VM) network interfaces as traffic mirroring sources.
• Mirrored traffic can be sent only to a load balancer or a VM network interface.
• VMs behind a Basic Load Balancer cannot be configured as a mirroring source. Basic Load Balancer is being deprecated.
• Inbound traffic from Private Link Service is not supported for mirroring.
• VMs in a virtual network with encryption enabled cannot be configured as mirroring sources.
• Virtual Network TAP does not support IPv6 or SWIFT.
• Virtual WAN (vWAN) peering is not supported between the source and destination virtual networks used with VTAP. Direct virtual network peering must be used instead.

Public Preview limitations

• v6 VM SKUs are not supported as source VMs.
• Before adding a VM as a source, you must first deploy a Virtual Network TAP resource, and then stop (deallocate) and start the source VM. This is required only once per VM that will be used as a source. If this step is not completed, you may receive an error indicating that the NIC is not on fastpath.
• When a VM is added or removed as a source, the VM may experience network downtime of up to 60 seconds.
• Live Migration is not supported for source VMs. Live Migration will be disabled for any VM configured as a source.

Supported Regions

• Asia East
• Southeast Asia
• Canada Central
• West Europe
• Germany West Central
• Central India
• Korea Central
• UAE North
• UK South
• Central US
• Central US EUAP
• East US
• East US 2
• East US 2 EUAP
• West US 3

  Public Preview limitations

  • v6 VM SKUs are not supported as source VMs.
    • Before adding a VM as a source, you must first deploy a Virtual Network TAP resource, and then stop (deallocate) and start the source VM. This is required only once per VM that will be used as a source. If this step is not completed, you may receive an error indicating that the NIC is not on fastpath.
      • When a VM is added or removed as a source, the VM may experience network downtime of up to 60 seconds.
        • Live Migration is not supported for source VMs. Live Migration will be disabled for any VM configured as a source.

  Supported Regions

  • Asia East
  • Southeast Asia
  • Canada Central
  • West Europe
  • Germany West Central
  • Central India
  • Korea Central
  • UAE North
  • UK South
  • Central US
  • Central US EUAP
  • East US
  • East US 2
  • East US 2 EUAP
  • West US 3

References

• Azure Virtual Network TAP (VTAP): https://learn.microsoft.com/azure/virtual-network/virtual-network-tap-overview
• NAT Gateway overview: https://learn.microsoft.com/azure/nat-gateway/nat-overview
• NAT Gateway FAQ: https://learn.microsoft.com/azure/nat-gateway/faq

If my answer helped you, please consider marking it as accepted. This helps others in the community find similar solutions.