Transparently relay Azure VM interface traffic to a nested VM Azure VM

Question

I am trying to set up OPNSense VM inside a Proxmox, which is running in a Azure VM with nesting enabled. I have my reasons to do it, so please spare me the "why not go native" questions.

Since azure VMs don't support vIOMMU (note the "v" in vIOMMU stands for virtualized IOMMU, for L2 instances), I cannot pass the interface further from Proxmox to OPNSense, so I need to get by using bridges.

The host setup is:

• WAN over eth0
• vmbr0 with eth0 assigned to it
• The bridged interfaces are in promiscuous mode.

The actual configuration is:

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet manual
bridge-ports eth0
bridge-stp off
bridge-fd 0

The guest configuration is:

• VirtIO NIC attached to vmbr0, with MAC overridden using same address as the eth0
• Firewall: NO
• MAC Filter: NO
• Running dhclient on eth0 or vmbr0 correctly discovers and assigns an IP address.

Now, I am trying to get the OPNSense in a VM to get that IP address instead and to relay its traffic via the vmbr0 transparently outside of the host. I have done something very similar previously between OpenWRT running in a VM and another VM, using OpenWRT's "trivial relay" (kmod-trelay, see https://forum.openwrt.org/t/howto-kmod-trelay/49610/2, also https://github.com/openwrt/openwrt/commit/c3bba7f8c61ee98265bcffef8ee86e22aa89bbe9), and despite that this particular case is much simpler, I can't get the VM to communicate with the ISP properly. I tried simply by spoofing the eth0's MAC address by setting the OPNSense VM's interface to it, but that's not enough.

I also checked the traffic on both ends using tcpdump, and, interestingly, vmbr0 does see the DHCP requests coming from the VM, and the ISP does respond, but that response never reaches the VM, nor the tap interface corresponding to the VM that Proxmox assigned to the bridge.

What am I missing here?

Answer 1

The ISP in this case is Azure itself. I fixed it with macvtap driver on L1 host.