VM's cannot communicate on Same Hyper-V SR-IOV enabled SET team

I C 21 Reputation points
2022-04-21T21:21:54.61+00:00

Hi Folks,

I have a couple of 2-node Hyper-V clusters running on Win2k22. They're fairly new Dells using Intel X710 10 Gb NIC's. SR-IOV is enabled on each adapter in the firmware. I've run Get-NetAdapterSriov on all of them, and all adapters show enabled and supported in the output. I've created a SET team of two of these adapters to use for the virtual switch (external network) on each cluster node using powershell, and enabled SR-IOV on each of them. One cluster is working great. I've enabled SR-IOV on each VM's adapter and everything is hunky dory.

The other cluster is where it gets weird. I have 3 VM's, all Windows server 2016+ that cannot communicate with each other when they are running on the same cluster node and hence using the same virtual switch. They can communicate with everything else in the network (even other SR-IOV enabled VM's on the same cluster node), but not each other. If I move one of these VM's over to the other cluster node, and hence to a different virtual switch, it can then ping the others, and they it, but if I move the remaining two over to the same node, they lose connectivity again.

If I disable one of the two adapters in the SET team, they start working. It doesn't matter which one. As long as the team only has one adapter in it, they have connectivity. The teams are switch independent dynamic teams, and nothing special has been configured on the switch ports (Cisco 9300). I've also tried changing the teams to hypervport type, but that didn't help. The only thing that will allow them to communicate with each other is if I disable SR-IOV on all the VM's NIC's (not just one). It seems as though since every other VM is working fine but these three, it's something related to these specific VM's, but what? I tried removing the NIC from one of the VM's and adding it back again, but no soap.

I'm kinda stumped on this one. Anyone got any thoughts?

Thanks,

Ian

Hyper-V
Hyper-V
A Windows technology providing a hypervisor-based virtualization solution enabling customers to consolidate workloads onto a single server.
2,534 questions
0 comments No comments
{count} votes