Windows Server Failover Cluster 2019 and SQL 2016 Availability Groups

Question

Hello everyone,

I am facing a very strange issue with the WSFC when I try to add additional network cards to the nodes for iSCSI. Here are the details:

I have 3 Windows Server 2019 nodes running SQL 2016 AG. #1 and #2 are in site A and #3 in a DR site. Each node has 2 NICs configured as follows:

SITE A

Node1:
NIC1 (VLAN A) Cluster and Client
NIC2 (VLAN B) Cluster Only
Node2:
NIC1 (VLAN A) Cluster and Client
NIC2 (VLAN B) Cluster Only

DR

Node3:
NIC1 (VLAN C) Cluster and Client
NIC2 (VLAN D) Cluster Only

Using a FileShare witness and Local disks. These are Virtual machines with local VMDKs on VMWare cluster.

With this configuration above, I have no problems. All nodes are online and I can failover the AGs from Node 1 to Node 2 and back without problems. Node 3 since it is in DR, it has no vote (set to 0) to allow DR process if Node 1 and Node 2 go down as described in here (318218

I can also reboot Node 1 and the AGs will automatically failover to Node 2 and vice-versa.

Here is where the fun part starts. A decision was made to add iSCSI disks to the guest and use iSCSI disks to store the SQL Databases (I know! Don't ask. I am not a fan of iSCSI to a guest VM but let's just treat it as if they are physical servers for a moment).

I added a 3rd NIC on all three servers and configured for the iSCSI VLAN. Set an IP address + Subnet and no GW. Moments later Node 2 and Node 3 show to be down in WSFC. The network are not really down as I am still able to remote access them via RDP. So, this tells me NIC 1 still works on all of them. I made sure the newly added NICs are set to None for cluster traffic but it makes no difference.

If I enable the iSCSI NIC on Node 3 and set to None all is still fine
If I then enable the iSCSI NIC on Node 2 then Node 3 shows Down and all network interfaces also show down in WSFC
If I enable iSCSI NIC on Node 1 then Node 2 shows down and all network interfaces also show down in WSFC.
If I set any of displayed iSCSI NICs in WSFC to None for cluster traffic, it does not make a difference.

So far, the only way I have been able to bring all nodes back online "Joined" in WSFC is by disabling the iSCSI NIC or if I enable ONLY one ISCSI NIC from any of the 3 servers.

Answer 1

Bump! Anyone?