Multicast in Azure RDMA networks (InfiniBand)

Question

Hi!

I have deployed two Azure HB60rs machines. I can establish a connection, and I can see that there's an SM running as the NIC has a correct value of lid:

hca_id: mlx5_ib0
transport: InfiniBand (0)
fw_ver: 16.28.4000
node_guid: 0015:5dff:fe33:ff1b
sys_image_guid: 9803:9b03:009f:613a
vendor_id: 0x02c9
vendor_part_id: 4120
hw_ver: 0x0
board_id: MT_0000000010
phys_port_cnt: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 4096 (5)
sm_lid: 214
port_lid: 393
port_lmc: 0x00
link_layer: InfiniBand

I've been trying to use the unreliable multicast in RDMA, but it seems that I can't learn in any way what are the multicast groups.

azureuser@test-rdma-3:~/multicast_udp$ saquery -g
ibwarn: [5619] sa_get_handle: umad_open_port on port (null):0 failed
ibpanic: [5619] main: Failed to bind to the SA: Permission denied

azureuser@test-rdma-3:~/multicast_udp$ sudo saquery -g
ibwarn: [5637] sa_query: umad_recv failed: attr 0x38: Connection timed out

Query SA failed: Connection timed out

Other commands don't work as well

azureuser@test-rdma-3:~/multicast_udp$ ibnodes 
ibwarn: [5944] mad_rpc_open_port: can't open UMAD port ((null):0)
/var/tmp/rdma-core/rdma-core-52mlnx1/libibnetdisc/ibnetdisc.c:802; can't open MAD port ((null):0)
/usr/sbin/ibnetdiscover: iberror: failed: discover failed
ibwarn: [5949] mad_rpc_open_port: can't open UMAD port ((null):0)
/var/tmp/rdma-core/rdma-core-52mlnx1/libibnetdisc/ibnetdisc.c:802; can't open MAD port ((null):0)
/usr/sbin/ibnetdiscover: iberror: failed: discover failed
azureuser@test-rdma-3:~/multicast_udp$ sudo ibnodes 
ibwarn: [5965] _do_madrpc: send failed; Invalid argument
ibwarn: [5965] mad_rpc: _do_madrpc failed; dport (DR path slid 0; dlid 0; 0)
/var/tmp/rdma-core/rdma-core-52mlnx1/libibnetdisc/ibnetdisc.c:811; Failed to resolve self
/usr/sbin/ibnetdiscover: iberror: failed: discover failed
ibwarn: [5970] _do_madrpc: send failed; Invalid argument
ibwarn: [5970] mad_rpc: _do_madrpc failed; dport (DR path slid 0; dlid 0; 0)
/var/tmp/rdma-core/rdma-core-52mlnx1/libibnetdisc/ibnetdisc.c:811; Failed to resolve self
/usr/sbin/ibnetdiscover: iberror: failed: discover failed



azureuser@test-rdma-3:~/multicast_udp$ ibnetdiscover 
ibwarn: [5979] mad_rpc_open_port: can't open UMAD port ((null):0)
/var/tmp/rdma-core/rdma-core-52mlnx1/libibnetdisc/ibnetdisc.c:802; can't open MAD port ((null):0)
ibnetdiscover: iberror: failed: discover failed

azureuser@test-rdma-3:~/multicast_udp$ sudo ibnetdiscover 
ibwarn: [5981] _do_madrpc: send failed; Invalid argument
ibwarn: [5981] mad_rpc: _do_madrpc failed; dport (DR path slid 0; dlid 0; 0)
/var/tmp/rdma-core/rdma-core-52mlnx1/libibnetdisc/ibnetdisc.c:811; Failed to resolve self
ibnetdiscover: iberror: failed: discover failed

Is this feature supported? I know that mcast is not supported in Azure virtual networks, but I assumed that the InfiniBand is a separate network.

Answer 1

Multicast is not supported in Azure:

https://learn.microsoft.com/en-us/answers/questions/290448/multicast-option-in-azure.html

Hope this helps.