Storage Spaces Direct Inter-Node SMB Traffic

Question

Hi,

I'm justing learning how to setup a hyperconverged S2D Cluster (thanks to Jaromir's MSLabs scripts on Github).

There is a question to which I can't find an answer.
Which NICs can be used to set the SMB/Inter-Node traffic for storage replication?
Can dedicated NICs be used for replication?

I come from VMware ESXi and I am used to be able to choose which services the NICs can use.

For network configuration of S2D, there are various posts related to RDMA, mainly RoCE with DCB. Dedicated NICs are not discussed here either. Configuration without RDMA is almost not discussed at all.

Similar questions were also asked in these older posts. But unfortunately without a clear answer.

Storage spaces direct Force Storage traffic on an specified nic:
https://social.technet.microsoft.com/Forums/en-US/e422c99f-b74e-40b4-a122-9ae15466b1ec/storage-spaces-direct-force-storage-traffic-on-an-specified-nic?forum=ws2016

Storage Spaces Direct networking:
https://social.technet.microsoft.com/Forums/windows/en-US/a16cf824-5345-4baa-a21d-f6cd21ed8e5d/storage-spaces-direct-networking?forum=winserverClustering

I would be happy if someone can give me a hint!
Regards

Answer 1

Hi,

Thanks for replying!

When you use SMB Multichannel with a RSS-capable network adapter, SMB creates multiple TCP/IP connections for that particular session. so, yes, SMB multichannel does automatically seek for high speed NIC ports for your storage. Besides, in S2D, SMB multichannel is enabled by default, so you do not have to worry about how to configure it. You just need to ensure all of your hardware device meet the configuration requirements and to follow the guidance to configure your S2D in a correct way. And if you have encountered any problems during your use of S2D, please feel free to feedback, we will try to help improve your user experience.

Here is some articles you may find useful:

Manage SMB Multichannel
https://learn.microsoft.com/en-us/azure-stack/hci/manage/manage-smb-multichannel

Deploy Storage Spaces Direct
https://learn.microsoft.com/en-us/windows-server/storage/storage-spaces/deploy-storage-spaces-direct

Thanks for your support! Have a nice day! : )

BR,
Joan

--------------------------------------------------------------------------------------------------------------------

If the Answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 2

Hi,

Thanks for your posting!

Based on your descriptions, firstly, I am not sure if dedicated NIC is a type of physical NIC, in my knowledge, storage replication require seperate storage network or bandwidth and that's why we call the NIC(s) that used to set up this seperate network as dedicated NIC. So firstly I would like to check if your dedicated NIC is what I think. If so, it seems like that you need to learn more about SMB multichannel in S2D.

Here are some articles for best practices for configuration SMB multi-channel in a S2D:

https://learn.microsoft.com/en-us/windows-server/failover-clustering/smb-multichannel

https://www.tech-coffee.net/design-the-network-for-a-storage-spaces-direct-cluster/

(Please note: Information posted in the given link is hosted by a third party. Microsoft does not guarantee the accuracy and effectiveness of information.)

Besides, it seems that you want to configure the network for storage replication as the fastest, so I will go to research on this demand and I will tell you anything useful as soon as possible.

Thanks for your support!

BR,
Joan

--------------------------------------------------------------------------------------------------------------------

If the Answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 3

Hello @JiayaoZhu

Thanks for your answer.

With "dedicated NIC" I actually mean one port from a network adapter. Let's assume I have the following server network configuration:

1x Onboard 1GbE NIC with 4x Ports
1x AddOn 10GbE NIC with 2x Ports

I would like to use the 2x Onboard 1GbE Ports for VM Traffic and the other 2x Onboard 1GbE Ports for Management / ClusterHB Traffic.
The 2x AddOn 10GbE Ports should be used only for S2D storage replication traffic.

I want to get sure that only the fastest ports are used for replication and the slower are not involved. Currently I don't know or see where to configure which ports are used for this.

It seems that SMB Multichannel automatically takes care of it without the need or possibly to configure it any further?

Regards