RDMA performance drop troubleshooting

Question

Hi Microsoft Community,

I’m facing with a weird issue where file transfers between high‑performance storage nodes start off using RDMA (RoCEv2) just fine, but then suddenly fall back to single‑channel TCP/IP without any warning. It’s really hurting throughput and latency. I’ve already checked the NIC firmware and drivers RDMA is enabled so I’m guessing the interface cache or capability negotiation might be stuck somewhere.

I’d like to figure out how to audit RDMA interface capabilities using Get‑SmbServerNetworkInterface and clear any cached restrictions that could be forcing the fallback. Has anyone here run into something similar or found a good way to reset the SMB Direct interface state without rebooting the nodes ?

Answer 1

Hi,

There are a few ways to audit and reset the RDMA interface state without rebooting the nodes.

First, use Get-SmbServerNetworkInterface and Get-SmbClientNetworkInterface to confirm which NICs are RDMA‑capable and whether they are currently flagged as available for SMB Direct. If you see interfaces marked as “RSS Capable” but not “RDMA Capable,” that’s a sign the capability negotiation failed. You can also run Get-NetAdapterRdma to check whether RDMA is enabled at the NIC level.

If cached restrictions are suspected, you can clear and refresh the SMB Direct state by disabling and re‑enabling the RDMA interface (Disable-NetAdapterRdma followed by Enable-NetAdapterRdma). Another option is to restart the SMB service (Restart-Service LanmanServer) which forces renegotiation of RDMA capabilities without requiring a full node reboot. Reviewing the SMBClient and SMBServer logs in Event Viewer will also show whether the fallback was triggered by congestion, packet loss, or capability mismatch.

As a best practice, keep NIC firmware and drivers aligned with the OS build, and validate RDMA connectivity with Test-SmbBandwidth to ensure sustained throughput. This combination usually helps prevent unexpected fallbacks to TCP/IP.

I hope this gives you a clear path to audit and reset RDMA interfaces safely. If this explanation helps, please hit “accept answer”

Jason

Answer 2

Use these checks and fixes for the specific RDMA-to-TCP fallback pattern:

1. Audit whether SMB still sees the interfaces as RDMA-capable.
   • On the server, run:

          Get-SmbServerNetworkInterface
     

   • On the client, run:

          Get-SmbClientNetworkInterface
     

   If an adapter is no longer reported as RDMA-capable, the documented cause is often older or incorrect NIC firmware or drivers.
2. Confirm whether the fallback matches a known Windows Server 2012 R2 issue.
   • If the nodes run Windows Server 2012 R2 and Event Viewer shows SMBClient Event ID 30818, RDMA connections can fail back to TCP with certain RDMA-capable adapters.
   • The documented resolution is to install update 2919355.
3. Check whether SMB Signing or SMB Encryption is causing the performance drop.
   • In Windows Server 2016 and Windows Server 2019, enabling SMB Signing or SMB Encryption significantly reduces SMB Direct performance.
   • In that case, SMB stops using RDMA direct data placement and falls back to a less efficient send/receive path, which reduces throughput.
   • In Windows Server 2022 and Windows 11, SMB Direct supports encryption with relatively minor performance degradation.
4. Verify SMB Multichannel is still available.
   • SMB Direct depends on SMB Multichannel to detect RDMA capability.
   • If SMB Multichannel is disabled, SMB Direct is also disabled.
   • SMB can then use regular TCP/IP even on RDMA-capable adapters.
5. Reproduce and measure the fallback cleanly.
   • Disable RDMA on the adapter.
   • Run a large file copy and measure the time.
   • Re-enable RDMA and repeat the same copy.
   • Compare results.
   • For cleaner testing, copy more data than memory can cache, perform the copy twice and time the second run, and restart both client and server before each test.
6. Monitor SMB Direct counters during transfer.
   • Open perfmon.
   • Go to Monitoring Tools > Performance Monitor.
   • Add counters from SMB Direct Connection:
     • Bytes RDMA Read/sec
     • Bytes RDMA Written/sec If these counters stop increasing during the transfer while the copy continues, that indicates SMB is no longer using RDMA for the data path.
7. If Event ID 30818 appears after restart on Windows Server 2012 R2, there is a documented workaround.
   • Warning: this requires a registry change and incorrect edits can cause serious problems. Back up the registry first.
   • Set the SmbDirect service to start automatically:
     • Registry path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\smbdirect
     • Change Start from 3 to 2. This is specifically documented for the startup-order issue on Windows Server 2012 R2.

The context does not provide a supported method to clear SMB Direct interface cache or reset SMB Direct interface state without a reboot. The supported guidance in the available documentation is to verify RDMA capability, check for Event ID 30818, update firmware/drivers, validate SMB Multichannel, review signing/encryption settings, and use the documented Windows Server 2012 R2 update or workaround where applicable.

---

• SMB Direct
• Event 30818 RDMA connections fail back to TCP caused by NDK disconnection
• SMB Direct
• Reduced networking performance after you enable SMB Encryption or SMB Signing in Windows Server 2016 and Windows Server 2019
• SMB troubleshooting guidance
• Performance tuning for SMB file servers