SMB share freezes some clients but not others.

Question

I have recently expanded my cold storage beyond where storage spaces seemed to be happy. I added another 120 12tb drives and the performance was abysmal. this is just cold storage so I decided to go with Truenas and was very happy but I only ever tried with 1 computer. now I am stuck. I have a 4-node hyperconverged hyper-v cluster that works beautifully. Each node is a Dell r820 with quad E5-4657 v2 procs and 768gb ram. each node has quad 40gb NICs in a SET bond (160gbps). the servers all have SSDs and convergent storage and it's perfect. vm's fly around the network at lightning speed. (All nodes were deployed at the same time with scripts, they should be equal)

Enter the cold storage. I had an old Windows 2019 server that hosted the cold storage(still around) It worked perfectly as well but was aging(Dell c2100). this has now been replaced with another r820 with the same specs as above but running Truenas and 2 60 drive DAS shelves. the server seems to be very happy. I built it only by testing with node 4. There are no issues. Node 4 can access the cold storage as if it's local, super fast(4GBps, yes, capital B). I thought I was done. Then I tried to access the share from nodes 1-3 and they all have exactly the same issue. drive maps are fine. you get 3 folders deep and it completely freezes. you transfer a file.....freeze. you go 2 folders down nd try to back up.....freeze. all 4 nodes can still access the share on the old cold storage (c2100) so its not SMB in general but maybe something with Windows and Truenas.

Digging further. I tested with both my domain controllers and they have no issue with the share either. they browse up and down at lightning speed.

Now I have made another interesting discovery. since this is a hyper-v cluster it has lots of Windows virtual machines running on it all sharing the networking......I have yet to find a virtual machine that has an issue with the share even though it's running on a host that can't access it leading me to believe it is not hardware or configuration-related. It's something in the OS.

all nodes are 2025. old storage server is 2019 and there are plenty of Windows 10,11 and Server 2022 vms that have no issue but nodes 1, 2 and 3 can't browse share or transfer any files. transfers usually move about 30MB and then freeze completely.

I would just blow them away and start over but its a cluster and running some production VM's. worst case I can move them to another server and start the cluster over but everything but SMB is working so I want to dive deeper into this.

Answer 1

Hello,

Thank you for posting in Microsoft Community forum.

1. SMB Version Compatibility
   Ensure that both TrueNAS and Windows Server 2025 are using compatible SMB versions. It’s possible that the SMB version on TrueNAS is outdated or incompatible, leading to connection issues. Try explicitly configuring TrueNAS to use SMB3 rather than relying on auto-negotiation. You can also check the SMB version on the affected Windows nodes by running the Get-SmbConnection command to verify if they are using SMB3.
2. Network Settings / MTU Mismatch
   Given your high-bandwidth setup (40Gbps NICs), an MTU mismatch could cause packet fragmentation, resulting in poor performance or freezes. Check that the MTU settings on both TrueNAS and the nodes are consistent, especially if you are using jumbo frames. Ensure that all devices involved (including the TrueNAS server and all nodes) have matching MTU settings, which can be verified through the command line or device configuration interfaces.
3. SMB Client Caching or Network Performance
   Disable SMB client-side caching on the affected Windows nodes, as caching might cause delays or freezes. This can be done via Group Policy or the registry. Additionally, use network monitoring tools like netstat or Wireshark to check the network performance on the nodes. Look for signs of packet loss, high latency, or retransmissions, which could indicate a network bottleneck or issue affecting SMB traffic.
4. Windows Network Stack or SMB Filters
   If the issue is related to the Windows network stack, you can reset it by running the following commands:

netsh int ip reset

netsh winsock reset

Also, make sure there are no third-party firewalls, network filters, or antivirus software interfering with SMB traffic. These can sometimes cause slowdowns or freezes when interacting with SMB shares, especially over high-performance networks.

1. Check TrueNAS Logs
   Examine the SMB logs on the TrueNAS server, especially when access issues or freezes occur. Look for any errors related to SMB protocol negotiation, permission issues, or other related errors. This could point to issues on the server side that need to be addressed.
2. Check Hyper-V Cluster Configuration
   Since you are running a Hyper-V cluster, ensure that the virtual network configurations are correct. Particularly check the network adapter settings on nodes 1–3 to ensure there are no configuration conflicts. It’s possible that the network traffic between the physical nodes and the virtual machines is misconfigured, causing the issues with SMB access.
3. Check for OS Updates
   There might be known SMB-related issues with Windows Server 2025. Ensure that all nodes have the latest OS updates and patches installed. Check the Windows update logs or official Microsoft documentation to confirm if any network-related issues have been addressed in recent patches.
4. Test with Other SMB Clients
   Try accessing the SMB share from a different OS, such as Linux or macOS, to see if the issue persists. If these clients can access the share without issues, it will help determine whether the problem is specific to Windows 2025 or related to the SMB configuration on TrueNAS.

Following these steps should help you narrow down the root cause of the freezing issue. Let me know if you find anything new or need further assistance!

Answer 2

1. SMB Version Compatibility
   Have disabled SMB1 and 2. Multichannel is off( all links to storage are SET or LACP)
2. Network Settings / MTU Mismatch
   Both switches use an MTU of 9216. Windows machines are all 9014, truenas is 9014
3. SMB Client Caching or Network Performance
   Disabled SMB client-side caching. Not seeing any packet loss
4. Windows Network Stack or SMB Filters
   reset network stack. There are no third-party firewalls, network filters, or antivirus software and Windows Firewall and Defender are both currently off.
5. Check TrueNAS Logs
   Not sure I am looking in the right place. SMB Audit logs show nothing but success.
6. Check Hyper-V Cluster Configuration
   All networking is identical and set up with scripts and managed with . I have manually looked through the settings and they are all equal. SET teams of 4 40GB NICs. 9012 MTU
7. Check for OS Updates
   All nodes are up to date, the first thing I tried was when this issue came up. Every day I double-check that a patch hasn’t dropped.
8. Test with Other SMB Clients
   Node 4 can access just fine, the old storage server is also fine, both DCs are fine, VM’s running on the broken nodes can access just fine(through the same networking). There are Windows 7, 10, 2012, 2016, 2022, 2025 machines along with a handful of Linux and docker containers….all just fine. It's only 3 of the 4 nodes. I can always get 2 folders deep. The next move freezes everything. Could be another folder down, hit the back button….doesnt matter. Always the 3^rd^ move inside the share. If my first move is several folders deep. It works. But 2 more moves and I am done. If I let it sit long enough it will usually display the folder and I can sometime get 3 more moves out of it. Usually around 2 minutes. No errors in the widows system logs

The nodes all talk to each other and another SMB shares on a windows server just fine. They have clustered shared storage so they must have flawless networking.

I had been running this with a local account on the truenas since I only needed to map it once on the nodes to share out to VM’s. I have now linked it to the domain and am using domain accounts to access the share with the same result.

Thought I would make a vidie to help people understand. the share on the right is editical to the truenas share on the left but hosted on server 2019. you can see I can brows this SMB share just fine. when browsing the Truenas share on the left I can only go 2 folders deep, the third will always cuase a freeze no mater what folders even going into a folder and backin out to a previuously working folder will freeze.

https://www.youtube.com/watch?v=W3Auk7a-xew

Answer 3

Based on your latest response, you have performed most of the diagnostic steps and ruled out many potential issues. Here’s a further analysis and set of suggestions:

1. SMB Version Compatibility: You've confirmed that SMB 1 and 2 are disabled, and all devices are using SMB 3, which is excellent. Please ensure that the SMB protocol version on both TrueNAS and Windows is correctly configured and that no other configuration issues exist. Use the Get-SmbConnection command on the Windows nodes to verify the SMB connection status. Additionally, make sure SMB 3 is explicitly enabled on TrueNAS if necessary.
2. Network Settings/MTU Issues: You’ve checked the MTU settings on the switches and confirmed that they are consistent. It's important to ensure that the MTU configuration on both TrueNAS and Windows nodes is fully aligned and that no packet fragmentation occurs during network traffic. You may want to further verify that MTU settings match across all devices involved.
3. SMB Client Caching and Network Performance: Disabling SMB client-side caching is a good step to resolve network freezes. Since you’ve done that, I recommend checking whether there is any firewall, network filtering, or antivirus software that could be interfering with SMB traffic, especially in high-bandwidth network environments.
4. TrueNAS Logs: You mentioned that you’ve checked the TrueNAS logs and didn’t find any errors. If possible, try checking more detailed logs (such as syslog) to see if there are any hidden network issues or permission errors that need to be addressed.
5. Hyper-V Cluster Configuration: You’ve stated that all nodes in the Hyper-V cluster are functioning properly, but certain virtual machines are unable to access the shared storage. Please ensure that the virtual network adapter configurations for the Hyper-V cluster are set up correctly, and check that network traffic between the physical nodes and virtual machines is not being lost or dropped.
6. OS Updates: You’ve confirmed that all operating systems are up to date and that patches have been installed. This is crucial. Please double-check the Windows event logs and any network-related logs on TrueNAS to ensure no patches or configurations are missing.
7. Testing with Other SMB Clients: Since you’ve tested access from Linux or macOS clients and the issue still persists, it suggests that the problem may not lie within the client operating systems but rather at the SMB share or network configuration level.

Further Suggestions:

1. If possible, try isolating virtual machines one by one to see if specific VMs or configurations are causing the issue. For example, check if the virtual network adapter configuration within Hyper-V might be contributing to the issue.
2. Recheck firewall, network filtering, and antivirus software settings, including Windows Defender, to ensure they are not interfering with network traffic.
3. If the issue persists, consider reconfiguring the network or testing a new virtual machine environment to rule out any problems with the cluster configuration.