Performance issue on SMB share with Windows 10

Question

Performance issue on SMB share with Windows 10

Tournier Jérôme 0

Hello,

I have some Windows 10 workstations with a SMB3 mounted share (on a Windows server) containing a directory structure with ~15k files.

When doing a recursive listing, the command (dir /s /b /a A: > list.txt) run in ~20s.

When doing a same test from a linux workstation with a mount point to the same share, the command (find /mnt/tmp/ > list.txt) run in ~5s.

Do someone have an explanation on that and how can I solve this issue ? This is very problematic to our Windows users to have such response delay.

Thx

Answer 1

Gary Nebbett 5,426

Hello Jérôme,

The difference is probably attributable to the "programs" that perform the recursive listing and is not directly related to the operating systems (Windows vs. Linux).

Network and/or event tracing can be used to gain insights into what the programs are doing and where the time goes.

I don't have a Linux system to check the "find" command, but analysing the "dir /s /b /a" command provides some hints as to why it is slower. It appears to traverse each folder/directory twice: once to obtain the names of the entries in the folder/directory and once to identify folders/directories for recursion. Furthermore, for each directory discovered, two queries for properties are made (FileStandardInformation, FileBasicInformation) even though the data is not needed when the "/b" qualifier is present.

The Windows "dir" command is not necessarily badly coded - it appears that emphasis was given to flexibility and reusability. This is normally not a problem, but mapping this style to network protocol operations on large trees reveals a performance problem.

It would be possible to write a program that is as quick as Linux "find" and it might be possible to find some public domain utility that is coded that way; however, it would not be easy to convince your users to use it.

Gary

Tournier Jérôme 0 Reputation points

2023-10-27T06:55:21.2166667+00:00

Thanks for your feedback.

I used here the "find" command as it was very easy to explain the issue. In fact, we use some python scripts that parse a directory structure on a SMB share. And the behavior is exactly the same : python script execution is much more longer when executed on Windows compared to when executed on linux.

We wrote a very simple python script that do a kind of "dir /s /b /a" and the execution time is the same of when running "dir /s /b /a", which is much more longer compared when launched on Linux. In that case, this is the same program that is executed.

I made some wireshark capture when run on both Windows & Linux (see linux.png & windows.png) when script is run on Linux and windows on the same share of course.

You can see that much more time is spent on some procedure calls (create,find,getinfo) when run on Windows which sounds to match what you explain.

Is the conclusion that nothing can be done to have some comparable time results ?

linux.png

windows.png
Gary Nebbett 5,426 Reputation points

2023-10-27T07:54:42.99+00:00

Hello Jérôme,

That is not the conclusion that I would draw.

It would probably be helpful to look at your network traces in more detail. For example, the number of folder/directory returned per query depends on the parameters (buffer size) passed to the query directory function: a smaller buffer size means a quicker response but more requests/responses. The parameter sizes might differ between the Python environments on Windows/Linux. Anti-malware software might also impact the performance.

You first need to identify in detail the causes for any like-for-like timing differences and then think about what can be realistically done to address the differences.

If you want any help with the trace analysis, feel free to ask.

Gary

Performance issue on SMB share with Windows 10

1 answer

Activity