Issue reported when doing write action on smb share

David Lechevalier 1 Reputation point
2021-03-16T11:11:24.583+00:00

Hello,

On Windows server 2019, I often have the error 59: An unexpected network error occurred.
I did a python sample script which reproduce the issue doing write test on a Windows smb share.
I got the following errors:

The error returned by the script can be

Traceback (most recent call last):
File "Test_integrity.py", line 153, in <module>
test_integrity()
File "Test_integrity.py", line 149, in test_integrity
file_write_content(path, info)
File "Test_integrity.py", line 125, in file_write_content
os.remove(path)
WindowsError: [Error 59] An unexpected network error occurred: 'U:\data\Desktop\.\pref.ini'

or

Traceback (most recent call last):
File "Test_integrity.py", line 153, in <module>
test_integrity()
File "Test_integrity.py", line 149, in test_integrity
file_write_content(path, info)
File "Test_integrity.py", line 140, in file_write_content
win32file.WriteFile(handle, data, None)
pywintypes.error: (59, 'WriteFile', 'An unexpected network error occurred.')

To reproduce

  1. Create a share on another Windows server
  2. Map a share using the letter U:
  3. Install python + pywin32 (pip install pywin32)
  4. Start the script test.py

Actual result

  • The above error occur

I made some tests:
* The issue seems to only occurs from a Windows server 2019.
* It occurs with many samba version, (tested on centos7:4.10.16, bionic:4.10.13, xenial:4.3.11), many windows server share (Windows server 2012R2, Windows server 2019)
* It occurs even if I force dialect to 3.10, 3.02 or 3.00.
* The Windows server is fully updated (Build 17763.rs5_release.180914-1434).
* On procmon, I have the following error

78235-windows1.png

I wanted to join the wireshark trace of my issue (U:\data\Desktop\.\pref.ini) and the script used to reproduce my issue.
But The interface does not allow it.

I would like to understand more this issue and eventually find a workaround.

Thank you for your help.

David Lechevalier.

Windows Server 2019
Windows Server 2019
A Microsoft server operating system that supports enterprise-level management updated to data storage.
3,900 questions
Windows Server Storage
Windows Server Storage
Windows Server: A family of Microsoft server operating systems that support enterprise-level management, data storage, applications, and communications.Storage: The hardware and software system used to retain data for subsequent retrieval.
667 questions
{count} votes

14 answers

Sort by: Most helpful
  1. Candy Luo 12,751 Reputation points Microsoft Vendor
    2021-03-17T08:30:34.687+00:00

    Hi ,

    Based on my understanding, you might have the error 59: An unexpected network error occurred when you create / write / close files. Is that right? Please feel free to let me know if I have any misunderstanding.

    In order to get a more concreted idea of this issue, I would appreciate your help to collect the following information for further troubleshooting:

    1.Noticed that you have mapped the share. Let's narrow down whether the issue is related with mapping. What's the result when you access the share via UNC patch directly? Will the issue still occur?

    2.When the issue happens can you ping SMB server's IP address successfully?

    You can run ping IP addresss -t to test network connection.

    Best Regards,

    Candy


    If the Answer is helpful, please click "Accept Answer" and upvote it.

    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    0 comments No comments

  2. David Lechevalier 1 Reputation point
    2021-03-17T11:10:48.037+00:00

    Hi @Candy Luo

    I use a script which delete, create, write, close multiple files one by one. (no threading).

    1. I have unmaped the drive and updated my script in order to use UNC path. I still have the same issue.
    2. The ping did not mention connectivity issue

    C:\Users\Administrator>ping 10.90.0.156 -t

    Pinging 10.90.0.156 with 32 bytes of data:
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64
    Reply from 10.90.0.156: bytes=32 time<1ms TTL=64

    Ping statistics for 10.90.0.156:
    Packets: Sent = 15, Received = 15, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

    Best Regards,

    David.

    0 comments No comments

  3. Gary Nebbett 6,106 Reputation points
    2021-03-17T12:31:57.02+00:00

    Hello @David Lechevalier ,

    Thanks for the trace files. The trace data does not directly reveal the reason for the error message but it does give hints for further investigation.

    There are a vast number of anonymous logon sessions being (successfully) created in the trace (probably one for each of the individual steps in your "delete, create, write" loop), but no session logoffs. The problem might be caused by hitting some limit.

    There are probably more efficient ways of performing the steps (the "delete" operation could be eliminated by using appropriate flags to the "create" operation (CREATE_ALWAYS)).

    Did you intend to make the Python script available (it is formatted as a hyperlink in your message)? It might throw some light on way so many sessions are established.

    My suggestion for a next step would be to use Event Tracing for Windows (ETW) and Windows Performance Recorder (WPR) to get a deeper view of what is happening.

    Save the XML below to a file (say "smb.wprp") and then start WPR with the command "wpr -start smb.wprp!SMB -filemode". Then reproduce the problem and stop WPR with the command "wpr -stop smb.etl".

    The smb.etl resulting file can probably be substantially compressed (if necessary). If you can make that file available, then I would take a look at it.

    Gary

       <?xml version="1.0" encoding="utf-8"?>  
       <!-- wpr -start smb.wprp!SMB -filemode -->  
       <!-- wpr -stop ?.etl -->  
       <WindowsPerformanceRecorder Version="1.0" Author="Gary">  
         <Profiles>  
           <EventCollector Id="ETW" Name="ETW">  
             <BufferSize Value="64" />  
             <Buffers Value="128" />  
           </EventCollector>  
           <EventProvider Id="SMBClient" Name="Microsoft-Windows-SMBClient" NonPagedMemory="true">  
             <Keywords>  
               <Keyword Value="0x403000003C4" />  
             </Keywords>  
           </EventProvider>  
           <EventProvider Id="MRxSMB20" Name="E4AD554C-63B2-441B-9F86-FE66D8084963" NonPagedMemory="true">  
             <Keywords>  
               <Keyword Value="0xFFFFFFFF" />  
             </Keywords>  
           </EventProvider>  
           <Profile Id="SMB.Verbose.File" Name="SMB" Description="SMB" DetailLevel="Verbose" LoggingMode="File">  
             <Collectors>  
               <EventCollectorId Value="ETW">  
                 <EventProviders>  
                   <EventProviderId Value="SMBClient" />  
                   <EventProviderId Value="MRxSMB20" />  
                 </EventProviders>  
               </EventCollectorId>  
             </Collectors>  
           </Profile>  
         </Profiles>  
       </WindowsPerformanceRecorder>  
    

  4. David Lechevalier 1 Reputation point
    2021-03-17T16:33:23.813+00:00

    Hi @Gary Nebbett ,

    Thanks for the instruction.

    The etl trace are there https://drive.google.com/file/d/1yJnb9cZmgGv_Ahq-cmibXo2PFTe_fTxz/view?usp=sharing.
    I added the python script here https://drive.google.com/file/d/1y9fWl5Ko5srY7r2BwyYA89dHHZYhZwiF/view?usp=sharing

    The script actually uses the CREATE_ALWAYS flags. but the issue was more often with the remove.
    The main part of the script is only doing this

    The beginning of the script is for generating a list of files to test.
    Then, there is the main part which do write operation

             if os.path.exists(path):  
                     os.remove(path)  
       
             try:  
                     handle = win32file.CreateFile(  
                             path,  
                             win32con.GENERIC_WRITE,  
                             0,  
                             None,  
                             win32con.CREATE_ALWAYS,  
                             0,  
                             0)  
             except Exception as err:  
                     print(err)  
                     sys.exit(1)  
       
             win32file.WriteFile(handle, data, None)  
             win32file.CloseHandle(handle)  
    

    Best regards,
    David.

    0 comments No comments

  5. Gary Nebbett 6,106 Reputation points
    2021-03-18T09:06:55.48+00:00

    Hello @David Lechevalier ,

    I performed some simple tests between two Windows 10 systems (one acting as the SMB server) and the behaviour of Windows Defender seemed similar to that in your traces: Defender "opens" the new files on the remote file share, possibly just to canonicalize/normalize the path to the new file. I also see a lot of Defender initiated attempts to create a guest session with the remote server (which fail in my case because guest access is not enabled).

    You mentioned in your first message that the problem occurs for you also with some Windows servers. Is guest access enabled on those servers?

    If the problem can be reproduced when the server does not allow guest access then that would help resolve concerns that the large number of successful guest SMB sessions is causing some limit to be hit.

    It would also be helpful/interesting to actually find evidence of a STATUS_INSUFFICIENT_RESOURCES error in an ETW/WPR trace. There are lots of things that could be traced and the tricky thing is finding a balance between the amount of trace data produced and its usefulness. I am happy to make suggestions about what to trace (by providing updated XML for the .wprp file) - if you are happy to share the resulting trace data.

    Gary

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.