SQL Server is terminating because of fatal exception c0000005

Question

Facing this SQL Server access violation error on SQL Server 2016 (13.0.6435.1) running on Azure VM during startup!

Error: 17311, Severity: 16, State: 1.

SQL Server is terminating because of fatal exception c0000005. This error may be caused by an unhandled Win32 or C++ exception, or by an access violation encountered during exception handling. Check the SQL error log for any related stack dumps or messages. This exception forces SQL Server to shutdown. To recover from this error, restart the server (unless SQLAgent is configured to auto restart).

Here are our observations,

• Occurs on our SQL Server 2016 servers. Not on 2019
• Upgrading to 13.0.6435.1 (SP3 + GDR) did not help
• Occurs often only on machine startup
• SQL crash stack dump is written
• SQL Server and Agent services gets stopped
• If SQL server automatic delayed start is setup, SQL server somehow gets restarted
• Occurs only on instances that has database(s) on Azure blob storage!!! <= suspect

Here's our suspected condition:

• webio module (Web Transfer Protocols API)
• Gets loaded only on SQL instances that has databases on Azure blob storage
  • verified with select * from sys.dm_os_loaded_modules where name like '%webio%'
• Perhaps this module is used to communicate with databases that are on Azure blob storage

Short Stack Dump : (This stack dump looks consistent with every time we got this error)

00007FFFE5906EA8 Module(KERNELBASE+0000000000026EA8)                                                             
00007FFFD21C185E Module(sqllang+000000000104185E)                                                                
00007FFFD21C55D9 Module(sqllang+00000000010455D9)                                                                
00007FFFD16CE846 Module(sqllang+000000000054E846)                                                                
00007FFFD896B232 Module(sqldk+000000000006B232)                                                                  
00007FFFE593B917 Module(KERNELBASE+000000000005B917)                                                             
00007FFFE9080B8B Module(ntdll+0000000000090B8B)                                                                  
00007FFFE9080B51 Module(ntdll+0000000000090B51)                                                                  
00007FFFE909D7F0 Module(ntdll+00000000000AD7F0)                                                                  
00007FFFE9086B96 Module(ntdll+0000000000096B96)                                                                  
00007FFFE9095C12 Module(ntdll+00000000000A5C12)                                                                  
00007FFFE909AB3D Module(ntdll+00000000000AAB3D)                                                                  
00007FFFE90249D3 Module(ntdll+00000000000349D3)                                                                  
00007FFFE9099CAA Module(ntdll+00000000000A9CAA)                                                                  
00007FFFD4AEDAB9 Module(sqlmin+000000000137DAB9)                                                                 
00007FFFD4AED709 Module(sqlmin+000000000137D709)                                                                 
00007FFFD4AED548 Module(sqlmin+000000000137D548)                                                                 
00007FFFE2BAB917 Module(WINHTTP+000000000001B917)                                                                
00007FFFE2BBDF53 Module(WINHTTP+000000000002DF53)                                                                
00007FFFD4AEF927 Module(sqlmin+000000000137F927)                                                                 
00007FFFD4AED620 Module(sqlmin+000000000137D620)                                                                 
00007FFFE2BB63DC Module(WINHTTP+00000000000263DC)                                                                
00007FFFE2BB1788 Module(WINHTTP+0000000000021788)                                                                
00007FFFE2BF1C5F Module(WINHTTP+0000000000061C5F)                                                                
00007FFFD969F4EC Module(webio+000000000000F4EC)                                                                  
00007FFFD969F21D Module(webio+000000000000F21D)                                                                  
00007FFFD9698577 Module(webio+0000000000008577)                                                                  
00007FFFD96977ED Module(webio+00000000000077ED)                                                                  
00007FFFD96975A0 Module(webio+00000000000075A0)                                                                  
00007FFFD96974A4 Module(webio+00000000000074A4)                                                                  
00007FFFD96A0E6C Module(webio+0000000000010E6C)                                                                  
00007FFFD96A07FB Module(webio+00000000000107FB)                                                                  
00007FFFD96A064D Module(webio+000000000001064D)                                                                  
00007FFFD96AE940 Module(webio+000000000001E940)                                                                  
00007FFFE593F7A0 Module(KERNELBASE+000000000005F7A0)                                                             
00007FFFE902AF58 Module(ntdll+000000000003AF58)                                                                  
00007FFFE900ECDD Module(ntdll+000000000001ECDD)                                                                  
00007FFFE8DA84D4 Module(KERNEL32+00000000000084D4)                                                               
00007FFFE9041A11 Module(ntdll+0000000000051A11)

We store one or more databases on Azure blob storage following this method. Please help fixing this error.

Answer 1

The ticket has been closed. Confirmed it is SQL 2016 bug with no fixes. Details below:

Based on analysis of this stack dump by Microsoft technical support team, the issue occurs when it starts a session with Azure Blob Storage (sqlmin!SystemThread::GetCurrentNoCheck).

Microsoft recommended upgrading to a later version of SQL Server. We have tested with 2019 without errors as said in my observation above.

As a partial workaround, setting SQL 2016 service to "delayed start" seems to retry after this error automatically and brings the service up (as said in my observation above). SQL Agent, however, remains stopped.