sqlservr.exe crash, faulting module MSVCR80.dll, after upgrading Windows server OS from 2012 R2 to 2019

Question

SQL server 2016 service was running fine on Windows 2012 R2. After Windows OS upgraded to 2019, service process kept crashing, faulting module MSVCR80.dll.

Any help is much appreciated!

Below application error log:

===========================

Log Name: Application
Source: Application Error
Date: 5/31/2022 11:19:12 AM
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Computer: ********
Description:
Faulting application name: sqlservr.exe, version: 2015.130.4224.16, time stamp: 0x5b783b15
Faulting module name: MSVCR80.dll, version: 8.0.50727.9554, time stamp: 0x5b6a5d17
Exception code: 0xc000000d
Fault offset: 0x000000000001da44
Faulting process id: 0x2898
Faulting application start time: 0x01d8749d2f334c5e
Faulting application path: C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLPOWERBI\MSSQL\Binn\sqlservr.exe
Faulting module path: C:\WINDOWS\WinSxS\amd64_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.9554_none_88e038cb2fae7c50\MSVCR80.dll
Report Id: f6d84682-bdd3-476a-8ad8-db90bfdefbf8
Faulting package full name:
Faulting package-relative application ID:

User mode process momery dump was captured

=====================================

0:004> !analyze -v

---

**

Exception Analysis

**

---

KEY_VALUES_STRING: 1

Key  : Analysis.CPU.mSec  
Value: 130546  

Key  : Analysis.DebugAnalysisManager  
Value: Create  

Key  : Analysis.Elapsed.mSec  
Value: 242550  

Key  : Analysis.Init.CPU.mSec  
Value: 3937  

Key  : Analysis.Init.Elapsed.mSec  
Value: 186221  

Key  : Analysis.Memory.CommitPeak.Mb  
Value: 395  

Key  : CLR.BuiltBy  
Value: NET472REL1LAST_C  

Key  : CLR.Engine  
Value: CLR  

Key  : CLR.Version  
Value: 4.7.3740.0  

Key  : Timeline.OS.Boot.DeltaSec  
Value: 5029  

Key  : Timeline.Process.Start.DeltaSec  
Value: 33  

Key  : WER.OS.Branch  
Value: rs5_release  

Key  : WER.OS.Timestamp  
Value: 2018-09-14T14:34:00Z  

Key  : WER.OS.Version  
Value: 10.0.17763.1  

Key  : WER.Process.Version  
Value: 2015.130.4224.16  

FILE_IN_CAB: sqlservr.exe.9996.dmp

NTGLOBALFLAG: 0

PROCESS_BAM_CURRENT_THROTTLED: 0

PROCESS_BAM_PREVIOUS_THROTTLED: 0

APPLICATION_VERIFIER_FLAGS: 0

CONTEXT: (.ecxr)
rax=0000000000000000 rbx=0000000000000000 rcx=000000054abfa500
rdx=00000000ac603c46 rsi=0000000000000000 rdi=0000000000000000
rip=000000005448b0ec rsp=000000054abfa420 rbp=0000000000000000
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000
r11=0000000000000000 r12=0000000000000000 r13=00000119061e9558
r14=0000000000000004 r15=0000000000000000
iopl=0 nv up ei pl zr na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
msvcr80!_invalid_parameter+0x6c:
00000000`5448b0ec 488d4c2440 lea rcx,[rsp+40h]
Resetting default scope

EXCEPTION_RECORD: (.exr -1)
ExceptionAddress: 000000005449da44 (msvcr80!_wcsicmp_l+0x00000000000000f4)
ExceptionCode: c000000d
ExceptionFlags: 00000000
NumberParameters: 0

PROCESS_NAME: sqlservr.exe

ERROR_CODE: (NTSTATUS) 0xc000000d - An invalid parameter was passed to a service or function.

EXCEPTION_CODE_STR: c000000d

STACK_TEXT:
000000054abfa420 000000005449da44 : 0000000000000000 000000054abfb3b0 00000000541a1488 0000000000000000 : msvcr80!_invalid_parameter+0x6c
000000054abfa9e0 00000000541a8c87 : 0000000000000000 0000000000000000 0000000000000000 00000119061e9558 : msvcr80!_wcsicmp_l+0xf4
000000054abfaa40 00000000541a6630 : 0000000000000000 0000000000000000 0000000000000000 00000119061e94e0 : perf_MSSQLSERVER_sqlctr10_50_1600_1!CountersExist+0x7f
000000054abfadc0 00007ffc9abe992b : 00000119061e9290 00000000541a6938 000000054abfb3b0 0000000000000000 : perf_MSSQLSERVER_sqlctr10_50_1600_1!OpenSQLPerformanceData+0x70
000000054abfadf0 00007ffc9abe8d17 : fffffffffa0a1f00 0000000000000000 0000000000000000 0000011a8770c1a0 : advapi32!OpenExtObjectLibrary+0x27f
000000054abfb110 00007ffc9abe8be1 : 00007ffc9ac41e90 00000119061e9290 0000000000000014 0000011a8772dd88 : advapi32!QueryV1Provider+0x9b
000000054abfb250 00007ffc9abdf7ac : 000000054abfb3b0 0000000500000000 0000000000000000 ffffffff00000000 : advapi32!QueryExtensibleData+0x381
000000054abfb320 00007ffc9abe0201 : 0000010abed169a0 000000054abfbb68 000000000000037f 0000000000008001 : advapi32!PerfRegQueryValueEx+0x234
000000054abfb780 00007ffc97e930fc : 0000000000000000 0000010abed10000 0000011a87520000 00007ffc9bd8c49d : advapi32!PerfRegQueryValue+0x31
000000054abfb7d0 00007ffc97e92733 : ffffffff80000004 000000054abfbc70 0000011a876c6ac0 000000054abfbc60 : KERNELBASE!LocalBaseRegQueryValue+0x4ec
000000054abfbb10 00007ffc88e66979 : ffffffff80000004 0000000000040000 0000000000080000 000000054abfbbe4 : KERNELBASE!RegQueryValueExW+0xf3
000000054abfbbb0 00007ffc88e63db4 : 0000011906144300 0000000000000000 0000011900080000 0000000000000000 : pdh!GetSystemPerfData+0xb5
000000054abfbc50 00007ffc88e936af : 0000010abf83ddb0 0000000000000000 00000112bab16340 00007ffc9bd8022b : pdh!GetMachineEx+0x1c4
000000054abfbec0 00007ffc88e969d3 : 0000000000000000 00007ffc9bd7fc27 0000000000000000 0000000000000074 : pdh!PdhiEnumObjects+0x77
000000054abfc040 00007ffc88e96b9a : 0000000000000000 0000000000000000 0000000000000000 000000054abfc1c0 : pdh!PdhEnumObjectsHW+0x173
000000054abfc0d0 00007ffc4491bca3 : 0000000000000000 0000000000000000 0000000000000000 000000054abfc1c0 : pdh!PdhEnumObjectsW+0x16a
000000054abfc160 00007ffc4491be23 : 00000112bab170b0 0000000000000000 00007ffc00000000 00007ffc97eedf9b : hkengine!HkGetPerfCounterHandle+0x13b3
000000054abfc1b0 00007ffc4491c512 : 0000000000000000 0000000000000000 0000000000000000 00000112bab16340 : hkengine!HkGetPerfCounterHandle+0x1533
000000054abfc210 00007ffc44751cbb : 00000112bab16340 00000112bab16340 00000000000026cb 0000000000000000 : hkengine!InitializeSystemPerfCounters+0x422
000000054abfc280 00007ffc465221f7 : 00000112bab16340 00000112b8340040 0000000000000000 0000000000000000 : hkengine!HkInitializeSystemPerfCounters+0x7b
000000054abfc2e0 00007ff7e0add49d : 00007ffc46d9ea80 000000054abfeb60 0000000000000001 8000000000000000 : sqlmin!InitHekaton+0x497
000000054abfc7d0 00007ff7e0adeeb3 : 0000000000000000 0000000000000000 0000000000000001 0000000000000000 : sqlservr!InitServerComponents+0x32d
000000054abfc830 00007ff7e0ad84f8 : 000000054abfeb80 0000000000000000 0000000000000000 0000000000000000 : sqlservr!sqlservr_main+0x603
000000054abfea90 00007ffc9b039d02 : 0000000000000000 0000010abed55c20 0000010abed55c20 0000000000000000 : sqlservr!TListElem<TList<NodeManager,SOS_Node,16,TListSLock> >::RemoveAndDestroy+0x1c9
000000054abffa90 00007ffc9bc87974 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : sechost!ScSvcctrlThreadA+0x22
000000054abffac0 00007ffc9bdca2f1 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : kernel32!BaseThreadInitThunk+0x14
000000054abffaf0 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : ntdll!RtlUserThreadStart+0x21

STACK_COMMAND: ~4s; .ecxr ; kb

FAULTING_SOURCE_LINE: f:\dd\vctools\crt_bld\self_64_amd64\crt\src\wcsicmp.c

FAULTING_SOURCE_FILE: f:\dd\vctools\crt_bld\self_64_amd64\crt\src\wcsicmp.c

FAULTING_SOURCE_LINE_NUMBER: 52

SYMBOL_NAME: msvcr80!_wcsicmp_l+f4

MODULE_NAME: msvcr80

IMAGE_NAME: msvcr80.dll

FAILURE_BUCKET_ID: INVALID_PARAMETER_c000000d_msvcr80.dll!_wcsicmp_l

OS_VERSION: 10.0.17763.1

BUILDLAB_STR: rs5_release

OSPLATFORM_TYPE: x64

OSNAME: Windows 10

IMAGE_VERSION: 8.0.50727.9554

FAILURE_ID_HASH: {ff4e314f-6660-492f-4a93-11e7fa2fa7ee}

Followup: MachineOwner
---------

Answer 1

You appear to be running a very old patch level of SQL Server. Please upgrade to the current SP3.

https://learn.microsoft.com/en-US/troubleshoot/sql/general/determine-version-edition-update-level

Answer 2

From what I can read in your logs...

sqlserver.exe is crashing cause of any problem with MSVCR80.dll

Faulting application path: C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLPOWERBI\MSSQL\Binn\sqlservr.exe
Faulting module path: C:\WINDOWS\WinSxS\amd64_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.9554_none_88e038cb2fae7c50\MSVCR80.dll
[...]
msvcr80!_invalid_parameter+0x6c:
[...]
ExceptionAddress: 000000005449da44 (msvcr80!_wcsicmp_l+0x00000000000000f4)

So there seems to be an issue with that dll... SSMS and/or all SQL Server Tools rely on that dll
Maybe your tools also need an update... as this dll also comes from .Net ... you might check your .Net environment on that server
https://support.microsoft.com/en-us/topic/microsoft-net-framework-repair-tool-is-available-942a01e3-5b8b-7abb-c166-c34a2f4b612a

What I am curious about...
I think you have another .Net Framework installed on that machine which is also in your $PATH and that one is also "old"...

FAULTING_SOURCE_LINE: f:\dd\vctools\crt_bld\self_64_amd64\crt\src\wcsicmp.c
FAULTING_SOURCE_FILE: f:\dd\vctools\crt_bld\self_64_amd64\crt\src\wcsicmp.c

So I suggest
-ignore the faulty SQL Server engine
-patch/install/fix all of the other issues with newer releases
-try again with sqlserver start

Answer 3

Hi @Guotao Huang ,

For debugging the crash dump files, it is beyond what we can do in the forum, if you want to know more, it is recommend that you contact Microsoft Customer Support Service (CSS) for assistance so that this problem can be resolved efficiently.
Application Service crashing - Faulting module name - MSVCR80.dll
1.Try to run a System File Checker (SFC) in CMD scan to fix issues related to system file corruption:

sfc /scannow

2.Try to register the concerned .dll file in CMD：

takeown /f MSVCR80.dll
regsvr32 MSVCR80.dll

For some advice:
It is suggested that upgrade your SQL Server to the latest version

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 4

Hello

Thank you for your question and reaching out. I can understand you are having issues related to SQL server not starting after update.

1. MSVCR80.dll is related to Visual Studio 2005 or 2008 run time , So Please uninstall it and re-install with latest version.
2. Disable any Antivirus program or Windows firewall you may have for temporary purpose.
3. Get the source of SQL server 2016 and try to repair from Programs and Features.
4. Please try to Re-install Dot.net feature from Server Manager -> Roles and Features
5. Please check if MSVCR80.dll is exists or not in C:\windows\System32\
6. Run sfc /scannow
7. Run below DISM comsmand from elevated prompt.

DISM /Online /Cleanup-Image /CheckHealth
DISM /Online /Cleanup-Image /ScanHealth
DISM /Onliane /Cleanup-Image /RestoreHealth

--------------------------------------------------------------------------------------------

--If the reply is helpful, please Upvote and Accept as answer--
PRAKASH T