Share via

VSS SQL Writer fails, looks at wrong registry location for WID ADSync database

Anonymous
2024-02-23T14:04:19+00:00

Hello,

We have a rather odd situation on our hands. A server 2016 machine was setup years ago to run our Active Directory Sync Service. Back then, the server was a regular machine without a specific role, and the ADSync Service was setup running as a local user as recommended.

A few years later this machine was promoted to a DC to replace our old SBS2008-machine. In the beginning of this year we noticed that something was off, event logs showed repeated fails of the automated update process for ADSync and somewhere along trying to get it to install the sync service failed completely.

After some head scratching I finally found out why (on the topic of: make something idiot proof, and the world invents a better idiot!). ADSync on a DC should not be run by a local service account, but rather a domain account. So our config had gotten illegal due to the promotion, but had not manifested itself until a specific update requirement.

I tried at first to create a new AD user for the service, and giving sysadmin role to it on the WID instance but no luck. As we have very simple sync rules, the next easiest option was to spin up a VM of an old backup of the server with a working ADSync, note down the config, uninstall ADSync from the current server and start from scratch.

This broke something though, because our backups (using UrBackup) started to fail, getting event 12289 from WIDWRITER in the event logs.

There is no description for this error, but it includes this:

RegOpenKeyExW(FFFFFFFF80000003,[SID goes here]\Software\Microsoft\Microsoft SQL Server\UserInstances,...)
0x80070005

This is under HKEY_USERS, the SID belongs to the domain administrator, the UserInstances key exists but it is empty.

The same key exists under the new domain user created to run ADSync. It contains two subkeys, one for the ADSync 2019 instance (LOCALDB#SHF48841) and one for the MSSQLLocalDB (LOCALDB#84A994F6).

In SQLWriterLogger.txt I can find this block after a failed backup: 

[02/23/2024 00:04:28, TID 105c] ****************************************************************
[02/23/2024 00:04:28, TID 105c] **  VSS SQL BACKUP BEGIN - ID: 105c
[02/23/2024 00:04:28, TID 105c] ****************************************************************
[02/23/2024 00:04:28, TID 105c] Component based backup selected.
[02/23/2024 00:04:28, TID 105c] Database count from metadata is 4
[02/23/2024 00:04:28, TID 105c] Database master on instance RADIO1\LOCALDB#SHF48841 found in metadata
[02/23/2024 00:04:28, TID 105c] Database model on instance RADIO1\LOCALDB#SHF48841 found in metadata
[02/23/2024 00:04:28, TID 105c] Database msdb on instance RADIO1\LOCALDB#SHF48841 found in metadata
[02/23/2024 00:04:28, TID 105c] Database ADSync on instance RADIO1\LOCALDB#SHF48841 found in metadata
[02/23/2024 00:04:28, TID 105c] Backup type is VSS_BT_FULL
[02/23/2024 00:04:40, TID 105c] Entering SQL Writer OnPrepareSnapshot.
[02/23/2024 00:04:40, TID 105c] SQLInstanceNames Registry was not found
[02/23/2024 00:04:40, TID 105c] Enumerating Local DB User Instances
[02/23/2024 00:04:40, TID 3814] HRESULT:	0X80040E14
[02/23/2024 00:04:40, TID 4108] HRESULT:	0X80040E14
[02/23/2024 00:04:40, TID 4108] SQLSTATE:	42000	Native Error:	3013
[02/23/2024 00:04:40, TID 3814] SQLSTATE:	42000	Native Error:	3013
[02/23/2024 00:04:40, TID 3814] Error state:	1	Severity:	16
[02/23/2024 00:04:40, TID 4108] Error state:	1	Severity:	16
[02/23/2024 00:04:40, TID 3814] Source:		Microsoft SQL Server Native Client 11.0
[02/23/2024 00:04:40, TID 3814] Error message:	BACKUP DATABASE is terminating abnormally.
[02/23/2024 00:04:40, TID 3814] HRESULT:	0X80040E14
[02/23/2024 00:04:40, TID 3814] SQLSTATE:	42000	Native Error:	18210
[02/23/2024 00:04:40, TID 3814] Error state:	1	Severity:	16
[02/23/2024 00:04:40, TID 4108] Source:		Microsoft SQL Server Native Client 11.0
[02/23/2024 00:04:40, TID 4108] Error message:	BACKUP DATABASE is terminating abnormally.
[02/23/2024 00:04:40, TID 4108] HRESULT:	0X80040E14
[02/23/2024 00:04:40, TID 1fe4] HRESULT:	0X80040E14
[02/23/2024 00:04:40, TID 1fe4] SQLSTATE:	42000	Native Error:	3013
[02/23/2024 00:04:40, TID 3814] Source:		Microsoft SQL Server Native Client 11.0
[02/23/2024 00:04:40, TID 3814] Error message:	BackupVirtualDeviceSet::Initialize: CoCreateInstance failure on backup device '{638784E3-54FF-4525-8F12-702CFFB0594E}2'. Operating system error 0x800703fa(Illegal operation attempted on a registry key that has been marked for deletion.).
[02/23/2024 00:04:40, TID 1fe4] Error state:	1	Severity:	16
[02/23/2024 00:04:40, TID 4108] SQLSTATE:	42000	Native Error:	18210
[02/23/2024 00:04:40, TID 3814] SqlConnection::ExecCommand: Error calling ICommandText::Execute.  hr = 0x80040e14.	SQLSTATE: 42000, Native Error: 3013	Error state: 1, Severity: 16	Source: Microsoft SQL Server Native Client 11.0	Error message: BACKUP DATABASE is terminating abnormally.	SQLSTATE: 42000, Native Error: 18210	Error state: 1, Severity: 16	Source: Microsoft SQL Server Native Client 11.0	Error message: BackupVirtualDeviceSet::Initialize: CoCreate
[02/23/2024 00:04:40, TID 3814] SqlConnection::ExecCommand: Throwing HRESULT code 0x8000ffff. Previous HRESULT code = 0x80040e14
[02/23/2024 00:04:40, TID 4108] Error state:	1	Severity:	16
[02/23/2024 00:04:40, TID 1fe4] Source:		Microsoft SQL Server Native Client 11.0
[02/23/2024 00:04:40, TID 1fe4] Error message:	BACKUP DATABASE is terminating abnormally.
[02/23/2024 00:04:40, TID 1fe4] HRESULT:	0X80040E14
[02/23/2024 00:04:40, TID 1fe4] SQLSTATE:	42000	Native Error:	18210
[02/23/2024 00:04:40, TID 4108] Source:		Microsoft SQL Server Native Client 11.0
[02/23/2024 00:04:40, TID 4108] Error message:	BackupVirtualDeviceSet::Initialize: CoCreateInstance failure on backup device '{638784E3-54FF-4525-8F12-702CFFB0594E}1'. Operating system error 0x800703fa(Illegal operation attempted on a registry key that has been marked for deletion.).
[02/23/2024 00:04:40, TID 1fe4] Error state:	1	Severity:	16
[02/23/2024 00:04:40, TID 1fe4] Source:		Microsoft SQL Server Native Client 11.0
[02/23/2024 00:04:40, TID 1fe4] Error message:	BackupVirtualDeviceSet::Initialize: CoCreateInstance failure on backup device '{638784E3-54FF-4525-8F12-702CFFB0594E}3'. Operating system error 0x800703fa(Illegal operation attempted on a registry key that has been marked for deletion.).
[02/23/2024 00:04:40, TID 4108] SqlConnection::ExecCommand: Error calling ICommandText::Execute.  hr = 0x80040e14.	SQLSTATE: 42000, Native Error: 3013	Error state: 1, Severity: 16	Source: Microsoft SQL Server Native Client 11.0	Error message: BACKUP DATABASE is terminating abnormally.	SQLSTATE: 42000, Native Error: 18210	Error state: 1, Severity: 16	Source: Microsoft SQL Server Native Client 11.0	Error message: BackupVirtualDeviceSet::Initialize: CoCreate
[02/23/2024 00:04:40, TID 4108] SqlConnection::ExecCommand: Throwing HRESULT code 0x8000ffff. Previous HRESULT code = 0x80040e14
[02/23/2024 00:04:40, TID 1fe4] SqlConnection::ExecCommand: Error calling ICommandText::Execute.  hr = 0x80040e14.	SQLSTATE: 42000, Native Error: 3013	Error state: 1, Severity: 16	Source: Microsoft SQL Server Native Client 11.0	Error message: BACKUP DATABASE is terminating abnormally.	SQLSTATE: 42000, Native Error: 18210	Error state: 1, Severity: 16	Source: Microsoft SQL Server Native Client 11.0	Error message: BackupVirtualDeviceSet::Initialize: CoCreate
[02/23/2024 00:04:40, TID 1fe4] SqlConnection::ExecCommand: Throwing HRESULT code 0x8000ffff. Previous HRESULT code = 0x80040e14
[02/23/2024 00:04:40, TID 4108] HRESULT EXCEPTION CAUGHT: hr: 0x8000ffff
[02/23/2024 00:04:40, TID 1fe4] HRESULT EXCEPTION CAUGHT: hr: 0x8000ffff
[02/23/2024 00:04:40, TID 3814] HRESULT EXCEPTION CAUGHT: hr: 0x8000ffff
[02/23/2024 00:04:40, TID 2e4c] HRESULT:	0X80040E14
[02/23/2024 00:04:40, TID 2e4c] SQLSTATE:	42000	Native Error:	3013
[02/23/2024 00:04:40, TID 2e4c] Error state:	1	Severity:	16
[02/23/2024 00:04:40, TID 2e4c] Source:		Microsoft SQL Server Native Client 11.0
[02/23/2024 00:04:40, TID 2e4c] Error message:	BACKUP DATABASE is terminating abnormally.
[02/23/2024 00:04:40, TID 2e4c] HRESULT:	0X80040E14
[02/23/2024 00:04:40, TID 2e4c] SQLSTATE:	42000	Native Error:	18210
[02/23/2024 00:04:40, TID 2e4c] Error state:	1	Severity:	16
[02/23/2024 00:04:40, TID 2e4c] Source:		Microsoft SQL Server Native Client 11.0
[02/23/2024 00:04:40, TID 2e4c] Error message:	BackupVirtualDeviceSet::Initialize: CoCreateInstance failure on backup device '{638784E3-54FF-4525-8F12-702CFFB0594E}4'. Operating system error 0x800703fa(Illegal operation attempted on a registry key that has been marked for deletion.).
[02/23/2024 00:04:40, TID 2e4c] SqlConnection::ExecCommand: Error calling ICommandText::Execute.  hr = 0x80040e14.	SQLSTATE: 42000, Native Error: 3013	Error state: 1, Severity: 16	Source: Microsoft SQL Server Native Client 11.0	Error message: BACKUP DATABASE is terminating abnormally.	SQLSTATE: 42000, Native Error: 18210	Error state: 1, Severity: 16	Source: Microsoft SQL Server Native Client 11.0	Error message: BackupVirtualDeviceSet::Initialize: CoCreate
[02/23/2024 00:04:40, TID 2e4c] SqlConnection::ExecCommand: Throwing HRESULT code 0x8000ffff. Previous HRESULT code = 0x80040e14
[02/23/2024 00:04:40, TID 2e4c] HRESULT EXCEPTION CAUGHT: hr: 0x8000ffff
[02/23/2024 00:04:41, TID 105c] HRESULT EXCEPTION CAUGHT: hr: 0x8000ffff
[02/23/2024 00:04:41, TID 105c] Freeze2000::WaitForPrepare: rethrowing
[02/23/2024 00:04:41, TID 105c] Freeze2000::WaitForPrepare: Throwing HRESULT code 0x8000ffff. Previous HRESULT code = 0x8000ffff
[02/23/2024 00:04:41, TID 105c] HRESULT EXCEPTION CAUGHT: hr: 0x8000ffff
[02/23/2024 00:04:41, TID 105c] Entering SQL Writer OnAbort.
[02/23/2024 00:04:50, TID 105c] Entering SQL Writer OnIdentify.
[02/23/2024 00:04:50, TID 105c] SQLInstanceNames Registry was not found
[02/23/2024 00:04:50, TID 105c] Enumerating Local DB User Instances
[02/23/2024 00:04:52, TID 105c] Entering SQL Writer OnPrepareBackup.

This corresponds well with what I see in the event log, but the questions are then:

  • Why/how
  • Can I get the SQL Writer to rebuild the information about the WID to remediate this error, or am I looking at yet an attempt to uninstall and reinstall the ADSync service?

Also, I don't see anthything about WSUS (which is also using the WID) anywhere, which is running on this server as well, and seems to be running fine.

Any information on this is greatly appreciated

BR

Alexander

Windows for business | Windows Server | Performance | Application technologies and compatibility

Locked Question. This question was migrated from the Microsoft Support Community. You can vote on whether it's helpful, but you can't add comments or replies or follow the question. To protect privacy, user profiles for migrated questions are anonymized.

0 comments

3 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Anonymous
    2024-02-26T03:04:02+00:00

    Hello

    Thank you for posting in Microsoft Community forum.

    Based on the description, I understand your question is related to the SQL.

    Since there are no engineers dedicated to SQL in this forum. in order to be able to get a quick and effective handling of your issue, I recommend that you repost your question in the Q&A forum, where there will be a dedicated engineer to give you a professional and effective reply.

    Here is the link for Q&A forum.

    Questions - Microsoft Q&A

    Click the "Ask a Question" button in the upper right corner to post your question and select "SQL" tag.

    Thank you for your understanding and support. If you have any question or concern, please feel free to let us know.

    Have a nice day.

    Best Regards,

    Molly

    0 comments
  2. Anonymous
    2024-02-26T14:50:04+00:00

    Hello Molly,

    Thanks for replying!

    It is a bit of a mixed bag, it is related SQL yes, but also Volume Shadow Services and its writers. Anyone here knowledgeable in VSS?

    Best regards

    Alexander

    0 comments
  3. Anonymous
    2024-02-27T02:20:20+00:00

    Hello Alexander,

    This VSS is SQL server backup and it is also under the SQL server topic as we can see: SQL Server Backup Applications - Volume Shadow Copy Service (VSS) and SQL writer - SQL Server | Microsoft Learn

    Since there are no engineers dedicated to SQL &VSS in this forum. in order to be able to get a quick and effective handling of your issue, I recommend that you repost your question in the Q&A forum, where there will be a dedicated engineer to give you a professional and effective reply.

    Best Regards,

    Molly

    0 comments