question

DreamCatcher-0722 avatar image
0 Votes"
DreamCatcher-0722 asked DreamCatcher-0722 commented

KB5001347 (April 2021 Update): Issue with Server Backup on Hyper-V Server 2016

Are there anyone else seeing issues with Windows Server Backup on Hyper-V Server 2016 after installing KB5001347? We can reliable reproduce failing backups at stage "system state".

After uninstalling KB5001347 backups run as expected again. After re-installing KB5001347 backups stop working again.

The error message(s) in backup management console is/are:
- The mounted backup volume is inaccessible. Please retry the operation.
- The operation ended before completion.

The following error(s) is/are logged in event viewer:
- 517, Backup: The backup operation that started at 'UTC‎TIMESTAMP' has failed with following error code '0x8078004F' (The mounted backup volume is inaccessible. Please retry the operation.). Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
- 10170, HYPER-V-VMMS: Requester reported unsuccessful backup for the virtual machine 'MACHINENAME'. (Virtual machine ID GUID).

Thank you in advance!








windows-server-2016windows-server-hyper-vwindows-server-backup
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

What are you using as the backup destination? I have been waiting for this type of inquiry as I have observed the same thing happening with WIndows 2016 and 2019, both serving as hyper-v hosts. I'm also getting the message 'The mounted backup volume is inaccessible'. This is happening when I attempt the backups to a synology NAS device.

I have been able to backup successfully to a windows-based network share.

Like you, I can remedy by removing KB5001347.

0 Votes 0 ·

At our site it is a Synology NAS device too. It‘s regardless of using SMBv2/3.

0 Votes 0 ·
jiayaozhu-MSFT avatar image
0 Votes"
jiayaozhu-MSFT answered

Hi,

Thank you for your posting!

Based on your descriptions and the link that you gave us, I would like to check first that have you tried the ways mentioned in the link that you yourself found. If so, can you give us some information about the result?

Secondly, based on these three error codes: 10170, 2155347997 and '0x8078004f', your issue may be related to VSS issue. Specifically, you may encounter backup corruption or configuration file corruption when your VSS requestor is trying to backup your server.

Here is an article that covers a bunch of available ways for you to resolve your issue:

https://www.auslogics.com/en/articles/fix-errors-0x807800c5-and-0x8078004f/

Please note: Information posted in the given link is hosted by a third party. Microsoft does not guarantee the accuracy and effectiveness of information.

You may also need to check if there is any offline backup destination shown after rescheduling backup with fresh destination. Remove these offline backup destinations and this shall mostly resolve the issue.

Thank you for your support!

Best regards
Joan


If the Answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

DreamCatcher-0722 avatar image
0 Votes"
DreamCatcher-0722 answered

Hi @jiayaozhu-MSFT,

I've already checked via DISM/SFC for corruption in component store. There is no issue at all. The VSS writers are working correctly too. As @DougDafforn-8218 wrote this seems to be related saving backups to a Synology NAS.

The bunch of articles you mentioned and a lot more of them I processed before I opened this thread. ;-)

Interestingly I monitored that some of the affected systems started backup automatically after (un-)installing KB5001347 (regardless of the monitored result).

Thank you

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

LadislavZelinka-1921 avatar image
0 Votes"
LadislavZelinka-1921 answered DreamCatcher-0722 commented

I have same state.
After update all server, all windows backup work not.
I run windows backup from commandline:
wbadmin.exe start backup -backupTarget:\\192.168.223.6\backup\Banka\2 -user:XXXXX\XXXXX -password:XXXXX -include:d:\Banka -vsscopy -quiet

and result:
*wbadmin 1.0 - Backup command-line tool
(C) Copyright 2013 Microsoft Corporation. All rights reserved.
Note: The backed up data cannot be securely protected at this destination.
Backups stored on a remote shared folder might be accessible by other
people on the network. You should only save your backups to a location
where you trust the other users who have access to the location or on a
network that has additional security precautions in place.
Retrieving volume information...
This will back up DATA(D:) (Selected Files) to \\192.168.223.6\backup\Banka\2.
The backup operation to \\192.168.223.6\backup\Banka\2 is starting.
Creating a shadow copy of the volumes specified for backup...
Please wait while files to backup for volume DATA(D:) are identified.
This might take several minutes.
Creating a shadow copy of the volumes specified for backup...
Please wait while files to backup for volume DATA(D:) are identified.
This might take several minutes.
Scanning the file system...
Please wait while files to backup for volume DATA(D:) are identified.
This might take several minutes.
Found (0) files.
Scanning the file system...
Found (0) files.
Scanning the file system...
Found (0) files.
Scanning the file system...
Found (0) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
Scanning the file system...
Found (5) files.
File identification is complete.
The backup of volume DATA(D:) to \\192.168.223.6\backup\Banka\2 is starting...
The backup of volume DATA(D:) could not be completed. Error: The mounted backup volume is inaccessible. Please retry the operation.
Summary of the backup operation:


The backup operation stopped before completing.
The backup operation stopped before completing.
Detailed error: The mounted backup volume is inaccessible. Please retry the operation.
Log of files successfully backed up:
C:\Windows\Logs\WindowsServerBackup\Backup-20-04-2021_07-39-36.log
Log of files for which backup failed:
C:\Windows\Logs\WindowsServerBackup\Backup_Error-20-04-2021_07-39-36.log
There was a failure in preparing the backup image of one of the volumes in the backup set.
The mounted backup volume is inaccessible. Please retry the operation.*

I send backup to Synology too. Local backup is OK

· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I try this:
Command: wbadmin.exe start backup -backupTarget:\\192.168.228.6\backup\Banka\1 -user:BckUpZA01\Backup -password:XXXXXXXX -include:d:\Banka -vsscopy -quiet
command is ended with error:
...
The backup of volume DATA(D:) could not be completed. Error: The mounted backup volume is inaccessible. Please retry the operation.
...

WireShark:
32594 117.188137 192.168.228.2 192.168.228.6 SMB2 186 Session Setup Request, NTLMSSP_NEGOTIATE
32595 117.188735 192.168.228.6 192.168.228.2 SMB2 300 Session Setup Response, Error: STATUS_MORE_PROCESSING_REQUIRED, NTLMSSP_CHALLENGE
32596 117.189043 192.168.228.2 192.168.228.6 SMB2 342 Session Setup Request, NTLMSSP_AUTH, User: PNM\ZILINA01$
32597 117.194366 192.168.228.6 192.168.228.2 SMB2 131 Session Setup Response, Error: STATUS_LOGON_FAILURE

Why PNM\ZILINA01$, it is source computer account ? In command is BckUpZA01\Backup !!!!!

1 Vote 1 ·

I think things you discovered go into the right direction. There seem to be other authentication issues with Fujitsu Server View Power Monitoring and KB5001347 too, I only saw in dependency of authentication issues.

Microsoft is very quiet about the support request I opened. Maybe they are working on a fix and hopefully release it with may's cumulative update.

1 Vote 1 ·

I am now ok, Synology support fix my NAS-es. Now work backup well.

0 Votes 0 ·

Strange. Synology support told me I have to wait until official patch. :-(

0 Votes 0 ·
jiayaozhu-MSFT avatar image
0 Votes"
jiayaozhu-MSFT answered

Hi,

Thank you for your reply!

Based on your descriptions, I suppose your issue may be more related to Synology product. For one thing, you can backup successfully to a windows-based network share. For another, when you uninstall KB5001347, your backup is working successfully, So can we presume that there is somthing imcompatible between your Synology and our KB5001347, since if your issue only occurs when backing up to Synology. So I may suggest you to seek for assistance from support engineers of Synology, to see if they have any ideas about this issue.

Thank you for your understanding!

Best regards
Joan


If the Answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

DreamCatcher-0722 avatar image
0 Votes"
DreamCatcher-0722 answered

@jiayaozhu-MSFT, I think you (mostly) summarized correctly. But I think further Microsoft engineers should take a look at this, because only this update runs into this issue. All previous versions were running with out any issue!

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

jiayaozhu-MSFT avatar image
0 Votes"
jiayaozhu-MSFT answered DreamCatcher-0722 commented

Hi,

Thank you for your patience!

After our discussion, my colleagues did many researches and I think I can offer you the following options, you can choose one from them based on your own condition:

1) Your issue needs further analysis of logs while it is beyond our Q&A scope, so if you want to further trace your backup logs to see why this issue happen, you may be suggested to directly seek for help from our engineer. In addition, if the issue has been proved as system flaw, the consulting fee would be refund. You can contact our engineers via:

https://support.microsoft.com/en-us/topic/global-customer-service-phone-numbers-c0389ade-5640-e588-8b0e-28de8afeb3f2

2) if just like our discussion, this issue is related to KB5001347 version, that is, the product design issue, then you can pend using this version and we can help you to feedback your issue to our product service group. However, since KB5001347 is the latest version, you may need to wait for a couple of days until our product release the next version, then you can try to install the next version.

3) You can also post your feedback on our KB5001347 version on the website:

https://windowsserver.uservoice.com/forums/295047-general-feedback

Thank you for your understanding and support! And If possible, can you "Accept Answer" for me? It seems to be a general issue and if you accept answer, then other people who have the same issue with yours can have a quick way to find their solutions.

Best regards
Joan


If the Answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks for your response, Joan. I'll open a regular support ticket, as soon as I've received my access ID. Further I committed this issue to Windows Server User Voice: https://windowsserver.uservoice.com/forums/295047-general-feedback/suggestions/43232229-update-kb5001347-breaks-windows-server-system-stat

0 Votes 0 ·

Thank you for your reply!

So can you "Accept Answer"? Your support will be our best motivation.

Hope you can solve your issue later, and you can also share your experience with us in this blog to let other people get access to this issue.

Thank you for your support! : )

BR,
Joan

0 Votes 0 ·

Well, I'm currently not sure to mark this as accepted answer. The original issue hasn't been solved at all. I think it would be better to wait until we're all able to provide a solution for the audience, isn't it?

1 Vote 1 ·
DougDafforn-8218 avatar image
1 Vote"
DougDafforn-8218 answered DougDafforn-8218 edited

@DreamCatcher-0722 FYI I opened a ticket with Synology yesterday but have yet to hear anything back.

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I made it too
https://account.synology.com/en-global/support/2784859/detail

2021-04-21 11:17:59
Dear customer,

Thanks for contacting Synology Technical Support.

If your SMB service works fine on another Windows I suspect that something wrong is with your updated Windows Server. From my expierence I can recommend to check SMB version on server and NTLM settings on it. If you are using DSM 7.0 your Windows needs to have enabled NTMLv2 (Send NTLMv2 response only). Please also check which version of SMB are you using on your DSM. If you are using SMB1 you need to enable it on Windows device. Below you can find more information about Windows configuration.


https://docs.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/network-security-lan-manager-authentication-level
https://docs.microsoft.com/en-us/windows-server/storage/file-server/troubleshoot/detect-enable-and-disable-smbv1-v2-v3
.....
== NOTHING :(

1 Vote 1 ·

I had a conversation with synology yesterday and they reviewed the logs of their device in relation to the moment of failure in the backup. I use a product called Backupassist that sits on top of windows backup. It is at the very end of the backup after all drive data is backed up that an 'application backup' quickly sweeps across the VMs and this is where the backup fails. The synology tech speculated that something has potentially changed in the vss writer due to the April cumulative update, or that possibly there is an NTFS command occurring at this final stage that is causing a problem since the synology device does not use NTFS. The NTFS command idea is interesting since I can successfully run the same backup jobs to a windows-based share without issue. I have also reached out to Backupassist to see what is occurring in this final VM application backup stage. Is it some kind of checkpoint removal that is failing?
@DreamCatcher-0722
@LadislavZelinka-1921

1 Vote 1 ·

That sounds great. We would be happy to get further information from your site about the ongoing process and what Synology says.

0 Votes 0 ·
ZoliSch-8379 avatar image
0 Votes"
ZoliSch-8379 answered ZoliSch-8379 edited

Hi,

I have same problem with 2012 R2 server, after updated with KB5001382. Can not backup Hyper-V guests to Synology NAS.
After removed the update, work fine again.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

DreamCatcher-0722 avatar image
0 Votes"
DreamCatcher-0722 answered ZoliSch-8379 commented

Hi all,

I've created a new service request directly at Microsoft. I'll update here, if I get any news regarding this issue.

I told the service engineers the link to this thread.

· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@DreamCatcher-0722 Thanks for the update. I heard from the developer of the backup software that I use that they were working on a resolution. This was on 4/22 and the claim was they were close to a resolution, but I have not heard anything further. That doesn't necessarily help anyone else directly, but my thought was that they were likely communicating with MS to accomplish this.

2 Votes 2 ·

Any update for this? I had April KB5001393 fail, a few weeks ago and cause access to our Synology NAS to break. Manually downloading and installing it did not help, nor did removing it. Checking now for updates does not present KB5001393 as being available.

I have 3 other 2012R2 servers backing up to the Synology, and the affected server (hopefully temporarily) backing up to a USB drive.

0 Votes 0 ·

The Update for this is to wait for the fix from Synology. Microsoft won't provide a fix for this as it is a Synology related issue. I think this month could be an option for next DSM release. v7 is currently in release candidate stage.

2 Votes 2 ·
Show more comments
RadoSK-1715 avatar image
0 Votes"
RadoSK-1715 answered

Same issue.
KB5001382
KB5001393
Source W2012R2 wbackup GUI
Target Synology NAS SMB2
An error occurred, when seeing: "scanning (destination) file system"
In the Wireshark log, authentication failed:
SMB2 Session Setup Request, NTLMSSP_NEGOTIATE
SMB2 Session Setup Response, Error: STATUS_MORE_PROCESSING_REQUIRED, NTLMSSP_CHALLENGE
SMB2 Session Setup Request, NTLMSSP_AUTH, User: \
SMB2 Session Setup Response, Error: STATUS_LOGON_FAILURE

Backup software does not send the right account credentials. I think. (User: )

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.