question

AbaniKumarDutta-1994 avatar image
0 Votes"
AbaniKumarDutta-1994 asked GettnBetter-8602 answered

Backup problem of Hyper-V machine

Hi
I am getting the error in events as per the screenshot attached here.

While taking backup of the VM , this error generated in my system.

I am using the operating system Windows Server 2016 Standard.

I have created 3 No's Guest Machine in this system with checkpoint enabled, No any checkpoint created yet.

Once the server is restarted backup is working for one to two days perfectly. Next same problem persists.

I have updated with the latest patches in my physical VM.

Need your kind support in this regard.

Error1
The description for Event ID 19100 from source Microsoft-Windows-Hyper-V-VMMS cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

Error2
Checkpoint operation for 'PICTADC' failed. (Virtual machine ID 370C6506-A6E5-4D2D-9E16-66463BA8A8C2)

windows-server-hyper-v
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I am getting above error in events when my backup is failed.

0 Votes 0 ·
MicoMi-MSFT avatar image
0 Votes"
MicoMi-MSFT answered

Hi,
Please open local security policy and check if NT VIRTUAL MACHINE\Virtual Machines is added to Log on as a service.
50252-image.png
Please run this command using Admin Privileges and try again:
Icacls “the path to the folder containing the VHDS files” /grant "NT VIRTUAL MACHINE\VM ID":(OI)F
Please make sure the Backup (Volums Shadow Copy) and the Guest Services are active as well.
50273-image.png
If the above doesn’t work, please make sure there is sufficient disk space on the VM and check if there is any error with your VHD.
Please run this command "vssadmin list writers" on your host and vm.
Try to restart the vss service and Hyper-V VMMS services and then try again.

Thanks for your time!
Best Regards,
Mico Mi


If the Answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.



image.png (90.2 KiB)
image.png (25.3 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

AbaniKumarDutta-1994 avatar image
0 Votes"
AbaniKumarDutta-1994 answered

Hi, I tried all above instruction but same problem persists.
I noticed once I restarted the VM backup is running with the thrid party software like Commvault.
Once vm restarted found the my Virtual hardisk .vhdx in path and backup was running after one day found backup is not working and Virtual hardisk path is changed to .AVHDX

Find screen shoot at below.

50848-image.png



image.png (12.4 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

MicoMi-MSFT avatar image
0 Votes"
MicoMi-MSFT answered

Hi,
Please run this command on your host and vm and check if there is any error: vssadmin list writers
Also please run this command to check if there is any checkpoint in your VM: Get-VMSnapshot -ComputerName“ MyHyperVHost” -VMName“VM”
Do you mean that you use the third party tool for backup? If so, maybe you can try to use Windows server backup and check if it works well.
Best Regards,
Mico Mi

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

AbaniKumarDutta-1994 avatar image
0 Votes"
AbaniKumarDutta-1994 answered

Hi,

I checked all the above action but same issue persists.

Tried windows backup also failed.

I observed that auto disk merging is being failed, creation of checkpoint being failed where standard checkpoint is also need to be created when production check point get failed as per my checkpoint settings.

When I tried manually merging the disk, it started to work. But after some days again .avhdx created and auto merging not happening as well as backup, checkpoint being failed.
Need your support like why merging is being failed. I completely explored from many angle and found it is a hard core hyper v issues.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

JamesBarrett-4319 avatar image
0 Votes"
JamesBarrett-4319 answered JamesBarrett-4319 edited

I have been having this same issue for quite some time now on Server 2012 R2. The backup creates a "hidden" snapshot when it runs and it intermittently fails to merge after the backup finishes for that VM. Any VM backups after the one with the failed merge will fail as the Hyper-V VSS is placed in an error state.

Shutting down the VM with the un-merged AVHD files and then restarting the Hyper-V Management Service will always successfully merge the AVHD file and reset the VSS error. Sometimes just restarting the Hyper-V Management Service merges the AVHD successfully (but not always) and resets VSS.

This happens on different Hyper-V VMs on different days, but it seems to happen most frequently on the VM with the most disk I/O (Our Exchange 2016 VM), so I believe it may be I/O related.

If it's consistently happening on the same VM for you, you may want to rename the VM in Hyper-V Manager so that it's alphabetically last. This way, if it fails, you still end up getting backups of all your VMs.

Out of curiosity, are you using Backup Exec?

I don't have a solution, but at least you now know you can get things working again without a reboot of the physical server.







5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

GettnBetter-8602 avatar image
0 Votes"
GettnBetter-8602 answered

I have this same problem it's getting so bad, some of our Hyper-V servers are crashing out of disk space with just a week of axhdx's that now refuse to merge. Using Backup Exec here, latest version. I have several tickets open with Veritas and they are blaming Microsoft on all tickets.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.