Hyper-V Crashes and Hyper-V-VMMS Event 16140

Question

I am running into either one issue, or two separate issues, with Windows Server 2022 running about 6 Hyper-V virtual Windows Servers.

Twice now, the Hyper-V Virtual Machine Management service has crashed, and although the VMs do seem to remain running, the service will not restart until I reboot the server.  I typically find out when this happens because I get backup alerts stating that the nightly backups are failing.

When researching this issue, I found an interesting Event Log that may or may not be related.  This particular Event Log is almost nonexistent when I look for it in search engines.  When I do find it, I get very vague descriptions of what it means:

The event is Hyper-V-VMMS 16140:

'SERVER' cannot delete file. The system cannot find the path specified.  (0x80070003). (Virtual machine ID [Followed by the virtual machine ID])

Interestingly, the Event Log only seems to happen for the virtual servers that are being replicated to an older, Windows Server 2016 Hyper-V server.  The replication is taking place via Hyper-V replication.

I am not sure how to prevent these crashes, and (if it's related) how to solve the 16140 error.  My research has led me to some tweaking of Hyper-V folder permissions but so far, nothing has seemed to work.

My goal is to prevent these crashes and my only theorized way of doing this is to solve the mystery 16140 error, which is difficult to research due to there being almost no available information about it.

Can anyone help here?

Answer 1

Hi,

What errors do you see besides the event 16140 when the Hyper-V Virtual Machine Management service crashes?

It seems some virtual machine files cannot be found. Please make sure the files still exist. If all the files are there you can create a new virtual machine with the existing virtual disk and see if the event 16140 occur to the new one.

Event ID 16140 — Virtual Machine Miscellaneous Operations | Microsoft Learn

Answer 2

Ian, thanks for your response.
I won't know for sure until it crashes again.

The '16140' error happens every 15 minutes, which led me to think maybe it was a 'cause' of the crash.

All files, that I know of, exist. The virtual machines run fine. The only thing I can think of is that there are some sort of support files that I don't know of that do not exist. It's very confusing and hard to pinpoint an answer. The articles, like the one you linked to, don't describe anything in enough detail to actually resolve the issue.

Thanks!

Answer 3

Can the VM be stopped/started normally? Does the event 16140 persist if you remove the VM and create a new one with the virtual disk of that?

Since VMMS runs as SYSTEM, I think it have the necessary permissions on the VM files.