RDP into VM issue

Question

I have a virtual machine on server 2019 hosting some apps and a SQL Database. Seemingly randomly during the day everyone who is RDP / MSTSC to the VM has their window freeze, we can still transact SQL data from external sources, the machine responds to pings and looks 'healthy' on the host, however, anyone with an open RDP window can't do anything, and no fresh RDP can open.

Users might see this pop up bottom right on their session. Clicking end or cancel has the same effect (nothing).

The only way to restore RDP access appears to be a VM reboot.

Trying to find a common denominator before the power is cycled from the host, the only thing that looks close is a winlogon event "The shell stopped unexpectedly and explorer.exe was restarted."

I've disabled UDP via RDP via group policy but the issue persists. Clients are a mix of Win10 and Win11. Have also tried the windows store remote desktop app and issue persists. From the host I notice that heartbeat comms is lost, but I don't know if that is the cause or an outcome of something within the VM.

Answer 1

Can you establish your own RDP session, launch task manager, and then just leave it sit there so when this problem occurs you can do basic performance analysis of running processes/cpu/memory/disk/network?

Does your RDP session also hang? Can you log on to the console of the VM?

If it's a temporary hang, the task manager graphs should still show something. Do you see a cpu or memory spike? What about handle count?

Process Explorer can provide a lot of good info.

https://learn.microsoft.com/en-us/sysinternals/downloads/process-explorer

Answer 2

Does your virtualization software allow you to log on to the system console? Don't run mstsc (RDP) if it hangs.

Try PSExec, that will at least get you to a command prompt. Download it to your PC. Launch a Powershell session before the server hangs.

https://learn.microsoft.com/en-us/sysinternals/downloads/psexec

psexec \YourServerName powershell.exe -executionpolicy unrestricted

Also download pslist to the problem server.

https://learn.microsoft.com/en-us/sysinternals/downloads/pslist

It sounds like you have some performance hang. Unfortunately, there's really no way for us forum users to look inside of your server to see what is going on. So if psexec can get a working command (PowerShell) prompt at the point of the hang, you might be able to see what's running and maybe recover.

"pslist -accepteula -t" Show processes by parent. If you see a lot of processes in the leftmost column that might indicate that their parent process has terminated and they are hung.

Get-Process will show what programs are running and key metrics. Run it before the server hangs and then after the hang has occurred. Compare the values. Do you see more programs executing? Are more handles in use? Is more memory in use?

Use netstat to see if there are an excess number of network connections. "netstat -aon | findstr -i estab"

Try killing application processes, or stopping services. See if you can free up the server.

I wrote a PS script that I called RecentEvents.ps1 to collect the events from all event logs and sort them into TOD sequence. After the server has hung up and freed up, run this to look for messages that might indicate what happened.

https://docs.microsoft.com/en-us/answers/questions/102481/eventlog-madness.html