we also see this on occasion and I'm looking for a way to prevent. We typically see the disconnected/stale session and the VM will show its still stopped/deallocated as shown above. We do what was said above - start the vm and logoff the disconnected session and then the host will show available again. Sometimes you can see in Boot Diagnostics that Windows didn't start successfully and the screenshot shows it's waiting for a response to continue or not (which obviously no one can respond to), so we end up stopping and starting the VM again.
Trying to understand if there is something to do to prevent the situation - there wasn't an session for the host before the user tried to connect (typically first thing in the morning). We make sure all sessions are logged off at the end of the day and stop their VMs, so in the morning when they connect, it should start on connect and most times it works successfully. On the Windows not loading, is there a Windows setting that would have Windows automatically try to restart again vs. prompting for response? Trying to find ways to handle these few exceptions. Also, we've never had to recreate the Host Pool - just get the VM up in a good state again, and end any disconnected sessions if they exist after the initial attempt to start on connect....