A couple of things to check
1) when you click on VM and navigate to Support + troubleshooting and in screenshot do you see the windows screensaver and in resource health status do you see any health events
2) Can you connect to the Virtual machine via Serial console
3) Probably you may have tried this but did you try to redeploy the VM and even before doing did you try to reset the RDP configuration from the Portal
4) Once you are able to login in the event viewer do you see any out of the ordinary events for the RDP issue
5) Last but not the least in VM extension tab do you see any failed extension, I will suggest just redeploy the VM first and then check
For the issue that you have reported that it takes time to start and stop, have seen these behaviour when api at times takes a while to stop start or reboot or redeploy a VM