@Anthony John Hinay Apologies for the delay in response and all the inconvenience caused because of the issue.
In order to do RDP to the Linux you should be following the steps as outlined here and see how it works for you. Assuming you might have already covered this up but still if not yet you can try to check if the mentioned settings are in place or not.
Another thing is that there are different methods to login to Azure VM. One such method is SSH as well. You can refer to this as well to understand how it works for Linux VM.
As for your RDP performance, make sure you are hosting your VM in the closest datacenter. http://azurespeedtest.azurewebsites.net/ is a quick way to determine which datacenter is likely to perform the best for you. Check your RDP settings are correct - in the Remote Desktop Connection client app go to the Experience tab and make sure the dropdown is set to 'Detect connection quality automatically'.
If still the issue persist do let me know we will try to help you in best way possible.
Hope it helps!!!
Please "Accept as Answer" if it helped so it can help others in community looking for help on similar topics.