When you connect to the linked server, SQL Server must be able to pass you Windows credentials to the other machine. This is not difficult, if you logged on to Windows on the same machine as SQL Server is running on.
But if you logged on to SQL Server remotely, it's a different matter. For this you need to have Kerberos set up correctly. Kerberos is more of Windows thing than SQL Server thing, so you should probably talk with you Windows administrator.
Here is a link to a presentation about Kerberos and SQL Server by David Postlethwaite which explains this in more detail. But I will need to add the caveat that he is only talking about on-prem scenarios. I have an uncanny feeling that the fact that the VM is in Azure may add further complications.