Yes you ideally need to access these VMs using their Private DNS names if using VNET peering or VNET GW to reach the resources. If you wish to use Azure provided Azure DNS names, you can configure private DNS Zone and integrate your VNET to it, here are more details. You can also implement a custom DNS, here are details for the same. Here are some details regarding vnet peering and how it works.
However, after some investigation, I figured that you can also use public DNS names if needed. You can achieve this using the hosts.txt file on Windows, or by adding a record to your local DNS server (if the computers use a local DNS, not a public one).
Please let me know if you have any further questions and we will be glad to answer them for you. Thank you!