Thank you for reaching out to us.
As you are unable to resolve the FQDNs of the devices, it is required to set-up a DNS solution to achieve this. You can go through the various scenarios described here to choose your DNS solution as per your requirements.
If you need to resolve on-prem services in Azure and vice versa, you can use Azure DNS Private Resolver to achieve this. Azure DNS Private Resolver is a new service that enables you to query Azure DNS private zones from an on-premises environment and vice versa without deploying VM based DNS servers. You can go through this documentation for implementation.
Hope this helps. Please let me know if you have any additional questions. Thank you!