How to Register an on-prem Self-Hosted Integration Runtime for Azure Data Factory that uses a Private Endpoint

Question

I have configured an Azure Data Factory using a private endpoint w/ private DNS for access. I am wanting to be able to connect to local resources within Data Factory. I have created a self-hosted integration runtime within DF Studio and downloaded the Integration Runtime to an on-premises server. When i attempt to register the runtime using the authentication key generated when creating the SHIR in DF, it cannot successfully register. It essentially keeps saying the authentication key is wrong or blank, looking at the windows event logs for the IR, it has HTTP request failed errors and says the key is incorrect. But if i switch Data Factory to Public Access, the registration completes successfully using the same exact key. So i know the authentication key is correct. There is just some unknown disconnect when trying to reach the services needed to register the IR when DF is behind a private endpoint.

We have in our local DNS servers, set up conditional forwarders for the datafactory private dns zones so that the azure FQDN for our DF resolves to the correct private IP address. I have confirmed that this DNS resolution is working by doing an NS lookup of the private FQDN of DF on the local server and seeing that it does indeed resolve to the correct IP.

Is there something else that needs to be done in order for the IR to communicate to the DF resources so that it can register when DF is behind the Private Endpoint.

Answer 1

Hi@Aaron Hurst
Welcome to Microsoft Q&A platform and thanks for posting your question here.

To register an on-prem self-hosted integration runtime for Azure Data Factory that uses a private endpoint, you need to ensure that the integration runtime can communicate with the Data Factory resources. Here are some steps you can take to troubleshoot the issue:

• Ensure that the self-hosted integration runtime (SHIR) is installed on the on-premises server and running correctly.
• Check that the SHIR is configured to use the correct authentication key generated when creating the SHIR in Data Factory.
• Verify that the private endpoint for Data Factory is correctly configured, and that the conditional forwarders for the Data Factory private DNS zones are set up correctly.
• Check the Windows event logs for the SHIR to see if there are any HTTP request failed errors or other errors that may indicate a problem with the registration process.
• If the registration process fails, try switching Data Factory to public access and see if the registration completes successfully using the same authentication key. If it does, then there may be an issue with the private endpoint configuration.
• Ensure that the on-premises server has the necessary permissions and access rights to communicate with the Data Factory resources.

You can find more information on how to configure a self-hosted integration runtime for Azure Data Factory in the official Azure documentation: https://docs.microsoft.com/en-us/azure/data-factory/create-self-hosted-integration-runtime. If you continue to experience issues, please provide more details about the error messages you are receiving, and we can assist you further.

I hope this information helps you. Let me know if you have any further questions or concerns.