Azure Data Factory - Self Hosted IR

Question

We are using Azure Data Factory to copy SQL data from on premise to Azure Data Lake. For this we have installed Self hosted IR in one of the VM in On-Premise environment.

Is there a way around where we do not need to manage our own VM or run the Self Hosted IR to get the data from on-prem databases to Azure Data Lake using Azure data factory?

Answer 1

Hi @Gunwant ,

Welcome to Microsoft Q&A and thank you for your query.

In order to copy data from local sources to the cloud using Azure Data Factory, it needs to be able to connect to those sources. This can be achieved only by using the self-hosted integration runtime.

The self-hosted integration runtime is a service running in Azure Data Factory, but you can add local compute nodes on local servers in your on-premises network. The installation of a self-hosted integration runtime needs an on-premises machine or a virtual machine inside a private network. A connection is created between the nodes and the integration runtime within your Azure Data Factory (ADF) in Azure. Through this connection, ADF can reach your local data and copy it securely to the cloud. This set-up is very similar to the Power BI on-premises gateway. In fact, the self-hosted integration runtime used to be called the "data management gateway".

An alternative to using the self-hosted integration runtime (IR), is to use an on-premises ETL tool - like SSIS - to push the data to the cloud. However, this means you need a SQL Server license and a server. The self-hosted IR service is free to use and can be run on any local server. If you want to migrate your data infrastructure to the cloud, Azure Data Factory (and other Azure tools like Logic Apps) are a better choice.

Hope this info helps.

----------

Thank you

Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.