How to optimize Azure Data FactoryPipeline?

Question

Hi All, Need help on the below requirement... I have to perform ADF migration from 90(source and target Azure MI databases) For Example Source on-prem servers(data server1, data server2, data server3,...data server90) to Target Azure MI Servers(Azure MI Database1, Azure MI Database2, Azure MI Database3,....Azure MI Database90) On-Prem dataserver1 db1 (Self Hosted Integration Runtime1)---> Azure MI Database1 My question is, do I need to create 90 (Self Hosted Integrated Runtime for each on-prem source data servers) and 90 pipelines for this similar activity, or is there a way to optimize it so that one self-hosted integration runtime service and same pipeline can be used for all 90 migration? Any help or suggestion would be a great help as I am new to ADF... Below are my pipeline activities... ![43766-pipelineactivities.png][1] [1]: /api/attachments/43766-pipelineactivities.png?platform=QnA

Answer 1

Hello anonymous user and welcome to Microsoft Q&A.

If all of your on-prem databases are in the same network, then you only need one Self-Hosted Integration Runtime (SHIR). If you had 90 different physical locations, each with its own server, network, and firewall, then you would need one SHIR in each location.

Given the size of your migration, you may want to make your SHIR more robust by giving it more nodes. This allows for high-availability and scaling. This can help increase throughput.

A node is a machine with the SHIR software installed on it. A SHIR consists of at least one node. A SHIR can have multiple nodes by installing the software on multiple machines, and registering them all with the same key. There are a few more steps. See the linked article for details.

Answer 2

Hi anonymous user ,

Hope the following link will be helpful:

Migrate on-premises SQL Server or SQL Server on Azure VMs to Azure SQL Database using the Data Migration Assistant

Best Regards,
Mona

----------

If the answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
Hot issues in November--What can I do if my transaction log is full?
Hot issues in November--How to convert Profiler trace into a SQL Server table?