How to optimize Azure Data FactoryPipeline?

Anonymous
2020-11-30T19:07:13.233+00:00

Hi All, Need help on the below requirement... I have to perform ADF migration from 90(source and target Azure MI databases) For Example Source on-prem servers(data server1, data server2, data server3,...data server90) to Target Azure MI Servers(Azure MI Database1, Azure MI Database2, Azure MI Database3,....Azure MI Database90) On-Prem dataserver1 db1 (Self Hosted Integration Runtime1)---> Azure MI Database1 My question is, do I need to create 90 (Self Hosted Integrated Runtime for each on-prem source data servers) and 90 pipelines for this similar activity, or is there a way to optimize it so that one self-hosted integration runtime service and same pipeline can be used for all 90 migration? Any help or suggestion would be a great help as I am new to ADF... Below are my pipeline activities... ![43766-pipelineactivities.png][1] [1]: /api/attachments/43766-pipelineactivities.png?platform=QnA

Azure Database Migration service
Azure Migrate
Azure Migrate
A central hub of Azure cloud migration services and tools to discover, assess, and migrate workloads to the cloud.
717 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,528 questions
SQL Server Integration Services
SQL Server Integration Services
A Microsoft platform for building enterprise-level data integration and data transformations solutions.
2,452 questions
0 comments
Accepted answer
  1. MartinJaffer-MSFT 26,021 Reputation points
    2020-12-01T00:38:32.09+00:00

    Hello anonymous user and welcome to Microsoft Q&A.

    If all of your on-prem databases are in the same network, then you only need one Self-Hosted Integration Runtime (SHIR). If you had 90 different physical locations, each with its own server, network, and firewall, then you would need one SHIR in each location.

    Given the size of your migration, you may want to make your SHIR more robust by giving it more nodes. This allows for high-availability and scaling. This can help increase throughput.

    A node is a machine with the SHIR software installed on it. A SHIR consists of at least one node. A SHIR can have multiple nodes by installing the software on multiple machines, and registering them all with the same key. There are a few more steps. See the linked article for details.

    1 person found this answer helpful.

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Monalv-MSFT 5,891 Reputation points
    2020-12-01T09:05:59.943+00:00

    Hi anonymous user ,

    Hope the following link will be helpful:

    Migrate on-premises SQL Server or SQL Server on Azure VMs to Azure SQL Database using the Data Migration Assistant

    Best Regards,
    Mona

    ----------

    If the answer is helpful, please click "Accept Answer" and upvote it.
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
    Hot issues in November--What can I do if my transaction log is full?
    Hot issues in November--How to convert Profiler trace into a SQL Server table?