ForEach Loop Copy Activity

grajee 346 Reputation points
2021-02-03T21:48:01.577+00:00

All,

I want to copy set of tables from SQLServer (On-Prem) onto AzureSQL database. I used the CopyData tool to set this one up and it works fine.

However, I want the tables to be copied in parallel. In the ForEach Loop screen, in the Settings Tab, The Sequential box is not checked which means it can execute in parallel. I set the "Batch Count" set to 30.

In the settings tab of Copy Activity, I have set "Degree Of Parellelism" to 32.

With all these settings, I see only two SPIDs on the source. Is there any way I can enhance the parallelism of the copy activity. Or else, I will have to create multiple pipelines.

https://learn.microsoft.com/en-us/azure/data-factory/control-flow-for-each-activity#parallel-execution.

Thanks,
grajee

Azure SQL Database
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,732 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Vaibhav Chaudhari 38,651 Reputation points
    2021-02-04T08:28:27.703+00:00

    When you run the pipeline, it should trigger multiple copy activities in parallel. Does this happen and did you check in Monitor section? Steps you followed seems correct and enough

    0 comments No comments