Will ADF For Each Parallel maintain batch count consistently ?

Pranay Katta 21 Reputation points
2021-06-22T12:58:36.943+00:00

Hello,

If I set IsSequential to false, Batch count as 5 and total items as 100. Will adf consistently execute 5 batches in parallel even if one of the activities gets completed with in the first 5 but starting up a new activity ?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,196 questions
0 comments No comments
{count} votes

Accepted answer
  1. ShaikMaheer-MSFT 38,401 Reputation points Microsoft Employee
    2021-06-23T06:12:11.083+00:00

    Hello @Anonymous ,

    Welcome to Microsoft Q&A Platform.
    Thank you for posting query on Microsoft Q&A Platform

    Below is the detailed Explanation:

    If you set IsSequential to false, Batch count as 5 and total items as 100.

    In this case, we will having total 100 iterations as we have total items are 100. And Since we have batch count as 5, ADF will execute 5 iterations at a time at any given point of time.

    So, ADF will start Iterations1,2,3,4,5 at same time and let's say iteration3 gets completed very soon and iterations 1,2,4,5 are still running then immediately ADF will take Iteration6 for execution because we defined batch count 5 so its ADF job to maintain 5 parallel iterations execution at any given point of time.

    Hope this helps you to understand better. Thank you.

    ----------------------

    • Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a email-notifications
    1 person found this answer helpful.
    0 comments No comments

0 additional answers

Sort by: Most helpful