Performance of Switch activity in Azure Data Factory

Joao Leal 0 Reputation points
2024-01-25T16:39:58.7833333+00:00

Can anyone explain me what is happening in here? We are using a switch in a pipeline that has about 15 different cases, each case will execute a different pipeline. The performance seems really bad and I don't understand why that's the case.

  1. On the first example, the pipeline that we call inside the switch takes 2m58s to run, but the switch takes 16m 1s: Pipeline run ID: ff03f80b-7a73-498b-9116-4c7e784aaded Activity run ID: 5cdc4c6b-04bb-447a-9aae-235a89cfaaf2 User's image
    1. On the second example, the Switch activity only starts 3m after the previous activity finished and then, once again, the activity inside it takes only 1m but the switch itself only finishes after almost 4m.
    Pipeline run ID: 25d67466-4334-4649-8498-d5bdbe7664e0 Activity run ID: 1b781a5a-a59a-4709-b2f2-443f5c68e6b6 User's image
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,624 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Subashri Vasudevan 11,226 Reputation points
    2024-01-30T14:25:38.7066667+00:00

    Looks like the above expression has some syntax issue. It will be

    @pipeline().parameters.step_name

    And if it works fine automatically without any changes to the logic most probably it should be an internment networking issue that has caused this slowness.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.