How to count number of activities in pipeline

Ryan Abbey 1,181 Reputation points
2021-08-03T05:04:54.23+00:00

I'm currently guessing that I'm hitting the failure "Triggering the pipeline failed due to large run size." because the process has determined there are 40 activities in my pipeline... however, if I count each box, I'm getting 37 so either I'm forgetting how to count, don't know what's in or out of the count or there's something else at play... anyone know what else causes a limit to be breached?

A 40 limit is something only Microsoft can come up with and coincide with making you jump the most ridiculous hoops to actually achieve something!

(Yes, I can go to a second pipeline but that means two pipelines to maintain... and I'm only hitting 37 activities because I have to use a switch since a Synapse notebook has to be hard coded, then catching the notebook output makes errors "be handled" so having to do additional failure on each and every notebook variant...!!)

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,804 questions
0 comments No comments
{count} vote

2 answers

Sort by: Most helpful
  1. Nandan Hegde 32,416 Reputation points MVP
    2021-08-17T09:59:31.827+00:00

    Hey @Ryan Abbey .
    can you please go through the below post : https://datasharkx.wordpress.com/2021/08/17/identifying-the-count-of-activities-within-a-pipeline-in-an-azure-data-factory/
    and try getting the count of activities in pipeline via powershell.

    Ideally my understanding is ADF wont allow to publish a pipeline with more than 40 activity

    1 person found this answer helpful.

  2. Vaibhav Chaudhari 38,746 Reputation points
    2021-08-03T05:44:48.953+00:00

    Do you have any child activity, like under Switch or IF or FOR loop activity? Probably that is also counted as an activity in a pipeline and 40 limit is crossed


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.