Triggering the pipeline failed due to large run size

Ryan Abbey 1,181 Reputation points
2021-09-23T00:05:06.38+00:00

A pipeline that ran yesterday, hasn't been changed since is now failing with "Triggering the pipeline failed due to large run size"... is there a limit to the number of pipelines/size of other running pipelines that can cause this? I do have another process running that is looping a large number of files, is that likely to be a cause? If so, what can be done to allow another pipeline to run at the same time?

If not, why is Data Factory so unstable that a pipeline sometimes has an issue with "Triggering the pipeline failed due to large run size"?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,030 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. ShaikMaheer-MSFT 38,486 Reputation points Microsoft Employee
    2021-09-23T08:41:54.1+00:00

    Hi @Ryan Abbey ,

    Thank you for posting query on Microsoft Q&A Platform.

    This might be because of your ADF may crossed some limitations. Below image shows few. Click here to see full list of ADF limitations.

    134622-image.png

    Please let us know if that helps or any further queries. Thank you.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.