Triggering the pipeline failed due to large run size

Ryan Abbey 1,136 Reputation points

A pipeline that ran yesterday, hasn't been changed since is now failing with "Triggering the pipeline failed due to large run size"... is there a limit to the number of pipelines/size of other running pipelines that can cause this? I do have another process running that is looping a large number of files, is that likely to be a cause? If so, what can be done to allow another pipeline to run at the same time?

If not, why is Data Factory so unstable that a pipeline sometimes has an issue with "Triggering the pipeline failed due to large run size"?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
7,173 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. ShaikMaheer-MSFT 31,721 Reputation points Microsoft Employee

    Hi @Ryan Abbey ,

    Thank you for posting query on Microsoft Q&A Platform.

    This might be because of your ADF may crossed some limitations. Below image shows few. Click here to see full list of ADF limitations.


    Please let us know if that helps or any further queries. Thank you.