How to stop ForEach activity if previous itertion fails in ADF/Synapse pipeline

asked 2022-06-13T19:20:26.183+00:00
Heta Desai 211 Reputation points

I am trying to pull multiple tables from a Azure SQL Database . When data is loaded to its destination I am calling an SP that will update the status of a table to Success. Using ForEach activity multiple tables are puling. But, While pulling one table if any activity in ForEach fails than after executing fail path the ForEach loop should exit. 210995-pipeline.jpg

As shown in the attached image if copy activity fails than after executing its Fail path which includes activities such as SetPipelineObjectStatus, AuditPipeline and SetPipelineObjectStatusToDefault, ForEach loop should stop means new Iteration should not be started and Pipeline should be failed.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,261 questions
1 vote

2 answers

Sort by: Most helpful
  1. answered 2022-06-13T19:50:10.413+00:00
    Nasreen Akter 10,601 Reputation points

    Hi @Heta Desai ,

    I think Fail activity might be able to help you here. :)

    210964-image.png

    No comments

  2. answered 2022-06-14T04:03:24.87+00:00
    Nandan Hegde 21,606 Reputation points

    Hey,
    @Nasreen Akter , based on my understanding the Fail activity would fail for that iteration but the FOR each activity would take it to the net iteration.

    @Heta Desai : for you to stop the iterations and pipeline if anything fails within the iteration, you would need to use web activity to disable the ADF pipeline execution via REST API.
    Below link is somewhat helpful w.r.t disabling the trigger, you can follow the similar approach to disable the execution:
    https://learn.microsoft.com/en-us/answers/questions/160681/how-to-automatically-stop-trigger-in-azure-data-fa.html

    Note: This is crude way to exit the loop as it would ensure that your pipeline execution is terminated and no other activities post that loop would be executed even if you wanted it to execute