Restart a pipeline from point of failure

Saha, Mithun (Cognizant) 21 Reputation points
2021-01-04T01:38:28.137+00:00

I have a pipeline with many activities in that. So the structure is something like this.

Copy activity (1)
Copy activity(2)
Copy activity(3)
Data Load execute pipeline activity (4)

1, 2,3 runs in parallel and only after 1,2,3 all finish, 4 runs to load into the table.

Now 1 and 2 are successful , but 3 has failed. If i restart from top, it will create data issue. So i dont want that. I want 3 to run and then 4. How can I achieve that with one shot ? Basically enabling the execution of rest of the pipeline in a single go.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,664 questions
0 comments No comments
{count} votes

Accepted answer
  1. Nandan Hegde 32,256 Reputation points MVP
    2021-01-04T09:40:53.733+00:00

    Hey @MITHUN SAHA ,
    There is an option of rerun from failed activity as seen below:

    53278-adf1.png

    Now when you click on Rerun from failed activity, it would skip successful runs and start from failed run:

    53154-skip.png

    Hope this answers your query.

    But ideally you should have your logic such that even if you rerun your pipeline, and even if activity 1 and 2 rerun again ; it shouldn't matter.

    2 people found this answer helpful.
    0 comments No comments

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.