Re-Running from Failed Activity: Why does it still have to iterate through all forEach and 'skip' upstream

Question

I have a pipeline with a ForEach that iterates over the results of a LOOKUP. At the end of this pipeline I update a table. In this situation, the table update failed. I simply wanted to just re-run pipeline where I update the final table after I fixed a problem with the table. All steps before that were good and did not need to be run again.

So, I chose 'Rerun from failed Activity'

However, It seems datafactory still has to re-run through all of the steps again prior to the failed activity, and skips them. Why is it not just ignoring every activity up to the point of failure?

So, in my case, my LOOKUP has about 2K records. It skipped through these 2K records, and the finally did my update. Is this expected behavior? It doesn't seem like it should work this way.

Answer 1

Hi @Derek Horrall ,

Thankyou for using Microsoft Q&A platform and thanks for posting your question.

As I understand your issue, you are trying to rerun the pipeline from failure and the activities prior to the point of failure are getting skipped. You want to know if that is an expected behavior or not. Please let me know if my understanding is incorrect.

It is an expected behavior . Activities prior to the failure point will get skipped. Also, Inner activities of a ForEach block may still be skipped based on the rerun rules.

Kindly have a look on the below image

For more details, kindly go through : Rerun pipelines and activities

How to Rerun Pipeline from Point of Failure in Azure Data Factory

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on and take satisfaction survey button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators