Azure Synapse pipeline failure behavior

Question

Azure Synapse pipeline failure behavior

Psychotechnopath 51

Hello, I'm running into issues regarding pipeline failures. Take a look at the following inner pipeline in a for-each loop:

As you can see, the CheckAccessTokenValidity activity fails in this case, but it is handled gracefully and pipeline succesfully completes because the failure was properly anticipated and handled. However, in my outer pipeline run it still shows the pipeline as failed.

Why is this? How do I circumvent or disable this behavior, as it is unintended. I want my pipelines to show status "success" if they sucessfully executed last activity in it.

Psychotechnopath 51 Reputation points

2023-01-10T09:12:08.183+00:00

Possibly related: https://learn.microsoft.com/en-us/answers/questions/189562/azure-data-factory-pipeline-run-status-34succeeded.html
MartinJaffer-MSFT 26,236 Reputation points

2023-01-24T17:44:42.9+00:00

@Psychotechnopath Are you still facing the issue? If my earlier response solved your issue, please mark as accepted answer. Otherwise let me know how I may better assist.
Psychotechnopath 51 Reputation points

2023-01-25T17:34:09.0266667+00:00

Hi Martin thnx for your response I added a comment below your answer if you could clarify a bit more I think my issue would be solved!!
John Edwards 35 Reputation points

2024-06-05T15:35:11.6466667+00:00

Martin's answer is, frankly, absurd. If an error is handled, it's handled. It should not throw an error and you shouldn't have to code a loop-around to deal with it. The image he shows with the unconditional is confusing as hell, and not logically sound.

ADF should have an "End Gracefully" activity that allows you to handle an error condition and not show the run as an error state in your Pipeline activity listing.

1 answer

Your answer

Psychotechnopath 51 Reputation points

2023-01-10T09:12:08.183+00:00

Possibly related: https://learn.microsoft.com/en-us/answers/questions/189562/azure-data-factory-pipeline-run-status-34succeeded.html
MartinJaffer-MSFT 26,236 Reputation points

2023-01-24T17:44:42.9+00:00

@Psychotechnopath Are you still facing the issue? If my earlier response solved your issue, please mark as accepted answer. Otherwise let me know how I may better assist.
Psychotechnopath 51 Reputation points

2023-01-25T17:34:09.0266667+00:00

Hi Martin thnx for your response I added a comment below your answer if you could clarify a bit more I think my issue would be solved!!
John Edwards 35 Reputation points

2024-06-05T15:35:11.6466667+00:00

Martin's answer is, frankly, absurd. If an error is handled, it's handled. It should not throw an error and you shouldn't have to code a loop-around to deal with it. The image he shows with the unconditional is confusing as hell, and not logically sound.

ADF should have an "End Gracefully" activity that allows you to handle an error condition and not show the run as an error state in your Pipeline activity listing.

Answer 1

Hello @Psychotechnopath , Thanks for the question and using MS Q&A platform.

As I understand, you are confused on the conditions for pipeline overall success / failure. You assert the inner pipeline succeeds. Could you please go to the monitoring tab and check the pipeline runs? This does not match my experience.

In my experience, when execution reaches an activity with an on-success dependency coming out of it, and that path is not taken, then the pipeline status is failure. Example:

So, to avoid the success dependency from making the pipeline status come out as fail, I replace the success with 2 dependencies, an on-completion where the success used to be, and an on-skipped dependency from the on-failure dependency's activity. Image below.

does-not-fail-pipeline

The logic here, is that the success activity only executes when the preceding activity complete (no matter success or failure) AND the failure branch is NOT run. The failure branch does not run when the preceding activity succeeds.

Please do let me if you have any queries.

Thanks Martin

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Psychotechnopath 51 Reputation points

2023-01-11T08:20:34.6333333+00:00

Thank you Martin for the swift answer. When I say the inner pipeline succeeds, I mean that for me it is "succesful" because on the "split condition" (The CheckAccessTokenActivity), the "failure" path is taken and completes succesfully.
You say: *"In my experience, when execution reaches an activity with an on-success dependency coming out of it, and that path is not taken, then the pipeline status is failure.". ***
**
What I expected is the behavior to be as follows; when there is a split condition (e.g. a boolean choice between two paths, depending on the success/failure of an activity, as is the case in your failpipe and in my pipeline), the pipeline succeeds if it succesfully completes one of the two paths. Apparently, as you say, this is not the case and I should use anothey way to implement this logic.

Could you help me understand your proposed solution? Is it true that a pipeline gets a failed status when any on-success dependency fails? How is the on-completion dependency between activities different than the on-success? What exactly does the on-skip dependency do? And how would I implement your solution into my pipeline, which is more complex than your example? I attached a screenshot of my pipeline for your reference. Thank you very much for your time.

Share via

Azure Synapse pipeline failure behavior

1 answer

Thanks Martin

Your answer