Why is FailureWebHook not executed in this case?

Question

Please consider the following pipeline. The Execute Pipeline activity failed, so the ProcessTMONData notebook was skipped. Why was the failure webhook not triggered, as it's supposed to trigger when EITHER:

• ForEachEndpoint fails
• ProcessTMONData is skipped
• ProcessTMONData fails

Please help me understand the control flow behavior of azure synapse.

Answer 1

Hello Psychotechnopath,

From your pipeline:

The ForEachEndpoint is connected to "failurewebhook" on failure, meaning 'failurewebhook' runs only when ForEachEndpoint fails.

Additionally, your ExecutePipeline is connected to the Notebook on success. The Notebook didn't run because the connected activity failed, and subsequently, the 'failurewebhook' didn't run because the Notebook didn't start.

Due to these two reasons, 'failurewebhook' didn't get executed.

I hope this answers your question.

Answer 2

Your notebook activity depends on the completion of the Execute Pipeline activity.

The FailureWebHook webhook is set to trigger under the following conditions:

• ForEachEndpoint fails
• ProcessTMONData is skipped
• ProcessTMONData fails

Given that the Execute Pipeline activity failed, it resulted in the ProcessTMONData notebook being skipped. Since the ProcessTMONData notebook was skipped, one would expect the FailureWebHook to trigger according to your specified conditions. This is the expected behavior.

So you may need to verify that the FailureWebHook activity is correctly dependent on the ProcessTMONData activity because it should have a dependency with a condition that it runs if ProcessTMONData is skipped.

It should be configured to trigger on the state of the ProcessTMONData notebook and you can use the expression @activity('ProcessTMONData').status to check if it is Skipped.