ADF : How to pass child pipeline failure status to master pipeline

Question

ADF : How to pass child pipeline failure status to master pipeline

Vindhya Sree Dharanikota 25

Hi,
My parent pipeline is calling multiple child pipelines designed to run sequentially, like child1 will call child 2 on completion not necessarily successful so on so forth , so whenever one of the child execution is failed am expecting parent sequence status should be failed which is not the case always, parent sequence is sometimes getting a success status and sometime a failed status. Expectation : Assuming 3 child pipelines, case 1 : When Cild1 and Child 2 and Child3 pipelines are successful parent pipeline should be successful case2 : when Child1 or Child2 or Child 3 are failed Parent pipeline should be failed. Pl suggest if there is any possible setting in pipeline without adding a set variable and tracking status of each child pipeline to handle the situation as I do have multiple sequences where this fix is required with each parent pipeline having 10+ child pipelines. PFB screenshot : Note : All child pipelines are linked on completion as no dependency across pipelines but need to be scheduled to run one after other to balance the load on the cluster User's image

Thanks,
Vindhya

AnnuKumari-MSFT 34,556 Reputation points Microsoft Employee Moderator

2023-04-25T10:37:54.7966667+00:00

Hi Vindhya Sree Dharanikota , Just checking in to see if the below answer helped. Please do consider clicking Accept Answer as accepted answers help community as well. Also, please click on Yes for the survey 'Was the answer helpful'
Vindhya Sree Dharanikota 25 Reputation points

2023-04-25T14:55:30.9233333+00:00

Hi @AnnuKumari-MSFT , Thanks for your response. Yes you understood the issue correctly, My design falls under first category where child pipeline does not have error handling but still master pipeline is not failing upon child pipeline failure pl let me know if you need further details to understand the issue.

Accepted answer

0 additional answers

Your answer

AnnuKumari-MSFT 34,556 Reputation points Microsoft Employee Moderator

2023-04-25T10:37:54.7966667+00:00

Hi Vindhya Sree Dharanikota , Just checking in to see if the below answer helped. Please do consider clicking Accept Answer as accepted answers help community as well. Also, please click on Yes for the survey 'Was the answer helpful'
Vindhya Sree Dharanikota 25 Reputation points

2023-04-25T14:55:30.9233333+00:00

Hi @AnnuKumari-MSFT , Thanks for your response. Yes you understood the issue correctly, My design falls under first category where child pipeline does not have error handling but still master pipeline is not failing upon child pipeline failure pl let me know if you need further details to understand the issue.

Answer 1

AnnuKumari-MSFT 34,556 Microsoft Employee Moderator

Hi Vindhya Sree Dharanikota , Welcome to Microsoft Q&A platform and thanks for posting your questions here. As I understand your requirement is that the master pipeline should fail in case the activity in any of the child pipelines is failing , however, it's not happening for you. Please let me know if that is not the correct understanding.

Kindly make sure that there is no error handling done in the child pipeline. If any activity is failing and there is an 'On Failure' conditional path attached to other activity , then it will render the pipeline to succeed. To avoid that, kindly ensure there is no try catch mechanism in the child pipelines.

Child pipeline would fail leading to master pipeline failure as well if there is no activity attached to on failure case:
Child pipeline would succeed leading to master pipeline success as well if there is an activity attached to on failure case:

For more details kindly check Common error handling mechanism Hope it helps. Kindly accept the answer if it's helpful. Thankyou

Vindhya Sree Dharanikota 25 Reputation points

2023-04-25T14:30:30.17+00:00

Hi @AnnuKumari-MSFT , Thanks for your response. Yes you understood the issue correctly, My design falls under first category where child pipeline does not have error handling but still master pipeline is not failing upon child pipeline failure pl let me know if you need further details to understand the issue.
AnnuKumari-MSFT 34,556 Reputation points Microsoft Employee Moderator

2023-04-26T06:56:33.0466667+00:00

Vindhya Sree Dharanikota , That is strange. I would recommend you to create a support ticket with Microsoft so that our product team can directly engage with you and provide solution after deeper investigation. Here is how you can do that: Create an Azure support request If you have a support plan you may file a support ticket to get to the root of this , else could you please send an email to azcommunity@microsoft.com with the below details, so that we can create a one-time-free support ticket for you to work closely on this matter. Subscription ID: Subject : Attn Annu
Vindhya Sree Dharanikota 25 Reputation points

2023-04-27T16:19:41.03+00:00

Sure @AnnuKumari-MSFT , but I would like to highlight that I have connected child pipelines with a condition of 'On Completion' as mentioned in my initial post(unlike your example where pipelines are connected as 'on Success'), so can you pl confirm child pipelines connected like this should pass the status as failure upon failure of any child pipelines ?
AnnuKumari-MSFT 34,556 Reputation points Microsoft Employee Moderator

2023-05-08T06:21:34.2666667+00:00

Hi Vindhya Sree Dharanikota ,

Thanks for highlighting the above point. So, the reason master pipeline status is showing as success is because even if the execute pipeline activity running the child pipelines are failing, the next activity is getting executed as they are connected via 'on completion' link . The pipeline status depends on the last activity status. So , in case the last activity succeeds, the pipeline will show as success else it will show as a failure. It's the same logic as stated above for handling try and catch scenario. Thanks . Kindly accept the answer if it's helpful.
Vindhya Sree Dharanikota 25 Reputation points

2023-05-10T15:33:31.2333333+00:00

Thanks @AnnuKumari-MSFT

Share via

ADF : How to pass child pipeline failure status to master pipeline

0 additional answers

Your answer