Update pipeline status variable to fail if any activity fails, otherwise succeed

Question

I have a pipeline that consists of multiple activities, and I want to update the value of a variable called STATUS to Fail if any activity fails. Otherwise, I want it to be set to Success using the Pipeline expression builder. How can I accomplish this?

Answer 1

Hey,

The below blog provides a framework to get the details:

https://datasharkx.wordpress.com/2021/08/19/error-logging-and-the-art-of-avoiding-redundant-activities-in-azure-data-factory/

based on dependencies and checking the expression whether the final concatination of the error messages are empty or has some value.
And based on that you can have the status as fail or success

Answer 2

Hello @Aditya Singh

You can achieve this by using the If Condition activity and Set Variable activity in your pipeline. Here are the steps:

1. Add an If Condition activity to your pipeline and set its expression to check if the activity has failed. You can use the dynamic expression @activity('activityName').output.status to check the status of the activity. If the activity has failed, the expression will return 'Failed', otherwise it will return 'Succeeded'.
2. Inside the If Condition activity, add a Set Variable activity to set the value of the STATUS variable. If the activity has failed, set the value of the variable to 'Fail', otherwise set it to 'Success'. You can use the dynamic expression if(equals(activity('activityName').output.status, 'Failed'), 'Fail', 'Success') to set the value of the variable. Here is an example of what your pipeline might look like:

{
    "name": "examplePipeline",
    "properties": {
        "activities": [
            {
                "name": "exampleActivity",
                "type": "Copy",
                "inputs": [
                    {
                        "referenceName": "exampleInputDataset",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "exampleOutputDataset",
                        "type": "DatasetReference"
                    }
                ],
                "typeProperties": {
                    "source": {
                        "type": "BlobSource"
                    },
                    "sink": {
                        "type": "BlobSink"
                    }
                }
            },
            {
                "name": "setStatusVariable",
                "type": "SetVariable",
                "dependsOn": [
                    {
                        "activity": "exampleActivity",
                        "dependencyConditions": [
                            "Succeeded",
                            "Failed"
                        ]
                    }
                ],
                "typeProperties": {
                    "variableName": "STATUS",
                    "value": {
                        "value": "@if(equals(activity('exampleActivity').output.status, 'Failed'), 'Fail', 'Success')",
                        "type": "Expression"
                    }
                }
            }
        ],
        "variables": {
            "STATUS": {
                "type": "String",
                "defaultValue": "Success"
            }
        },
        "annotations": []
    }
}

3. In this example, the pipeline has two activities: the exampleActivity and the setStatusVariable activity. The setStatusVariable activity depends on the exampleActivity

---

I hope that this response has addressed your query and helped you overcome your challenges. If so, please mark this response as Answered. This will not only acknowledge our efforts, but also assist other community members who may be looking for similar solutions.