the reqirement is, i have a folder having 4 txt files(1.txt,2.txt,3.txt,4.txt) . the Flow is controlled by a parameter called all or some which is of string type. i want mail notification of missing file.

Question

Happy New Year !!

continuing my requirement >>

If i select all in the parameter, all 4 file should be processed. the requirement starts here >>

IF any file is missing from the folder(for ex 2.txt and 3.txt is not present and i selected ALL in the parameter) , i need a mail saying file is 2.txt and 3.txt is missing.

If i select some in the parameter, for ex 1.txt and 4.txt and if any of the file is missing 1.txt and 4.txt is missing(for example 1.txt is missing) , i need a mail with the missing file name(i.e 1.txt in our case).

As a part of solution i was trying with a variable( with all file names ) and was trying to use intersection in the condition , but couldn't achieve it.

Answer 1

Hey, assuming you have a list of files preserved somewhere as an Array (for now lets assume a Parameter : AllFiles of type array)
follow the below flow:

1. use get meta data activity to get list of all child items (All files present in location)
2. Use Filter activity to check whether we are missing any files from file list

where item would be : @pipeline().parameters.AllFiles
Condition: @hamed1374 (contains(string(activity('<<Meta data activity>>').output.childItems),item()))

The filter activity output would be your missing file and you can leverage a logic app to send an email notification via logic app
https://blog.devgenius.io/how-to-use-azure-logic-app-to-send-email-notification-from-azure-data-factory-pipelines-c77081bf09ae

Edit :
missing files

Below is a sample JSON code for the same:
{
"name": "pipeline4",
"properties": {
"activities": [
{
"name": "Filter1",
"type": "Filter",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"items": {
"value": "@pipeline().parameters.S1",
"type": "Expression"
},
"condition": {
"value": "@hamed1374 (contains(pipeline().parameters.S2,item()))",
"type": "Expression"
}
}
}
],
"parameters": {
"S1": {
"type": "array",
"defaultValue": [
1,
2,
3
]
},
"S2": {
"type": "array",
"defaultValue": [
1,
5,
6
]
}
},
"annotations": []
}
}

Filter:
item: @pipeline().parameters.S1
Condition: @hamed1374 (contains(pipeline().parameters.S2,item()))