question

JasonW-5564 avatar image
0 Votes"
JasonW-5564 asked SathyamoorthyVijayakumar-MSFT commented

How to implement logic if one of any five different boolean variable are true...

I have a daily ELT process that picks up some file from blob storage. This process has five required files that need to be present for it to proceed. This is the part of the logic I am trying to implement.

I already have the logic that checks to see if 5 required files are present or not and sets a boolean type value for each if they are missing (example: File1_is_missing = true if the file is not present in BLOB storage). I want to send an email and fail my pipeline if any of the five boolean variables are true. How can implement this? All I can think of is some kind of big nasty nested IF activity and I am not even sure how to do it...

NOTE: I want to evaluate all five variables before sending the email because I only want to send one email and have logic in the body of that email that lists any of the missing files. Hope that makes sense!!! Thanks for any help/ideas!

azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

SathyamoorthyVijayakumar-MSFT avatar image
0 Votes"
SathyamoorthyVijayakumar-MSFT answered SathyamoorthyVijayakumar-MSFT commented

Hello @JasonW-5564,

Thanks for the question and using MS Q&A platform.
I see you already have 5 Boolean variables.

So, you could do the below :

From the output where you set the variable to true (For instance, File1_is_missing is true)- You could have an IF condition

This IF condition has one activity which adds/appends 'Filename' to an Array Variable - when value is true - using the Append to Variable activity - Let's assume this to be MissingFilesList. You need to declare this as an array.

137012-image.png

Refer the AppendFile1,AppendFile2 in the below screenshot.

136950-image.png

On every true, it will add the filename to the variable MissingFilesList

Now, there is another IF (If Condition 3) which checks the length of MissingFilesList variable, if it is 0 - it means that there are no files missing. Else, there is a missing a file. You now have the list of missing files in the variable MissingFilesList

 @equals(length(variables('MissingFiles2')),0)

If you want to proceed you can define the activities in the true section of this condition.

Alternative

The other approach I thought of is below

136926-image.png

For every non existing File, the filename is appended to an array variable - MissingFiles.
The below is the expression for AppendMissingFile1.

 @if(activity('GetFile1').output.exists,'','File1')

You will repeat the process for all the five files.

Note: You will be appending empty strings ''when doing the above action to the array.

So, you will have to eliminate the same. That is being taken care the in the GettheMissingFiles (Set the Variable activity) - MissingFileswithoutBlank

 @intersection(createArray('File1','File2'),variables('MissingFiles'))

The output of this action will contain the name of all missing files in an Array variable (MissingFileswithoutBlank)
Like the above explained step, you can get the length of the collection (MissingFileswithoutBlank) - if it is 0 - you can proceed with the necessary activities, if not you can do the failure activities in the false activities.

On Failure - when files are missing

You can execute the below activity false section of last the IF condition (If condition 3 & 1 ) in above screenshots.

For email - You could make use of logic apps to trigger an email. The list of files is still in the array variable which can be passed as request body which triggers the logic app for further processing - like including in the email body. Came across this step-by-step article that may help.

You could gracefully exit the pipeline - if you want to fail the pipeline explicitly you could use one of the methods referenced in this thread

136927-image.png

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators



image.png (11.9 KiB)
image.png (38.6 KiB)
image.png (33.5 KiB)
image.png (24.9 KiB)
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @JasonW-5564,

Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·

Hello @JasonW-5564,

Following up to see if the above suggestion was helpful. And, if you have any further query do let us know.

0 Votes 0 ·