Conditional sink in Azure data factory

Question

Hello,

I have a mapping data flow that is checking for invalid data, with conditional split, in my file. (example row in column one < than in column two.)

I the case I don't have any invalid data I would like to copy the file in new folder. if I have at least one error in my file I want to copy the file in an error folder.

The problem is that the validation is on the row level and I can't use a lookup.

Thank you for your help

Answer 1

Hi @Alexandre Munday ,

Welcome to Microsoft Q&A Platform. Thanks for posting the query.

Possible approach I can think of is by using the additional file for invalid data and later the size can be evaluated to check if there any invalid records are not and assign the folder name accordingly. The invalid data can be retrieved by using filter activity in dataflow. Below GIF and attached JSON shows the details of the approach used. Please check and let us know if this helps.

31281-dynamicfolderjson.txt

Kindly let us know if it does not align with the requirement or for further queries and we will be glad to assist.