How to get column name from a validation step

Question

hello

I'm executing a pipeline in order to validata an input dataset, the validatio is simple, some fields from the file should always come populated in case to be empty or null I need to skip that records.
However I need to generate a kind of error file with the records that didn't achieve the validation.
from every record I need to know which columns are in error, if's possible to gather the column name?

per example : my input file is a csv file delimited by semicolon with a header
the mandatory fields are : field01&field02

Field01;Field02;Field03;Field04
A;B;C;D
;E;F;G
H;;J;K
;;M;N

error file that i Want to have will follow the same structured as the inpus plus a comment with the missing fields
;E;F;G;Record discarded Field01 not populated
H;;J;K;Record discarded Field02 not populated
;;M;N;Record discarded Field 01 not populated Field02 not populated

please can you help me?i don't know how to do it

Answer 1

@CORONADO GRANADOS Diana Milena Thanks for using Microsoft Q&A !
You can try to achieve this with the Data Flow by using Conditional Split to split the rows and distribute the data into groups to filter out the null columns ( like below)-

You can then send not null columns to the destination and union Null columns with a derived column to a separate sink.
Derived Column example -