How do I delete rows with certain characters in the data flow?

Question

I want to delete this '〇' row in the image in the data flow.

Is there a better way to do this?

Answer 1

@NishimuraChinatsu-9854 ,

In addition to Subashri Vasudevan 's suggested approach, you could leverage Alter row transformation to delete the rows which are not needed.

Alter Row transformation allows us to set insert, delete, update, and upsert policies on rows.

You can add only one condition that is Delete if : col1!='〇' || col2='〇' || col3='〇' and so on.

Helpful resources:

Azure Data Factory - Use Alter Row transformation to delete NULL records in XL file

Alter Row Transformation in Mapping Data Flow in Azure Data Factory

How to update sql table after performing data validation checks using mapping dataflow

---

Hope it helps. Kindly accept the answer if it helps. Thanks.

Answer 2

Hi @NishimuraChinastu-9854,

You can use filter transformation. For instance, if your column names are col1,col2,col3, using the below expression you can filter out those rows which have the special characer in the screenshot.

col1!='〇' && col2!='〇'

like this you can filter rows which dont have the special character mentioned. please try this and let us know!

Regards

Suba

Answer 3

You can use the "Conditional Split" transformation in your data flow to separate rows with certain characters into a separate output stream and then use the "Sink" transformation to write the remaining rows to your desired destination. Alternatively, you can use the "Filter Rows" transformation to exclude rows with certain characters based on a defined condition.