Some Rows are getting dropped while sinking in CSV format in Azure Data Factory using Data flows

Question

I am using csv as a row, and tranforming it in Azure Data flow in Pipeline.

while sinking it in CSV format, some rows are getting dropped, but they are present in data preview when I am adding filter condition to look that particulat ID.

Answer 1

The possible issue for the behaviour you are having

1. Data Type Mismatches
2. Null or Special Characters
3. Filter Conditions
4. Schema Mismatch

If you find that null values are causing rows to drop, you might want to replace nulls with a default value before sinking:

Derived Column Transformation:

ReplaceNull(column_name, default_value)

If the issue persists after these checks, you might want to provide more specific details or screenshots of your data flow configurations to further diagnose the problem.