Azure Data Flow Not reading Entire Csv File

Rajput, Ravi 5 Reputation points
2024-01-18T17:59:44.4366667+00:00

Hi ,
I have azure data flow that reads the csv file with 15M rows.
This data flow does lot of transformation like joining, select, filter etc.
it was working fine until last month, however now it is not reading the entire file. Out of 15M rows it is reading only 10M rows from .csv file.
I checked in the monitor, only 10M rows were read by source activity. Original Data Flow run ID : 9f62d4db-762c-4137-b321-e837ae562f10

I created a test data flow by removing all the other activities except the source and sink and tried running it, this time the data flow reading the entire file.

Test Data Flow run Id : 0b6d3743-949f-450f-842f-d29206b0cc41

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,095 questions
{count} vote