ADF pipeline lookup distinct rows from csv

arkiboys 8,301 Reputation points

1- lookup1 reads csv rows --> .csv may have duplicate rows
2- foreach1 is fed from lookup1
3- inside foreach1, there is dataflow1 which and copy activity which processes data

How is it possible to get only the distinct rows from .csv and then proceed?
is it possible for the lookup to get distinct rows only?


Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
7,139 questions
0 comments No comments
{count} votes

1 additional answer

Sort by: Most helpful