Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
A community member has associated this post with a similar question:
Remove duplicate rows from xlsx in Azure Data Factory
Only moderators can edit this content.
Remove Duplicate Row from xlsx using azure adf
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 76%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDP%3C/text%3E%3C/svg%3E)
Dinesh Prajapati
126
Reputation points
hi team,
I want to remove duplicate rows from xlsx via azure adf. It should work like if the data of all the columns of row 1 matches with all the data of all the columns of row2, then any one of the row should be removed. I read on ms site and it says use aggregrate activity to do that, but how can I use dynamically. As it states that I have to select column in group by manually (in debug mode), which I do not want to, as there are multiple files that would be running through the same pipeline.
Following error occurs which is valid as I have off the debug mode in data flow:
Azure Data Factory
{count} votes