Data Filter Expression in ADF

Kakehi Shunya (筧 隼弥) 201 Reputation points
2022-12-19T08:45:28.35+00:00

I am trying to filter data in Azure Data Flow.
However, I do not know how to do this.
What I want to do is to extract only the records with the largest value in the "seq_no" column among those with duplicate IDs.
I just don't know what function to use to achieve this.
I await your answer.
Any answer would be appreciated.
Sorry for my bad English, I am Japanese.
Thanks for reading.
271936-image.png

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
577 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,627 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Nandan Hegde 36,151 Reputation points MVP Volunteer Moderator
    2022-12-19T12:46:33.273+00:00

    Hey
    you can use Aggregate transform in dataflow and Group by your ID column and then take the max of that column.
    Similar to partition in SQL.

    1 person found this answer helpful.

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.