Add an aggregate transform after the surrogate key to get the min() and max() key values for your filter.
Azure Data Factory Data Flow removing the first and last rows of a blob-stored file
Bakhshpour, Daniel
41
Reputation points
Hello,
I have a file stored in blob storage that has a header and a footer. I want to remove the first and last rows and do some further manipulation within data flow. What I have done is created a surrogate key and tried to use the filter modifier to remove the rows. I have success removing the first row but am having trouble removing the last row because I can't call max(key) or last(key) in the filter expression. What is the best solution for removing these rows?
Thanks,