Unable To Use Expression In Window Transformation Over Clause

Paul Good 26 Reputation points
2022-04-21T09:40:10.443+00:00

Hi,

I would like to specify a data flow parameter as the column that is used for the Over clause in a Window transformation but it is failing validation with the message "Over column PKColumn must reference at least one column" as below. I am able to reference the same parameter in other places in the data flow e.g., a sink's key columns. Please let me know if there is a solution to this?

Thanks, Paul

195123-screenshot-2022-04-21-at-103737.png
195131-screenshot-2022-04-21-at-102628.png
195122-screenshot-2022-04-21-at-102637.png

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,117 questions
0 comments No comments
{count} votes

Accepted answer
  1. AnnuKumari-MSFT 32,011 Reputation points Microsoft Employee
    2022-04-22T07:06:55.59+00:00

    Hi @Paul Good ,

    Welcome to Microsoft Q&A platform and thanks for posting your query.

    As I understand your query, you are trying to use Window transformation in data flow and making use of parameter to reference the source column but it's throwing an error saying "Over column PKColumn must reference at least one column" . Please let me know if it's not the case.

    First of all, while referencing any column please try to use byName function which is used to select a column value by Name and do a typecasting to string if needed.

    Coming to your query, I was able to reproduce your scenario and faced the same error when I used over clause of windows transformation and tried referencing a column using parameterization . However, the same approach was successful in other transformation like Rank,Aggregate , etc. Even the other tabs apart from Over in window transformation i.e. Sort, and window are working using parameterized column value.

    195438-image.png

    It looks like a product bug to me. I am taking this ahead with the product team and will keep you posted after I hear any response from them.

    Meanwhile, I would recommend you to make use of aggregate function to achieve the same requirement which is meant for performing aggregations of columns similar to Windows transformation.

    195380-image.png

    195449-image.png

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
      Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

0 additional answers

Sort by: Most helpful