Dataflow does not refresh column information in Projection

Patil Rajas 40 Reputation points
2024-03-07T13:58:56.6166667+00:00

Hello,

I am trying to make dataflow to read data from source and merging into sink table.
User's image

I am trying to achieve it dynamically by passing information in variables.

3 objects used in dataflow are reading information like dataset table name, query, Alter row conditions and key column in sink .

But when I run this for 2 different tables, the columns in projection tabs not getting refreshed. which causing issue in fetching information dynamically.

If there any way we can do it dynamically or its not possible as of now ?

Regards,PR

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,489 questions
{count} votes

Accepted answer
  1. BhargavaGunnam-MSFT 25,881 Reputation points Microsoft Employee
    2024-03-11T20:04:18.0633333+00:00

    Hello Patil Rajas,

    I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

    Issue:

    • Dataflow does not refresh column information in Projection

    Solution:

    The issue was due to import projection on the sink and source, which have column structures stored within them.

    Re-created dataflow as below and it resolved the issue. User's image

    If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

    I hope this helps!

    If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

    Please remember to "Accept Answer" if any answer/reply helped, so that others in the community facing similar issues can easily find the solution.

    0 comments No comments

1 additional answer

Sort by: Most helpful
  1. Patil Rajas 40 Reputation points
    2024-03-11T10:23:26.19+00:00

    Hello,

    Thank you BhargavaGunnam-MSFT for your suggestion.

    I realized that my sink and source have column structured stored in it due to click on Import projection.

    I re-create dataflow as below:
    User's image

    And now its working fine

    Hope this will be helpful.

    Regards,

    PR

    0 comments No comments