question

arkiboys avatar image
0 Votes"
arkiboys asked HimanshuSinha-MSFT commented

pass multiple keycolumns to dataflow to use in join

Hello,
At present I have a dataflow which looks at the KeyColumn (Passed in dynamically) and uses that inside the Exists and join transformations...
The parameter setting example of the dataflow is as follows where the dfpKeyColumn is a string:
dfpKeyColumn-----string-------"DepartmentID"

In the dataflow, Inside a derivedcolumn, I get the value of the KeyColumn from source as follows:
JoinFieldSource toInteger(byName($dfpKeyColumn))

Then in another derivedcolumn, I get the value of the KeyColumn from target as follows
JoinFieldTarget toInteger(byName($dfpKeyColumn))

Then in Exists transformation, I do the following
doesn't exist
JoinFieldSource == JoinFieldTarget

Hope you see what I am doing...

All good so far.

Now I would like to be able to use multiple keycolumns rather than one in the dataflow parameter.
i.e. "DepartmentID, ProductID"
Questions:
1-in the parameter of the dataflow, do I use string[] rather than just string?
2- In the dataflow derived columns, to get JoinFieldSource or JoinFieldTarget, what do I do?
3- an example is appreciated.

Thank you

azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

HimanshuSinha-MSFT avatar image
0 Votes"
HimanshuSinha-MSFT answered HimanshuSinha-MSFT commented

Hello @arkiboys ,
Thanks for the ask and using Microsoft Q&A platform .

Yes you can pass the parameters in an array .

['EMPLOYEEID','LASTNAME']

![140648-image.png][2]

In the exist transform we can use something like .
140657-image.png

In the derived column , you can use something like .

140658-image.png





Please do let me know how it goes .
Thanks
Himanshu


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators


    [2]: /answers/storage/attachments/140649-image.png


image.png (78.1 KiB)
image.png (95.7 KiB)
image.png (62.8 KiB)
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks for the message, however this method will not be dynamic.
Please note that the keycolumns should be dynamic because each table that is loaded has a different keycolumn(s).
Thanks

0 Votes 0 ·

Hello @arkiboys ,
Thanks for the feedback and yes i think the above implementation should suffice that ask . If you see that the columnsd are passed as an array . I am sure I am missing out anything
here .
Thanks
Himanshu

0 Votes 0 ·