ADF v2 : Dynamics 365 Define keys for Upsert Operation (Lookup and Text)

Mihir Kadam 1 Reputation point
2021-08-06T13:56:44.24+00:00

Hi,

I would like to perform an Upsert based on two fields.

  1. Lookup
  2. Text

Example
Account Entity : Alternate Key Name (new_accountnumber) - consist of new_accountnumber field

Contact Entity : Alternate Key Name (new_contactkey)- consist of new_contactnumber(Text) and new_accountid(Lookup)

new_accountid is a lookup field to Account Entity.

I would like to perform Upsert using text and lookup field in ADF. I am not able to find a correct way to set value for the lookup.

Please find a similar solution implemented using C#
https://xrmdynamicscrm.wordpress.com/2020/09/14/cds-developer-quick-tip-set-lookup-field-using-altername-key-during-create-and-update-record-from-c/

121109-image.png

121224-image.png121175-image.png

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,624 questions
{count} votes

3 answers

Sort by: Most helpful
  1. ShaikMaheer-MSFT 38,546 Reputation points Microsoft Employee Moderator
    2021-08-18T07:09:47.137+00:00

    Hi @Mihir Kadam ,

    Below is the response from Product team.

    "The upsert can work as long as the source contains the columns which are defined by the alternate key."

    So, If your source dataset doesn't have columns which you want to be defined by the alternate key then its not possible directly from ADF. Thank you.


  2. Farooq, Omer 1 Reputation point
    2022-09-29T11:10:32.827+00:00

    I was worried about the same stuff how one should specify 2 keys (composite Primary key) in ADF when one of them is not showing in the ADF. Should i write the different logic etc.. But !!!

    You don't need to specify in case of Dataverse. ADF take care of sink table composite keys by itself.

    you might need to specify it for SQL SERVER etc.

    246063-compositeprimarykey.png

    0 comments No comments

  3. Anonymous
    2023-04-14T09:37:12.43+00:00

    The same issue here, if I designe the table in Dataverse using simple text column as alternative key, everything works as expected and upsert is working as expected within the Dataflow.

    If I do the same but I will create composite key using e.g. standard text field and a lookup Im not able to save anymore the data as datflow is throwing errors that the lookup field value is missing in the payload. The oinly workaround that works for me is to sink data to dummy json file from Dataflow and in next pipeline action use CopyActivity to upload json to Dataverse. I already opened a case within MS to get a clear message why this is happening, they are still investigating the issue.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.