Share via

Transform Dataverse Data to Delta via Azure Synapse Analytics

Artem Kuznetsov 0 Reputation points
2023-04-24T06:42:53.6+00:00

I'm using Synapse Link to transfer data from Dataverse to Storage and following way to transform Dataverse Data to Delta: https://learn.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-parquet But when I do following steps I face with issue:

  1. Update row in Dataverse table
  2. Execute pipeline to transform Dataverse Data to Delta Expected result: Updated row catches by pipeline Actual result: Pipeline doesn't see updated row I thought about delay between Dataverse and Storage, but I'm pretty sure that files are updated. I don't understand why Synapse Analytics orchestrator doesn't catch it.

1

enter image description here

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,196 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. AnnuKumari-MSFT 32,161 Reputation points Microsoft Employee
    2023-04-26T10:14:11.1066667+00:00

    Hi Artem Kuznetsov , Welcome to Microsoft Q&A platform and thanks for posting your question here.

    As I understand your issue, you are trying to copy data from Dataverse to Delta sink. However, you are getting unexpected results where the column values are not getting updated in the sink.

    Kindly make sure in the sink settings, you have unchecked the 'Allow insert' checkbox and have selected either 'Allow update' or 'Allow upsert' option. You can set 'update if' policies using Alter row transformation .

    For more details, kindly check the below resources: Alter row transformation in mapping data flow How to update sql table after performing data validation checks using mapping dataflow Alter row transformation Hope it helps. Kindly accept the answer if it's helpful. If it doesn't help resolve your issue, kindly revert back with your observations. Thankyou