How to fix Synapse data flow not updating data in delta lake?

JaneTeo-1997 0 Reputation points
2023-03-17T09:05:27.8466667+00:00

Hi,

A dataflow mapping in Synapse writing to a delta sink should update rows based on an Alter row transformation. In the data preview of the debug session I can see that the row should be updated, however after running the pipeline the record is not being updated in the Delta lake.

This is the data preview of the sink in data flow.

User's image

Does anyone know how to solve this?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,329 questions
{count} votes

2 answers

Sort by: Most helpful
  1. JaneTeo-1997 0 Reputation points
    2023-03-21T06:44:51.7833333+00:00

    @MartinJaffer-MSFT Thanks for the tip to check the diagnostics. I had checked it before, I didn't check the source step. Turned out I still had sampling enabled.

    Thank you both @Suba Balaji


  2. MartinJaffer-MSFT 26,011 Reputation points
    2023-03-21T16:23:16.4866667+00:00

    @JaneTeo-1997 What if you enable upsert?

    Also, you can find more diagnostic information if you go to monitoring tab, find the pipeline run, go into it and get the dataflow run details like rows affected.

    User's image

    0 comments No comments