This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Hi,
A dataflow mapping in Synapse writing to a delta sink should update rows based on an Alter row transformation. In the data preview of the debug session I can see that the row should be updated, however after running the pipeline the record is not being updated in the Delta lake.
This is the data preview of the sink in data flow.
Does anyone know how to solve this?
Hi @JaneTeo-1997
This preview data could be obsolete. Try changing the name of any activity before the sink and click refresh in data preview of sink, this will clear the cached data and you will see the data to be written to the sink.
Try this and let us know.
@MartinJaffer-MSFT Thanks for the tip to check the diagnostics. I had checked it before, I didn't check the source step. Turned out I still had sampling enabled.
Thank you both @Subashri Vasudevan
Thanks for letting us know the cause/resolution. I changed my comment to answer if you care to mark as accepted answer.
@JaneTeo-1997 What if you enable upsert?
Also, you can find more diagnostic information if you go to monitoring tab, find the pipeline run, go into it and get the dataflow run details like rows affected.
Hi @Subashri Vasudevan
Thanks for your reply, I tried wat you said but the preview is still showing the same thing. Just to clarify, the preview is showing the desired behavior, it's the sink that is not showing the updated data.
Any other idea how to solve this?
Hi @JaneTeo-1997,
Could you please post more screenshots of your data flow, if possible?
@Subashri Vasudevan
Overall dataflow:
Alter row conditions:
Sink setting (I've also tried the overwrite table action):
Sign in to comment