Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,706 questions
Azure Data Factory -> Experiencing issues with mapping data flows -> Sink activity for update method
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 53%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVK%3C/text%3E%3C/svg%3E)
Vivek Komarla Bhaskar
911
Reputation points
When I try to use the update method on sink activity of mapping data flows, the backend table in Synapse of replicate type does not show these updates. Even with the alter row transformation before the sink activity, I can clearly see the record for the update in the data preview, but it doesn't appear in the sink activity data preview. On the sink settings, the key column is set correctly.
- --Alter Row
- --Sink
{count} vote
-
AnnuKumari-MSFT 32,906 Reputation points Microsoft Employee2023-10-30T13:51:37.1166667+00:00 Could you please confirm if both the sinks are pointing to same table in synapse DB? Could you please cross check if the other sink doesn't have any condition which is overwriting the updated data from first sink?
Could you please share the sink setting screenshot from the second sink as well? Thankyou -
Vivek Komarla Bhaskar 911 Reputation points
2023-10-31T10:48:51.6533333+00:00 Hi @AnnuKumari-MSFT Here is the screenshot as you requested and yes both the sink settings are pointed to the same table.
-
AnnuKumari-MSFT 32,906 Reputation points Microsoft Employee
2023-10-31T11:43:31.28+00:00 I suspect that sink2 is overwriting the values updated by sink1 .
Could you please change the write order for both sinks. Kindly keep the sink with 'Allow insert' as order=1 and sink with 'Allow update' as order=2 . You can define the sink order by clicking on the 'white canvas' in dataflow outside of any transformation and then go to the 'settings' and define the custom sink order.
Kindly let us know how it goes. Thankyou
-
Vivek Komarla Bhaskar 911 Reputation points
2023-10-31T11:57:25.8966667+00:00 I'm intentionally testing my data flow to see how it performs when the sink dataset has matching rows. In this case, my source contains only one record, and hence the sink setting does not matter.
In addition, the settings are correct this way -
- Should be the update
- Should be the insert.
Please refer to this video between 28:00 - 32:00 for clear reasoning behind this. https://www.youtube.com/watch?v=Xzs6OWJT3kc
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 73%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYH%3C/text%3E%3C/svg%3E)
Yeo Hwee Kit Sabrina 0 Reputation points2023-11-03T11:07:32.1833333+00:00 Hi, I am facing the same issue with the sink update - in fact, my data preview at the sink shows the row being updated correctly, but when I execute the pipeline, nothing happens in my sink. The key columns are correct. My sink order is insert before update, so there shouldn’t be any overwriting. This only happened when I changed one of my key columns from date to datetime. May I know if you managed to solve it?
Sign in to comment
Sign in to answer