Azure SQL Database
An Azure relational database service.
5,239 questions
ADF Data Flow - CDC - Schema Drift not working
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 83%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Ashwani SRIVASTAVA
60
Reputation points
Hi, I have setup ADF pipeline for data sync of tables between Azure SQL Database to SQL Managed Instance databases. I am using Data Flow and CDC to track changes and sink activity to replicate it to target. Since there are multiple tables involved, I am using dynamic mapping in pipeline by reading table information from a settings table and then pass it to data flow and process for each table. "Schema Drift and Infer drifted column" options have been used as per below screenshot at both source and sink however when adding or modifying any column in source table, it does not apply new column or updated column to the target. Has anyone experienced such issue?
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu 10,885 Reputation points Microsoft Vendor2024-05-08T08:35:00.6266667+00:00 @Ashwani SRIVASTAVA Thank you for bringing this to our attention. We are currently investigating the issue internally and will get back to you as soon as possible with an update. We appreciate your patience and understanding in the meantime.
-
Smaran Thoomu 10,885 Reputation points Microsoft Vendor
2024-05-13T09:18:49.89+00:00 @Ashwani SRIVASTAVA Thank you for reaching out. Could you please check and try rerunning again to see if you are still experiencing the issue?
-
Smaran Thoomu 10,885 Reputation points Microsoft Vendor
2024-05-14T05:57:18.8633333+00:00 @Ashwani SRIVASTAVA We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
-
Ashwani SRIVASTAVA 60 Reputation points
2024-05-15T22:31:00.2966667+00:00 hi @Smaran Thoomu - I have tested below things again however it is not working:
1 - Disabled and enabled CDC on source table, Changed checkpoint key in pipeline's data flow activity. Ran pipeline and could see records flowing in.
2 - Add a new column in source table, ran the pipeline. New column did not replicate to the target.
3 - Manually created the same column in target and inserted a new record in source table with value for the new column. Ran the pipeline, but new column value is not getting replicated.I am not sure how schedule drift works with CDC enabled table because when I look into "cdc.MyTableName_CT" table, I could not see entry of newly added column.
-
Smaran Thoomu 10,885 Reputation points Microsoft Vendor
2024-05-16T12:14:34.3966667+00:00 @Ashwani SRIVASTAVA Apologies for that, let me check again with my internal team.
Sign in to comment