This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 94%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EG%3C/text%3E%3C/svg%3E)
I’m exploring solutions to sync/mirror an external database (SQL Server/Azure SQL DB) to Azure analytics services (Microsoft Fabric, Synapse, Databricks), but my source tables do not have primary keys (PKs).
Current Challenges:
Questions:
Define primary keys?
Preferably from the existing data.
Thank you for the suggestion! Defining primary keys (PKs) is indeed the ideal approach, but in my scenario:
Seeking Workarounds:
If PKs cannot be added, what are the best Azure-native alternatives for:
Incremental syncs (not just full refresh)?
Handling updates/deletes without PKs?
I don't think there are any best alternatives. If you are going to mirror/sync data from A to B, you need a way to know where a changed row is new row, or if it is an update to an existing row. And if a row is deleted in the source, you need to be able to identify that row in the source.
If the tables have primary keys, this permits for implementation of a generic framework like the technologies you mentioned in your original post. Without PKs, it gets difficult. If you have knowledge to identify rows, you can roll your own. That is you write the SQL statements based on what you think are they keys (that would be keys in an informal meaning). It goes without saying that this is quite a big effort.
Another alternative is to make a full refresh every time. Maybe OK for a small data set, but not for terabytes of data.
So that's why I think defining primary keys is the best way to go.
Or just forget about the whole thing.