Hello @2JK ,
Thanks for the question and using MS Q&A platform.
My understanding is that you'd like to copy only the new/updated data from multiple tables of your Oracle and copy it to the destination SQL.
From my research there is no out of box approach to do this. I shall share a high level approach that could meet your requirement.
You would need to add a column to all the tables subjected - that keeps track of the
Everytime an item is added / updated - the
last_modified gets updated with the current time.
**Step 2 **
You'd need an additional table that would track the last pipeline run for every table
Pipeline activity - You'd look up the Step 2 table - for the last pipeline run activity for each table
Filter the rows that has last modified date greater than the last pipeline run - This would be your source.
These filtered rows would be copied to the corresponding SQL table.
There is step by step detailed documentation for a similar requirement for tables in SQL DB for the above steps outlined : https://learn.microsoft.com/en-us/azure/data-factory/tutorial-incremental-copy-multiple-tables-powershell
From my research , oracle DB also don't have any inbuilt feature to track the new/updated rows.
The approach in this case as well would be similar.
Hope this Helps