In Data Factory CDC (Preview), I am forced to choose an incremental column when configuring a source SQL table, even though CDC is enabled on the source table.

Question

When configuring a change data capture resource in ADF, I used as a reference this video, in which it is stated that if we have CDC enabled on a source SQL table, we don't need to specify an incremental column.

I have decided to do reproduce the same strategy used in the video, and I have enabled CDC on my source database, and on my source table, as shown in the picture below:

I have set up a database connection and proceeded to configure the CDC (preview) resource. However, unlike in the video, Data Factory Studio is still prompting me to select an incremental column.

Can someone clarify what steps are required to enable the option to use native CDC instead of an incremental column?

Answer 1

As per the documentation, when using the Data Factory CDC (Preview) :

The newly updated rows or updated files can be automatically detected and extracted by ADF mapping data flow from the source stores. When you want to get delta data from the databases, the incremental column is required to identify the changes. When you want to load new files or updated files only from a storage store, ADF mapping data flow just works through files’ last modify time.

Follow this guide to implement directly the Change data capture (CDC) with Azure SQL Database :

https://learn.microsoft.com/en-us/azure/azure-sql/database/change-data-capture-overview?view=azuresql