How to Access "Checkpoint Key" Property in Data Flow Activity Settings in Azure Synapse Analytics and Data Factory

Question

Context I am currently using Azure Synapse Analytics to extract change feed data from CosmosDB. My setup involves:

• CosmosDB configured with Synapse Link.
• A parameterized query run against the analytical store using an inline dataset within a data flow.
• Within the Data Flow, the source options on the only source show "Change Data Capture" as selected and is a disabled checkbox.
• This data flow is part of a parameterized pipeline in Azure Synapse Analytics.

Issue: I need to override the checkpoint key in the pipeline. However, I am encountering a challenge:

• In the settings tab of the data flow activity within the pipeline, I am unable to locate the option to override the checkpoint key as indicated here.

Question: What are the prerequisites or specific conditions that trigger the visibility of the "Checkpoint Key" property in the settings tab of a data flow activity in both Azure Synapse Analytics and Azure Data Factory?

Attempts to Resolve: I have recreated the pipelines and dataflows in Data Factory in order to determine if it was an issue with my Synapse configuration with no success. The issue exists in both Synapse and ADF.

Expected Outcome: I am looking to understand how to access or enable the "Checkpoint Key" property in the data flow activity settings to override it in my pipeline.

Answer 1

To access the "Checkpoint Key" property in the settings tab of a data flow activity in both Azure Synapse Analytics and Azure Data Factory, you need to enable the "Change Data Capture" option in the mapping data flow source transformation. Once enabled, the "Checkpoint Key" property will be visible in the data flow activity settings tab. If the "Change Data Capture" option is not enabled, the "Checkpoint Key" property will not be visible.

In your case, since the "Change Data Capture" option is already enabled in the mapping data flow source transformation, the "Checkpoint Key" property should be visible in the data flow activity settings tab. If it is not visible, you may want to check if you have the latest version of Azure Synapse Analytics and Azure Data Factory.

---

References:

• Data Flow activity in Azure Data Factory and Azure Synapse Analytics
• Change data capture in Azure Data Factory and Azure Synapse Analytics

Answer 2

@Robert Hurwitz I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

Ask: What are the prerequisites or specific conditions that trigger the visibility of the "Checkpoint Key" property in the settings tab of a data flow activity in both Azure Synapse Analytics and Azure Data Factory?
Expected Outcome: I am looking to understand how to access or enable the "Checkpoint Key" property in the data flow activity settings to override it in my pipeline.

Solution: The Checkpoint Key shows only when you enable Change data capture in the source configuration of a dataflow activity. I have checked with DelimitedText as a source, and it was working fine.

However, when I selected the source as CosmosDb, by default Change data capture was enabled, but I still faced the same issue: the Checkpoint Key was not showing.

The Checkpoint Key will be displayed according to the dataset selected.

So, I figured out that I first needed to enable Change data capture by selecting DelimitedText as the input type in Source settings.

and

Then, you will get the Checkpoint Key.

After that, you can change the source data to CosmosDb and will be able to Override.

I hope this helps!

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

---

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Answer 3

Hello, it looks like this bug has not been solved yet and seems to be affecting integrations as well (not only UI), for example the Terraform AzureRM provider is not able to set an overridden checkpoint key - see issue here. Can you please look into this and fix that? Thank you.