How to remove Change Data Capture in Azure Data Factory

Question

How to remove Change Data Capture in Azure Data Factory

Eddy Tran 40

Mar 5, 2024, 2:45 AM

I'm using Azure DevOps to deploy ADF from Dev Environment to UAT Environment.

In Dev Environment, there is a CDC in ADF. After deploying, the CDC is also appeared in UAT Environment but I encounterd some issues. Below is my problems:

When I started the CDC, the notice "Unable to Start the CDC. Error: {"_body":"{"error":{"code":"ADFDataMapperAlreadyStarted","message":"ADF CDC adfcdc1 already started." appeared User's image

This notice means I already run the CDC, but when I checked the monitor, it showed: User's image

The CDC take the process like the above picture forever and CDC hasn't worked.

I have 2 questions:

Why did CDC not work when I deployed it from Dev to UAT? Is there anything I need to check when deploying ADF with CDC?
I can't stop or delete this CDC in UAT. I also used powershell to stop it but it's not worked How can I stop or delete the CDC in UAT?

Harishga 5,995 Reputation points Microsoft External Staff

Mar 5, 2024, 7:08 AM
@Eddy Tran
Welcome to Microsoft Q&A platform and thanks for posting your question here.

I understand that you are facing issues with CDC while deploying from Dev to UAT environment, and you are unable to start or delete the CDC in UAT.

Regarding your first question, there could be several reasons why CDC is not working in UAT. Some of the common reasons are:

Incorrect configuration of the CDC pipeline in UAT environment.

Mismatch in the source and destination data stores between Dev and UAT environments.

Incorrect permissions or access to the data stores in UAT environment.

Network connectivity issues between Dev and UAT environments.

To troubleshoot the issue, you can check the following:

Verify the configuration of the CDC pipeline in UAT environment and ensure that it matches with the Dev environment.

Check the source and destination data stores in UAT environment and ensure that they are accessible and have the required permissions.

Check the network connectivity between Dev and UAT environments and ensure that there are no firewall or network issues.

Regarding your second question, to stop or delete the CDC in UAT, you can follow the below steps:

If you have already tried stopping or deleting the CDC instance using the Azure portal, PowerShell, and CLI, and it's not working, there could be a few reasons why this is happening.

Here are some additional troubleshooting steps you can try:

Check if there are any dependencies on the CDC instance that are preventing it from being stopped or deleted. For example, if there are any pipelines or activities that are using the CDC instance, you may need to stop or delete them first before you can stop or delete the CDC instance.

Check if you have the necessary permissions to stop or delete the CDC instance. You may need to have the "Contributor" or "Owner" role assigned to your Azure account to perform these actions.

Check if there are any issues with the Azure Data Factory service itself. You can check the Azure status page to see if there are any ongoing issues or outages that could be affecting your ability to stop or delete the CDC instance.

It's important to note that stopping or deleting a CDC activity can have implications on downstream processes, so it's important to carefully consider the impact before taking any action.

I hope this information helps you. Let me know if you have any further questions or concerns.
Eddy Tran 40 Reputation points

Mar 5, 2024, 7:47 AM
Dear Harishga,

Following your instruction to stop or delete CDC:

Check if there are any dependencies on the CDC instance that are preventing it from being stopped or deleted. For example, if there are any pipelines or activities that are using the CDC instance, you may need to stop or delete them first before you can stop or delete the CDC instance. => there isn't any pipeline or other things depending on CDC instance, because in my ADF, I just created only CDC instance

Check if you have the necessary permissions to stop or delete the CDC instance. You may need to have the "Contributor" or "Owner" role assigned to your Azure account to perform these actions. => I have "Owner" role

Check if there are any issues with the Azure Data Factory service itself. You can check the Azure status page to see if there are any ongoing issues or outages that could be affecting your ability to stop or delete the CDC instance. => I checked and there isn't any issues.

The CDC instance is always at the status of Started and Last processed ..

Is there anyway to force exiting this process?
Harishga 5,995 Reputation points Microsoft External Staff

Mar 5, 2024, 10:57 AM
Hi @Eddy Tran
If you have already checked for dependencies, permissions, and issues with the Azure Data Factory service, and you are still unable to stop or delete the CDC instance, there are a few additional troubleshooting steps you can try:

Check if there are any locks on the CDC instance that are preventing it from being stopped or deleted. You can check for locks by navigating to the "Locks" section of your Azure Data Factory instance in the Azure portal. If there are any locks on the CDC instance, you will need to remove them before you can stop or delete the instance.

Check if there are any issues with the Azure subscription that is associated with your Azure Data Factory instance. You can check for issues by navigating to the "Health" section of your Azure subscription in the Azure portal. If there are any issues with the subscription, you may need to resolve them before you can stop or delete the CDC instance.

If none of the above step's work, you can try creating a new Azure Data Factory instance and deploying the CDC instance to the new instance. This will allow you to start fresh and avoid any issues that may be associated with the current instance.

I hope this information helps you. Let me know if you have any further questions or concerns.
Harishga 5,995 Reputation points Microsoft External Staff

Mar 6, 2024, 10:36 AM

@Eddy Tran
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.