How to enable change data feed in path based delta table

Question

How to enable change data feed in path based delta table

Saurav Kumar Barnwal 1

Dear Tech Members,

There is an existing path based delta table ('abfss://******@xyz.dfs.core.windows.net/main/testcdc') and i want to enable the change data feed to it , i have gone through the blogs, articles and different forum and all are suggesting to run the alter command like below :

ALTER TABLE myDeltaTable SET TBLPROPERTIES (delta.enableChangeDataFeed = true)

but here myDeltaTable is not path based delta table this I think is lakehouse database table (i may be incorrect not 100% sure) also when i am trying to run the below command in python or using only sql then also i am not able to run like mentioned below

spark.sql("ALTER TABLE 'abfss://******@xyz.dfs.core.windows.net/main/testcdc' SET TBLPROPERTIES (delta.enableChangeDataFeed = true)")

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2023-11-17T02:31:58.2633333+00:00

@Saurav Kumar Barnwal We still have not heard back from you. Just wanted to check if the below information was helpful? If it answers your query, please do click Accept Answer and Yes for "was this answer helpful", as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thank you

1 answer

Your answer

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2023-11-17T02:31:58.2633333+00:00

@Saurav Kumar Barnwal We still have not heard back from you. Just wanted to check if the below information was helpful? If it answers your query, please do click Accept Answer and Yes for "was this answer helpful", as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thank you

Answer 1

Amira Bedhiafi 33,071 Volunteer Moderator

Try to initialize a Delta table object in your Spark environment usinyou already have :

from delta.tables import DeltaTable
deltaTable = DeltaTable.forPath(spark, "abfss://******@xyz.dfs.core.windows.net/main/testcdc")

Once you have the Delta table object, use the setTableProperties method to enable the Change Data Feed :

deltaTable.setTableProperties({"delta.enableChangeDataFeed": "true"})

If you prefer Spark SQL, you might need to create a temporary view pointing to your Delta table and then use the ALTER TABLE command on that view. This approach may not be supported for path-based Delta tables in all environments.

Saurav Kumar Barnwal 1 Reputation point

2023-11-10T03:05:43.5333333+00:00
@Amira Bedhiafi thank you for your help but it is throwing the error as below
Amira Bedhiafi 33,071 Reputation points Volunteer Moderator

2023-11-10T14:10:26.1933333+00:00

Does your Delta Lake version supports Change Data Feed ? Because not all versions of Delta Lake have support for Change Data Feed.

Share via

How to enable change data feed in path based delta table

1 answer

Your answer