Clarification on Column Descriptions Sync Between Databricks Unity Catalog & Microsoft Purview

Question

Hi,

We are scanning Azure Databricks Unity Catalog in Microsoft Purview as a part of Data governance initiative, and need clarification on the column-level metadata sync behavior for the following scenarios:

Setup:

Unity Catalog Column Comments → left empty/blank

• Purview Column Descriptions → manually populated by us

Queries:

1. Will Purview retain manually populated Column Descriptions, or overwrite them with the blank value from Unity Catalog in below two scenarios:
   1. Incremental Scan in Purview
   2. Full Scan in Purview
2. Future State - If we later populate Column Comments in Unity Catalog, will subsequent purview scans overwrite existing Purview Column Descriptions?
3. Is there any configuration or mechanism to prevent scan-driven overwrites on manually maintained Purview Column Descriptions?

Kind request to provide point wise clarification.

Regards,

Aamer

Answer 1

Hi @Mohammed Aamer
Good question - this comes up often when teams start governing Databricks assets through Purview.

When you scan Unity Catalog into Microsoft Purview, column descriptions are treated as source-of-truth metadata coming from Databricks. Because of this behavior:

• If the column comment in Unity Catalog is blank, Purview will not overwrite an existing manually added description during incremental or full scans. Your manually entered description will remain as long as the source continues to send no value.
• If you later add a column comment in Unity Catalog, the next scan (incremental or full) will update the Purview column description with the value from Unity Catalog, since source metadata takes precedence.
• This overwrite behavior is expected because Purview scanning is designed to synchronize technical metadata from the source system.

Currently, there is no configuration setting in Purview to permanently protect manually entered descriptions from being updated by scans. If you need to maintain governance-authored descriptions separately, a common approach is to:

• Maintain business glossary terms or custom attributes in Purview
• Use source comments in Unity Catalog as the technical metadata source

This helps avoid conflicts between manually curated metadata and scanned metadata.

Reference:

• Connect to and manage Azure Databricks in Microsoft Purview
• Learn about Microsoft Purview Data Map

I hope this helps.