I can't scan Databricks Delta lake and Notebook when using Purview.

Question

I can't scan Databricks Delta lake and Notebook when using Purview.

김기만 40

When using purview, only Databricks' Workspace and Hive metastores are scanned, but Notebooks and Delta Lake are not scanned.

What could be the cause of this problem? What should I do to solve this problem?

Accepted answer

0 additional answers

Your answer

Answer 1

Vinodh247 34,666 MVP Volunteer Moderator

Hi,

Thanks for reaching out to Microsoft Q&A.

There is a recently released 'Azure Databricks to Purview Lineage Connector' which might help in your case, link is below with all the demo and code samples.... could you check and confirm if this is helpful?

https://learn.microsoft.com/en-us/samples/microsoft/purview-adb-lineage-solution-accelerator/azure-databricks-to-purview-lineage-connector/?source=recommendations

Please Upvote and Accept as answer if the reply was helpful, this will be benefitting the other community members who go through the same issue.

김기만 40 Reputation points

2023-07-11T00:23:23.39+00:00

Thank you very much for your reply. We are currently working on that basis.

Q1. If you just scan Databricks in Purview, is it normal that only workspace and hive metastore are displayed, but notebook and delta lake are not scanned?

Q2. If Databricks' resource group and Purview's and OpenLineage's resource groups are different, is there any problem?

Q3. "(Manual Configuration) After the installation finishes, you will need to add the service principal to the data curator role in your Purview resource. Follow this documentation to Set up Authentication using Service Principal using the Application Identity you created as a prerequisite to installation." I don't understand the content, can you tell me exactly which ID of the pre-task?
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Share via

I can't scan Databricks Delta lake and Notebook when using Purview.

0 additional answers

Your answer