Synapse Pipeline (Lineage) ingestion into Purview - Failed to report lineage to catalog

Hargopal Singh 20 Reputation points
2024-07-08T20:11:16.2266667+00:00

Hi Team,

The copy operation is successfully executed, but the lineage is not getting generated. When I click on the "Lineage Status" link, I get an error saying, "The catalog provided for lineage reporting cannot be connected, please check the catalog uri and network connection."

I verified all the permissions, i.e. "Storage Blob Data Reader" in Storage account, "Reader" access in Synapse and executed the below commands:

CREATE USER [purview-pdmp] FROM EXTERNAL PROVIDER

GO

EXEC sp_addrolemember 'db_datareader', [purview-pdmp]

GO

and can review the result:-

SELECT p.name AS UserName, r.name AS RoleName

FROM sys.database_principals p

LEFT JOIN sys.database_role_members rm ON p.principal_id = rm.member_principal_id

LEFT JOIN sys.database_principals r ON rm.role_principal_id = r.principal_id

WHERE p.authentication_type_desc = 'EXTERNAL'

ORDER BY p.name;

Also, added my Purview account in Synapse -> Manage -> MS Purview -- Connected.

Scanning is also successful from the Purview portal for the Synapse and Storage account as Data sources.

Can you please help in letting me know what could be the reason for failing to generate lineage ?

In addition to this, there is no issue in the QA environment. We migrated our code (pipelines, tables, etc.) from QA to prod. I can view the other pipeline lineages in Purview from the same Synapse Prod instance, but Purview specific pipelines are not working as expected.

Regards,

Hargopal

Microsoft Purview
Microsoft Purview
A Microsoft data governance service that helps manage and govern on-premises, multicloud, and software-as-a-service data. Previously known as Azure Purview.
1,236 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.