ADF Lineage Data in Purview Errors

Question

We are currently encountering ADF lineage sync issues, below is the error message:

Cannot connect - the error happens on pipeline where the Copy Activity from SQL to Storage Account fails (see error below).

Copy Activity uses an Integration runtime on a VM, which enables it to connect to the on-premise SQL server instance. Other pipelines with Copy Activities are functioning well, albeit using the built in integration runtime.

Any ideas to resolve this would be appreciated.

Answer 1

@Mark Gerald - Thanks for the question and using MS Q&A platform.

When Purview needs to know ADF lineage, Purview should be able to communicate with ADF effectively even though ADF is having Managed Private endpoint or when the Customer's SHIR is in a secured Azure VNet.

Why this issue occurs?

• This error might occur due to network configuration or wrong DNS configuration whenever SHIR is used.
• Yours Purview account might be using Private endpoints, but you are using Azure Integration Run time (Public) while running activities in ADF.

To resolve the issue, you may follow the below steps:

• If you are using Managed Vnet IR in ADF, follow these steps Managed Private Endpoints for Purview to create Managed Private endpoints in ADF's Managed VNet.
• If you are using SHIR in ADF, follow these steps Private Endpoints for Purview to create Private endpoints in Customer's SHIR VNet.
• Some more information on available Deployment options.

For more details, refer to How to access secured Purview account from ADF

Once these steps are followed, you should be able to see the lineage information.

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.