Synapse pipeline lineage is not pushed to Purview

Manuel Bustamante 42

Hi guys,

I have a Synapse workspace connected to my Purview:

I just ran a synapse pipeline that launch several mapping dataflows sequentially:

The last two mapping data flows fill two Synapse tables: DF_CUR_ODS_OBJET_ASSURE, DF_CUR_ODS_VEHICULE which have been catalogued in my Purview previously. For some reason the data lineage status is "Not reported"

Which I dont understand because in the documentation it says that the lineage is supported for Synapse tables and views.

My dataflow has a source parquet files and as destination a Synapse table and a delta table in the data lale. The mapping dataflow looks like:

Thank you in advance,

PRADEEPCHEEKATLA-MSFT 77,901 Reputation points Microsoft Employee

2022-06-17T09:12:18.13+00:00

Hello @Manuel Bustamante ,

Thanks for the question and using MS Q&A platform.

Are you unable to see the lineage status details for only single activity or for all the activates?

In case, if you are unable view the lineage status for all the activities, please do share the source and sink details.

Meanwhile, you may check out the official document on How to get lineage from Azure Synapse Analytics into Microsoft Purview.

1 answer

Manuel Bustamante 42 Reputation points

2022-06-17T13:06:43.343+00:00

Thank @PRADEEPCHEEKATLA-MSFT for your answer.

I'm unable to see all activities. I see nothing for the lineage.

Source (parquet file):

Destination 1 (Synapse table) :

Destination 2 (delta table) :
Please sign in to rate this answer.
PRADEEPCHEEKATLA-MSFT 77,901 Reputation points Microsoft Employee

2022-06-20T05:42:38.583+00:00

Hello @Manuel Bustamante ,

I agree that this issue looks strange and I wasn't able to reproduce this issue. If you have a support plan could you please file a support ticket for deeper investigation and do share the SR# with us?

PRADEEPCHEEKATLA-MSFT 77,901 Reputation points Microsoft Employee

2022-06-24T08:08:10.38+00:00

Hello @Manuel Bustamante ,

Did you get a chance to open a support ticket?

Manuel Bustamante 42 Reputation points

2022-06-24T08:10:56.51+00:00

Hi @PRADEEPCHEEKATLA-MSFT ,

Yes the TrackingID is #2206210050000290

Jan Clint Viernes 1 Reputation point

2022-09-23T01:48:50.063+00:00

Hi Sorry for bumping this thread. I'm having the same problem. Can you share how did you resolve this issue?

Manuel Bustamante 42 Reputation points

2022-09-23T07:16:00.747+00:00

Hi @Jan Clint Viernes ,

Right now, there are a lot of limitations concerning the lineage, several scenarios are not yet handled by Purview. In this specific case, I was using delta tables so the whole Synapse pipeline was dropped, it is not just that Purview will not get the lineage corresponding to the delta tables but Purview will not get any lineage existing in the same pipeline.
The same happen if one of the sources/destinations are resource sets in Purview, you will not get the lineage.

Kr,
Sign in to comment