Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Azure Data Factory Api to get Lineage Path of flow between activities inside Pipeline
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 73%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vegnesa, Savio
0
Reputation points
Hi team,
Is there an API available within Azure Data Factory that can provide the Lineage Path of an Activity's source and target inside pipeline ?
To illustrate, I would like to understand the Lineage Path activities:
for example :
source : GetMetadata To target: Copy Data
Azure Data Factory
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 8%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anonymous2023-10-17T09:17:31.6433333+00:00 Thanks for using Microsoft Q&A platform.
Yes, there is an API available within Azure Data Factory that can provide the lineage path of an activity's source and target inside a pipeline.
To retrieve the lineage information for a specific activity, you can use the Get Activity Run API. This API returns information about the activity run, including the activity's inputs and outputs. You can use this information to determine the lineage path between two activities.
Here's an example of how to use the Get Activity Run API to retrieve the lineage path between two activities:
- First, you need to retrieve the activity run ID for the source activity. You can use the List Activity Runs API to retrieve the activity run ID for the source activity. Here's an example of how to use this API:
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelines/{pipelineName}/activities/{activityName}/runs?api-version=2018-06-01- Once you have the activity run ID for the source activity, you can use the Get Activity Run API to retrieve the inputs and outputs for the activity run. Here's an example of how to use this API:
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelines/{pipelineName}/activities/{activityName}/runs/{runId}?api-version=2018-06-01- Repeat steps 1 and 2 for the target activity.
- Once you have the inputs and outputs for both the source and target activities, you can determine the lineage path between the two activities.
For more information about the Azure Data Factory API, you can refer to the following documentation:
I hope this information helps you. please do let us know if you have any further queries.
-
Anonymous
2023-10-18T10:03:41.84+00:00 We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
-
Anonymous
2023-10-19T09:17:01.8933333+00:00 @Vegnesa, Savio just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Sign in to comment
Sign in to answer