Any correlation between PipelineId and CorelationId in ADF

Nandan Hegde 31,511 Reputation points MVP
2020-06-22T06:51:57.263+00:00

Hello ,
Is there any relationship between pipelinerunid and correlationId in ADF v2?

We have enabled LogAnalytics in ADF v2 and now we need to backtrack a pipelinerun for which I have its PipelineRunid.

So how should I filter out the log data based on pipelinerunid?

There are some posts wherein they have mentioned that corelationid and pipelineid are same :
https://stackoverflow.com/questions/54034373/adf-log-analytics-how-to-correlate-multiple-pipelines
But that is not the case.

So wanted the help on the same

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,171 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. ChiragMishra-MSFT 956 Reputation points
    2020-06-23T11:03:59.943+00:00

    Hi @Nandan Hegde ,

    PipelineID is the ID of the pipeline run.

    CorrelationId is the unique ID for tracking a particular request.

    GEEyC.png

    Ref: Monitor and Alert Data Factory by using Azure Monitor

    How should I filter out the log data based on PipelineRunId?

    When we open the logs, we query the log tables ADFPipelineRun and ADFActivityRun by column PipelineRunId/RunID.

    For example:

    DftfH.png

    For more details, you could ref: Tutorial: Get started with Log Analytics queries.

    Ref - https://stackoverflow.com/a/62526542/10653466

    Hope this helps.


  2. Prashant Rajendran 1 Reputation point
    2021-03-12T17:16:46.493+00:00

    Can we have a query that shows the parent pipeline that failed and just that exact activity that failed within the parent pipeline. Is there a way to traverse from the activity that failed all the way back to the pipeline that triggered this activity.

    0 comments No comments