Share via

Synapse: difference CorrelationId vs RunId

Benedikt Pittl 80 Reputation points
2023-10-23T10:09:59.3933333+00:00

When logging to Synapse the runs are stored in the SynapseIntegrationPipelineRuns table. There, I find a CorrelationId and a RunId. The discussion here has no clear answer.

An example looks like the following:

grafik

Can you explain, when the CorrelationId changes?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
0 comments
Accepted answer
  1. Vinodh247 34,666 Reputation points MVP Volunteer Moderator
    2023-10-23T10:29:35.7233333+00:00

    Hi Benedikt Pittl,

    Thanks for reaching out to Microsoft Q&A.

    RunId

    A RunID is a unique identifier for the particular pipeline run. Each pipeline run will have a RunID which you can use for querying the logs and finding status of the pipeline activity.

    CorrelationId

    A Correlation ID is a unique identifier that is added to the very first interaction (incoming request) to identify the context and is passed to all components that are involved in the transaction flow. Correlation ID becomes the glue that binds the transaction together and helps to draw an overall picture of events.

    Both RunId and CorrelationId are commonly used as pipeline system variables to call those details as output or added as details.

    Please 'Upvote'(Thumbs-up) and 'Accept' as answer if the reply was helpful. This will be benefitting other community members who face the same issue.

2 additional answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Amira Bedhiafi 33,631 Reputation points Volunteer Moderator
    2023-10-23T11:02:17.24+00:00

    CorrelationId is a aunique identifier to group or correlate multiple activities or logs that are part of a single logical operation or workflow. Think of it as an ID that helps you relate different actions or steps that belong to the same overarching process or task.

    RunId represents an instance of execution or a run. In the context of Synapse Analytics, it would represent a specific execution or instance of a pipeline or activity.

    A new CorrelationId would typically be generated for a new overarching process or task. If you start a new logical operation or workflow, a new CorrelationId is generated to track all the steps or actions related to that operation.

    For example, if you have recurring pipelines or jobs, each run of that pipeline/job might have the same CorrelationId if they are part of the same logical group. But each individual execution would have a unique RunId.

    Another situation is if a pipeline is rerun because of a failure or if it's scheduled to run periodically, while each run will have a distinct RunId, they might share a CorrelationId if they're considered part of the same overarching process.

    Links to help you :

    https://learn.microsoft.com/en-us/answers/questions/38502/any-correlation-between-pipelineid-and-corelationi

    https://learn.microsoft.com/en-us/azure/data-factory/monitor-schema-logs-events

    1 person found this answer helpful.
  2. Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator
    2023-10-25T20:03:07.4666667+00:00

    HelloBenedikt Pittl

    The CorrelationId is used to group related pipeline runs, activity runs, and trigger runs together, while the RunId is used to identify each individual pipeline run, activity run, or trigger run.

    The RunId does not change for the same pipeline run, activity run, or trigger run, unless they are deleted and recreated

    In your example, the RunId is identical because it is the same pipeline run.

    I hope this helps.

    https://learn.microsoft.com/en-us/azure/azure-monitor/reference/tables/synapseintegrationpipelineruns

    1 person found this answer helpful.
    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.