How to assign Variable Value to Parameter

Amy Zhang 35 Reputation points
2024-11-02T02:51:24.05+00:00

Hi, we need to create a "batch_no" in the format YYYYMMDDHHMMSS within our pipeline as a human-readable job identifier , since system variables like pipeline run ID or trigger ID aren’t easily readable.

The "batch_no" should remain consistent throughout the entire lifecycle of a single pipeline run.

We also plan to pass this batch number to child data flows and use it in other steps (e.g., copy data, etc.).

We tried initializing "batch_no" using a variable expression, but we're unsure how to assign this value to a parameter.

When we directly reference the variable(batch_no) in downstream steps, its value changes, so storing it in a parameter seems more stable.

We’re currently stuck on the setup—any expert advice on achieving this would be greatly appreciated!

@concat(formatDateTime(utcnow(),'yyyMMdd_HHmmss'))

User's image

Thank you!

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
{count} votes

2 answers

Sort by: Most helpful
  1. Amira Bedhiafi 41,121 Reputation points Volunteer Moderator
    2024-11-02T13:36:35.53+00:00

    To create a consistent "batch_no" value that can be reused throughout the pipeline run, you need to define a pipeline parameter called batch_no at the pipeline level.

    Use a "Set Variable" activity or a "Set Parameter" activity at the beginning of the pipeline to generate the value for batch_no using an expression like:

    
    @concat(formatDateTime(utcnow(), 'yyyyMMddHHmmss'))
    
    

    When calling child pipelines or data flows, pass the batch_no parameter to them by mapping it in the "Execute Pipeline" activity or the parameters section of a data flow.

    Reference the batch_no parameter in subsequent activities (like copy data, conditional checks) to verify that the same value is used throughout the pipeline.


  2. Amira Bedhiafi 41,121 Reputation points Volunteer Moderator
    2024-11-02T13:38:43.09+00:00

    To create a consistent "batch_no" value that can be reused throughout the pipeline run, you need to define a pipeline parameter called batch_no at the pipeline level.

    Use a "Set Variable" activity or a "Set Parameter" activity at the beginning of the pipeline to generate the value for batch_no using an expression like:

    
    @concat(formatDateTime(utcnow(), 'yyyyMMddHHmmss'))
    
    

    When calling child pipelines or data flows, pass the batch_no parameter to them by mapping it in the "Execute Pipeline" activity or the parameters section of a data flow.

    Reference the batch_no parameter in subsequent activities (like copy data, conditional checks) to verify that the same value is used throughout the pipeline.

    0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.