Azure Data Factory - Execute Pipeline Not Passing Parameters

Question

he purpose of the pipelines I'll outline below is to retrieve a list of inbound and outbound folder locations, and then move files from inbound to outbound based on a file naming pattern. We do this in SSIS with no issues.

I have a parent pipeline that retrieves a list of records from Snowflake, then passes that list to a ForEach loop. Inside the ForEach loop, I have an Execute Pipeline activity; the child pipeline called here is responsible for filtering the records and then using its own ForEach loop to go through the inbound location for the current record, grab the files matching the naming pattern, and then move on to the next record. Simple enough, right?

The parent pipeline has a lookup activity and a foreach loop. The ForEach loop has the Items property set as @activity('lookup').output.value. The Execute Pipeline activity inside the ForEach loop has 3 parameters defined and set thusly:

folderPath:

The child pipeline has the same 3 named parameters, all spelled the same, and all as string.

Anyhow, when I debug/execute the parent pipeline, I can see that the input for the Execute Pipeline activity matches what is expected - values populated based upon the results of the query in the lookup activity. However, a check of the output shows that pipelineRunId and pipelineName are being sent, and not the parameters/values I've defined.

Anybody know why this is? I cannot find an answer in Microsoft's documentation, and Google is of no help. I've tried assigning the parameters without the open/close parens, but there is no change in the output. Why would the Execute Pipeline activity send the default system values but not the parameters? Is there some flag I forgot to set, or some line of json I can write to force this to happen?

Answer 1

Hi ,

Thanks for reaching out to Microsoft Q&A.

The issue of parameters not being passed correctly in ADF's execute pipeline activity can occur due to improper parameter mapping or incorrect configuration.

Here's how you can address and resolve it:

Steps to Ensure Parameters are Passed Correctly:

Define Parameters in the Invoked Pipeline:

• Open the target (invoked) pipeline and make sure that parameters are defined in the "Parameters" tab.
  • Each parameter you want to pass from the parent pipeline must exist here.

1. Configure Parameters in the Execute Pipeline Activity:

• In the parent pipeline, locate the "Execute Pipeline" activity.
• Open the activity's settings and scroll to the "Parameters" section.
  • Map the parent pipeline's values (ex: variables or expressions) to the parameters in the invoked pipeline.
  • Example: "parameters": {

    *"TargetParam": "@pipeline().parameters.SourceParam"*
    

    }

Validate Expressions and Debug:

• Ensure the expressions used to pass parameter values are valid. For instance, use @pipeline().parameters.<ParameterName> to reference parent pipeline parameters.
• Debug the parent pipeline to see if parameters are being passed correctly. You can monitor the inputs passed to the Execute Pipeline activity in the Output section of the debug window.

Use Default Values (Optional):

• If a parameter is optional in the invoked pipeline, you can set a default value in its configuration. This prevents issues if no value is passed from the parent pipeline.

Inspect Activity Run Logs:

• After executing the parent pipeline, check the "Activity Runs" under the Monitoring tab in the ADF UI. This can help identify issues with parameter transmission.

Please feel free to click the 'Upvote' (Thumbs-up) button and 'Accept as Answer'. This helps the community by allowing others with similar queries to easily find the solution.

Answer 2

I figured out that the pipeline was passing parameters. When checking the output, i was seeing the pipeline I'd. Clicking that brought me to the child pipeline where I could see the parameters. Apologies for this; I'm new at ADF and trying my best!