Trigger fails to pass parameters to nested activites in pipeline

Julius 40 Reputation points
2023-03-14T14:10:02.3266667+00:00

I have a main pipeline in ADF which uses parameters. If I debug that main pipeline, the pipeline works properly. If I use a second pipeline with an "Execute Pipeline" activity which exexutes the main pipeline and I manually debug that 2nd pipeline the main pipeline also works properly. Only if I use a trigger (no matter if I exceute the main or the second pipeline with that trigger, also no matter if I use "Trigger now" or and Event/time based trigger) it gives me an error which says that the "Operation on target <pipelineName> failed: Missing parameter definition for <parameterName1> ,<parameterName2>".

After further investigations (triggering the activites step by step) I found out that the problem occurs with my last activity block: My last activity block is a "for each" acitivity (top layer) which has a "if condition" activity nested in it (2nd layer) which itself has "copy" and "delete" activities nested in the "false" branch (3rd layer). The "copy" and "delete" activities in the 3rd layer also use the parameters within their datasets as well as linked services.

Again, if I debug the pipeline or manually execute an "Execute Pipeline" activity, the parameters reach the 3rd layer. But once I use a trigger, the activites in the 3rd layer are not provided with the parameters. I also tried not defining the parameters in the trigger so that the pipeline uses the default values defined in the main pipeline but this did not change the result.

How do I resolve this issue? Is this a technical issue or is this intended?

Main_Pipeline_top_layer

Main_Pipeline_2nd_layer

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,491 questions
{count} votes

Accepted answer
  1. AnnuKumari-MSFT 30,601 Reputation points Microsoft Employee
    2023-03-15T09:08:29.56+00:00

    Hi @Julius ,

    Welcome to Microsoft Q&A platform and thanks for posting your query here.

    I understand that the ADF pipeline is failing for you during the trigger run with the error: "Operation on target <pipelineName> failed: Missing parameter definition for <parameterName1> ,<parameterName2>" , however, the same is not the case while debugging the pipeline.

    If you have provided the default value to the pipeline parameter, it should be working fine . However, even after providing default values you are experiencing the issue. I suspect one of the below approaches should help here:

    1. Check the linked services used in the dataset of copy/delete activity. Probably defining the default values to the parameters of linked service User's image
    2. Similarly , provide default values to dataset parameters , in case while passing the values dynamically , it is missed in some activities.

    User's image

    1. Check the settings tab of execute pipeline activity, and if it's missing the values to be passed, kindly passed the parameter value there, either hardcoded value or pass dynamically from the pipeline parameter

    User's image


    Hope it helps. Please accept the answer if it was helpful.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful