ADF Pipeline How to pass a table to a Sub-Pipeline and address rows in ForEach activity

Dr Chris 0 Reputation points
2023-08-02T09:23:29.03+00:00

In a ADF Pipeline I have a filter activity (ac_filter_files_loaded) that has a table as an output:

"p_process_tab": [ {
                "PRIMARYKEY_NUMBER": "34",
                "LOAD_UUID": "85d09d44-9ff4-497a-bf03-11d07c1f2bfd",...}]

Now I want to pass this to a Sub-Pipeline in which I want to do a ForEach process.

I tried to pass this as an Object using: @activity('ac_filter_files_loaded').output.value which seems to be ok

and used @pipeline().parameters.p_process_tab.items to address the rows in the table which gives an error:

"The expression 'length(pipeline().parameters.p_process_tab.items)' cannot be evaluated because property 'items' cannot be selected. Array elements can only be selected using an integer index." also ...rows is not working.

My questions are:

  1. How to pass this table to the Sub-Pipeline (Object/Array)?
  2. How to address the rows in the table?
  3. Where can I find detail documentation of the data types used in ADF Pipeline Expression Language and related properties? (https://learn.microsoft.com/en-us/azure/data-factory/concepts-pipelines-activities?tabs=data-factory is not detailed enough at all !!!)
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,196 questions
{count} votes

1 answer

Sort by: Most helpful
  1. QuantumCache 20,261 Reputation points
    2023-08-11T17:14:48.0233333+00:00

    Hello @Dr Chris did you try the below steps/
    In the parent pipeline, add an "Execute Pipeline" activity to call the sub-pipeline**.**

    1. In the "Execute Pipeline" activity, set the "Pipeline" property to the name of the sub-pipeline.
    2. In the "Execute Pipeline" activity, set the "Parameters" property to pass the table as a parameter. For example, you can use the following expression to pass the table as a JSON string: @json(activity('ac_filter_files_loaded').output.value).
    3. In the sub-pipeline, add a "ForEach" activity to loop through the rows in the table.
    4. In the "ForEach" activity, set the "Items" property to the parameter that contains the table. For example, you can use the following expression to reference the parameter: @pipeline().parameters.p_process_tab.
    5. In the "ForEach" activity, use the "Current item" expression to reference the current row in the table. For example, you can use the following expression to reference the "PRIMARYKEY_NUMBER" column in the current row: @item().PRIMARYKEY_NUMBER.
    0 comments No comments