Loop in adf

Question

Hi Nested loop in ADF - how it work in runtime???

scenario: default it will load data in finance then Department and Region and Cities and location or so on

how we can set this default condition in for each loop and nested further based on condition...please help with screenshot

Answer 1

Try the following :

1. Finance → Department → Region → Cities → Location: The sequence is defined and should load based on a set condition.
2. Nested Loops and Conditions: Use a ForEach loop with dynamic child pipelines or activities to iterate and conditionally process nested structures.

Steps to Implement in ADF:

1. Create Pipelines for Each Level:
   • Create separate pipelines (e.g., LoadFinance, LoadDepartment, LoadRegion, etc.) to manage data loading for each level.
   • Ensure these pipelines accept parameters if they depend on dynamic inputs.
2. Parent Pipeline:
   • Set Up Variables: Create variables to hold your default order and any conditions (e.g., ["Finance", "Department", "Region", "Cities", "Location"]).
   • Use an Array type variable to define the sequence.
3. ForEach Activity:
   • Add a ForEach activity in your parent pipeline.
   • In the Items property, reference the variable holding your sequence.
   • Example: @variables('LoadSequence').
4. Set Up Nested Logic:
   • Inside the ForEach loop, use an Execute Pipeline activity to call the specific pipeline for each step.
   • Pass parameters to the pipeline dynamically based on the current iteration using expressions like:

     
          {
     
            "pipelineName": "@item()",
     
            "parameters": {
     
                "Condition": "@if(equals(item(),'Department'), 'ConditionValue', 'DefaultValue')"
     
            }
     
          }
     
     

5. Conditional Nested Loop:
   • If a step involves nested dependencies (e.g., Region depends on Department), you can add another ForEach loop inside the current iteration or conditionally execute nested pipelines based on outputs.
   • Use a Switch activity for branching within a loop based on conditions.
6. Chaining Sequential Loads:
   • If the load order must be sequential, enable sequential execution in the ForEach activity by toggling the Batch Count to 1.
   • Example: Finance must complete before Department starts.