I have getmetadata to get the folder metadata in a container and loop over each folder and get metadata in each folder. now since nested foreach is not allowed how can i get the childitems which are filenames inside this for loop and then do a dataflow

Question

I have getmetadata to get the folder metadata in a container and loop over each folder and get metadata in each folder. now since nested foreach is not allowed how can i get the childitems which are filenames inside this for loop and then do a dataflow

Jayanta Madhab Barman 20

I have getmetadata to get the multiple folders metadata in a container and loop over each folder and get metadata of files in each folder inside this for loop. now since nested foreach is not allowed how can i get the childitems which are filenames inside this for loop and so that i can pass foldername and filename as parameters to run a dataflow activity

PipeLine picture:

User's image

inside the for loop

User's image

J N S S Kasyap 3,700 Reputation points Microsoft External Staff Moderator

2025-06-19T11:54:30.83+00:00

@Jayanta Madhab Barman Just checking in to see if the answer provided

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know. I am happy to help on your questions

Accepted answer

0 additional answers

Your answer

J N S S Kasyap 3,700 Reputation points Microsoft External Staff Moderator

2025-06-19T11:54:30.83+00:00

@Jayanta Madhab Barman Just checking in to see if the answer provided

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know. I am happy to help on your questions

Answer 1

J N S S Kasyap 3,700 Microsoft External Staff Moderator

Hi @Jayanta Madhab Barman
You're trying to get file names inside a folder using Get Metadata inside a ForEach loop in Azure Data Factory, and attempted to use the expression:

@activity('getMetadataForEachFolder').output.childItems[??]

This error occurs because:

The user is attempting to access an index (??) inside the childItems array, but did not provide a valid index or iterable context.
In ADF, nested ForEach loops are not allowed, so you cannot directly iterate over files (childItems) inside an outer ForEach loop.
Trying to simulate this by accessing an array index with childItems[??] is invalid unless you're referring to a specific number or a defined loop iterator.

Please follow this approach:

Inside your ForEach that loops over folders, add a Get Metadata activity (getMetadataForEachFolder) to retrieve childItems (file names).
Add a Set Variable activity (array type, e.g., fileListVar) and assign:
```
   @activity('getMetadataForEachFolder').output.childItems
```
3.Pass fileListVar as a parameter to a child pipeline or a Data Flow, like:
```
           folderName = @item().name 
           fileList = @variables('fileListVar') 
```
4.Inside the child pipeline, use a ForEach loop over the fileList array to process files one by one.This bypasses the nested loop limitation by offloading the second loop into a separate pipeline.

I hope this information helps. Please do let us know if you have any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

As your feedback is valuable and can assist others in the community facing similar issues.

Thank you.

Jayanta Madhab Barman 20 Reputation points

2025-06-18T13:27:44.7733333+00:00

@Kashyap, many thanks for your answer. so this is what I did

as per you mentioned in point 2. Now how to do point 3:

3.Pass fileListVar as a parameter to a child pipeline or a Data Flow, like:

folderName = @item().name

fileList = @variables('fileListVar')

how to pass the above 2 as parameter. I have heard abou this in other articles but counls not figure it out. Also Child pipeline should also be inside the for loop ?
J N S S Kasyap 3,700 Reputation points Microsoft External Staff Moderator

2025-06-18T13:43:55.2366667+00:00
Hi @Jayanta Madhab Barman

How to pass fileListVar and folderName to a child pipeline or data flow inside ForEach:

Create parameters in your child pipeline or data flow:

folderName (String)

fileList (Array)

Inside your ForEach, after setting the variable:

Add "Execute Pipeline" (for child pipeline) or "Data Flow" activity.

Add "Execute Pipeline" (for child pipeline) or "Data Flow" activity.

In the "Settings" tab, pass parameters like:

folderName → @item().name

fileList → @variables('fileListVar')

The child pipeline execution should happen inside the same ForEach loop one per folder so each call gets the current folder’s files.

Main Pipeline ├── Get Metadata (Get all folders) ├── ForEach Folder ├── Get Metadata (Get files in folder) ├── Set Variable (store file list) └── Execute Pipeline (send folderName + fileList)

I hope this information helps. Please do let us know if you have any further queries.
Jayanta Madhab Barman 20 Reputation points

2025-06-18T14:08:32.22+00:00

Hello again, thanks for helping me out at this.

my child pipeline in red

and child pipeline is a foreach and inside is the dataflow activity.

where I should create the parameters and where I should pass it. I can see parameters option only in foreach activity. There is no parameters option in execute pipeline or set activity. In my data flow activity I have two parameters coming from dataset, one i need to pass the folder name and second I need to pass the file name. see below.

Thanks and kindly help.
J N S S Kasyap 3,700 Reputation points Microsoft External Staff Moderator

2025-06-18T15:08:26.5566667+00:00
@Jayanta Madhab Barman
In ADF, pipeline parameters are only defined at the pipeline level, not inside activities like ForEach or Set Variable. Go to the child pipeline, click the "Parameters" tab (top panel) ,this is where you add folderName and fileList.

In Child Pipeline

Open your child pipeline (the one with the red circle).

Click the Parameters tab at the top (Pipeline-level, not inside ForEach).

Add two parameters:

folderName → Type: String

fileList → Type: Array

Inside the ForEach in the Child Pipeline

Inside your child pipeline's ForEach activity (the one looping through the file list), go to the Items property and set:
@pipeline().parameters.fileList
2.Inside the ForEach, open the Data Flow activity and click on the Settings tab Pass the parameters:
@item().name @pipeline().parameters

@item().name gives you the file name from the file list.

@pipeline().parameters.folderName gives you the folder name passed from parent.

In Parent Pipeline > Execute Pipeline

In your main pipeline, where you’re calling this child pipeline inside ForEachFolder:

Select the Execute Pipeline activity.

In the Settings tab, after selecting the child pipeline, you'll see parameter fields.

Pass values like:
folderName:@item().name fileList:@variables('fileListVar')

I hope this information helps. Please do let us know if you have any further queries.
Jayanta Madhab Barman 20 Reputation points

2025-06-19T11:50:22.91+00:00

Hi Kashyap, I found it, many thanks for your time and effort

Share via

I have getmetadata to get the folder metadata in a container and loop over each folder and get metadata in each folder. now since nested foreach is not allowed how can i get the childitems which are filenames inside this for loop and then do a dataflow

0 additional answers

Your answer