question

VigneshMurali-6205 avatar image
0 Votes"
VigneshMurali-6205 asked ShaikMaheer-MSFT commented

How can I append the name of all files in Folders and Sub-Folders inside the same Variable in Azure Data factory?

I have a requirement in Azure Data Factory where there are files in Folders and Sub-Folders. I have highlighted the structure of the same below for reference.
198474-image.png


I want to Understand how I can use an Array Variable and append all the filenames inside the variable and set the array as the variable's Value.

azure-data-factory
image.png (8.3 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @VigneshMurali-6205 ,

Just checking in to see if the below answer helped. If this answers your query, do click 130616-image.png and upvote 130671-image.png for the same. And, if you have any further query do let us know.

0 Votes 0 ·

1 Answer

NandanHegde-7720 avatar image
0 Votes"
NandanHegde-7720 answered NandanHegde-7720 commented

Hey,
You can use Get metedata activity to get the list of all files within a folder.
In case of sub folders, reiterate over the new path.
Then iterate it over via For activity and use Append variable activity to append the file names in an Array variable

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi,

Thanks for your Answer but it does not work actually. I have already tried to use the getMetadata Activity. The issue is getMetadata Activity will only retrieve the files and Folders which are direct children of the current Folder or Container mentioned in the Dataset of getMetadata Activity and does not traverse recursively and fetch all files at 2nd ,3rd and 4th Levels.

For Instance, In the above mentioned case getMetadata Activity will only retrieve Files1-4.csv and Sub-folder 1&2. I want to traverse down Sub-folders 1&2 and get the files present in there too.

Thanks again for your efforts to answer the question. Let me know if you have a working solution

1 Vote 1 ·