Azure Files
An Azure service that offers file shares in the cloud.
1,176 questions
How to dynamically rename files to data lake while copying from source (File share) with sub folder structure using ADF?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 31%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFD%3C/text%3E%3C/svg%3E)
Frank D'Souza
1
Reputation point
Hi Team,
How to dynamically rename files to data lake while copying from source (File share) with sub folder structure using ADF?
Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > File_A.csv
Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > File_D.csv
Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > File_E.csv
Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > Folder 51 > File_B.csv
Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > Folder 52 > File_C.csv
Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > File_F.csv
Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > File_I.csv
Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > File_J.csv
Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > Folder 51 > File_G.csv
Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > Folder 52 > File_H.csv
The output should have files concatenated dynamically while copying from the above structure in File share and data lake should be as below.
Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > File_A_YYYY_MM_DD.csv
Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > File_D_YYYY_MM_DD.csv
Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > File_E_YYYY_MM_DD.csv
Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > Folder 51 > File_B_YYYY_MM_DD.csv
Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > Folder 52 > File_C_YYYY_MM_DD.csv
Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > File_F_YYYY_MM_DD.csv
Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > File_I_YYYY_MM_DD.csv
Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > File_J_YYYY_MM_DD.csv
Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > Folder 51 > File_G_YYYY_MM_DD.csv
Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > Folder 52 > File_H_YYYY_MM_DD.csv
Note: YYYY_MM_DD is a current date which should be added while saving from file share to data lake.
Is that possible? Please suggest.
Regards,
Frank D
{count} votes
-
Pratik Somaiya 4,201 Reputation points2022-02-22T06:37:54.54+00:00 Hi Frank,
You can use Dynamic Content option while choosing the file name and paste the following:
@markus.bohland@hotmail.de ('Folder L01 > Folder L12 > Folder L22 > Folder L32 > Folder L42 > File_F_',formatDateTime(utcNow(), 'yyyy-MM-dd'),'.csv')
This would allow to add the Date dynamically
-
Frank D'Souza 1 Reputation point
2022-02-22T08:05:53.973+00:00 Apologies if I was not clear, I am trying to copy files in folder/ sub folder dynamically and also renaming the file as well.
-
Pratik Somaiya 4,201 Reputation points
2022-02-23T10:18:52.63+00:00 okay, maybe you can create a Metadata / Config file in your Data Lake Container wherein you need to add details like the Folder Name that you need and then use Lookup activity in ADF to get the metadata and use a ForEach to create folders dynamically and rename files by receiving input of the Lookup activity
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft 19,381 Reputation points Microsoft Employee2022-03-02T06:21:17.43+00:00 Hello @Frank D'Souza ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu -
HimanshuSinha-msft 19,381 Reputation points Microsoft Employee
2022-03-09T19:44:08.727+00:00 Hello @Frank D'Souza ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others .
If you have any question relating to the current thread, please do let us know and we will try out best to help you.
In case if you have any other question on a different issue, we request you to open a new thread .
Thanks
Himanshu
Sign in to comment