ADF pipeline to increment the date back to six months

Question

Hi,

i have folders created on each day [20201114], so there are 12 months folders created based on date.
i want my pipeline to copy the files from each folders ; Starting from current date [20201114] till 6 months back folders data [20200514]

i have created two variables start date [@formatDateTime(AddToTime(utcnow(), -6, 'Month'), 'yyyy-MM-dd')] and end date [@formatDateTime((utcnow(), 'yyyy-MM-dd')]

now how to increment dates one by one from start to end date

Now i want the copy activity to increment 1 + day to this variable and copy the data from respective date folder

for eample: varibale output is [2020-05-14] >> adding +1 [2020-05-14 + 1] = [2020-05-15]>> copy activity should search for the folder name [2020-05-15] and process

like wise the process should continue from [2020-05-14] till [2020-11-14]

how to achieve this through ADF Pipeline

Answer 1

Hi @ranjit kurian ,

Welcome to Microsoft Q&A Platform. Thanks for posting the query.

I have used below approach using Until activity to implement the same. For testing, I used one month folders to loop and copy activity can be placed inside until loop and it can use "Date" variable in below example to access corresponding date folder which I skipped adding in below repro.

Due to limitation in using self referencing variable in "Set variable" activity, I have used index value and counter variables to add 1 day in every loop to the date. Also, attaching the JSON of the pipeline.

Please let us know if requirement does not align with implementation or for further queries and we will be glad to assist.

-----------

• Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
• Want a reminder to come back and check responses? Here is how to subscribe to a notification.