How to copy file based on date in Azure Data Factory

Sonhit 1 Reputation point
2022-11-20T12:15:46.473+00:00

I have a list of files in a adls container which contain date in the name as given below:
TestFile-Name-20221120. csv
TestFile-Name-20221119. csv
TestFile-Name-20221118. csv
and i want to copy files which contain today date only like TestFile-Name-20221120. csv on today and so on.
I've used get metedata activity to get list of files and then for each to iterate over each file and then used set variable to extract name from the file like 20221120 but not sure how to proceed further.

Kindly help anyone as I get stuck on this.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,605 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Samy Abdul 3,366 Reputation points
    2022-11-20T14:41:41.133+00:00
  2. Nandan Hegde 29,896 Reputation points MVP
    2022-11-21T03:11:42.503+00:00

    Hey,
    Assuming all the file names start with TestFile-Name-,
    and you want to copy the data of file with todays date,
    use get metadata activity to check if the file exists and the file name can be dynamic like

    @markus.bohland@hotmail.de ('TestFile-Name-',utcnow(),'.csv')

    Note: you need to fromat utcnow as per the needed format

    and if file exists, then proceed for copy else ignore