Azure Data Factory Copy Order

Question

Hi,

I have a blob storage with multiple files with the same schema. The files use a standard naming convention i.e. Filename_yyyy-mm-dd. A copy activity is setup to Upsert records (SINK) to a SQL database. I have setup Get Metadata and ForEach activity to read the files. What order does ADF read and process the files by default? Is it in alphabetical and numeric order, or does it process the files by the date and time when the file was added to the Blob storage?

Ideally, I want the files to process in the following order no matter when the files are added i.e.:

Filename_2022-10-16.csv
Filename_2022-10-17.csv
Filename_2022-10-18.csv
..
Filename_2022-10-15.csv <-- added to the Blob storage after the 3 files above

In this scenario, the pipeline will be scheduled to run once a day.

Thank you in advance and I appreciate any input anyone can give.

Answer 1

Hi @Jay ,

Welcome to Microsoft Q&A platform and thanks for posting your question here.

As I understand your query, you want to know in which sequence order the file would be copied in ADF, kindly let me know if that is not your ask here.

First of all, when we upload the files to blob storage, irrespective of the date and time, the file automatically gets sorted alphabetically.

In ADF , files would be picked alphabetically to copy the data if you have selected 'Sequential' option in ForEach block.

Kindly refer to the below video for more details:

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators