Azure Data factory Dynamically create folder to export files

Question

Running an ADF daily pipeline that runs multiple table copy activity from Azure MySQL tables to text. the need is to create today's date in format YYYYMMDD and send all files (database export files) to today folder.
How can I create dynamically this folder and then send all files inside this folder?

thank you,

Answer 1

Hi @drhorg ,
Thankyou for using Microsoft Q&A platform and thanks for your question.
As I understand your query, you are trying to load files into specific folders based on today's date. Please let me know if my understanding is incorrect.

You can create a control table having details of the table list you want to copy from Az SQL to .txt format, assuming you want to load each table as a separate .txt file.

Use copy activity inside ADF pipeline and parameterize the source and sink dataset by creating the parameter named TableName

In the dataset connection tab, use the following expression to make the foldername dynamic:

@concat(dataset().TableName,'_',addminutes(addhours(formatdatetime(utcnow(),'yyyy-MM-dd HH:mm:ss'),5),30),'.txt')

For more details, please check this video: https://youtu.be/FXw1gPaa2-M?t=1903

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators