Copy multiple files from Storage

Question

Dear All,

I an new to Data Engineer and need your suggestion here.
I have a pipeline with defaulted File Path and File Name as shown below.

Now we have multiple files available in the same File Path.So what changes should i do so that all the files are copied from the source.

Also this a one time activity where i have to copy multiple files.How can i achieve this without publishing the Pipeline.

Thanks
Faizan

Answer 1

Hello @Faizan Arefin ,

Thanks for the question and using MS Q&A platform.

If it is a onetime action and if you don't want to publish the changes, then you can clone the pipeline and just update the copy activity settings as shown in below image using wild card file name and folder path pipeline parameter for your folder path and then click on debug button (click only once) highlighted below. Since you will be using wild card naming (*) for files, all files in that folder path will be copied to your sink data store. Once you are done with this action you can undo the changes.

For more information please refer to this documentation: Folder and file filter examples

Also here is a demonstration video by one of the community volunteers - ADF Wildcard Explained in Detail | Copy activity

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators