Using wildcard characters to Delete the files

Question

Hi - In my datalake 2.0 I need to delete the files which DO NOT end with number. In the below list of files i need to delete the highlighted files. I am using delete activity in my pipeline(ADF) with wildcard expression in file name "*[abcdefghijklmnopqurstuvwxyz]".

filenew.parquet
file12.parquet
fileold.parquet
file.parquet
file14.parquet
file10.parquet

However, its not working. How can I delete the files which DO NOT end with numbers. What are the wildcard characters need to use.

Answer 1

Hi @rajendar erabathini ,

Thank you for posting query in Microsoft Q&A Platform.

In Azure data factory, allowed wild cards are *(matches zero or more characters) and ?(matches zero or single character). you can use ^ to escape if your file name has a wildcard or this escape character inside. Click here to see more examples.

Your case cannot able achievable using above wildcards in Azure data factory directly. Hence, you can leverage Azure functions may be to write a piece of code which filters filenames which don't have numbers in it.

Pass output array of filtered file names from Azure functions to ForEach activity to loop each file name and use delete activity inside ForEach activity to dynamically point to the file in the iteration and delete it.

You can use Azure Functions activity to run Azure function in Azure data factory.

Hope this will help. Please let us know if any further queries. Thank you.

----------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators