how to delete a folder in ADLS with different name

Question

Hi,

I have a pipeline who generate parquet file in ADLS gen2 with similar naming patternfor each folder every day. bust some times a bug cause an empty folder generate in all folders with different naming. I would like to use delete activity befor my pipeline to delete folder with different naming pattern.

can you help me in this please?

thank you

Answer 1

Hi @17285674 ,

Welcome to Microsoft Q&A platform and thanks for posting your query here.

As I understand your issue, you want to delete the folders which are empty and have some random names.

You can try the following approach to achive that:

1. Use Get metadata activity where dataset is not pointing to any container or any folder. Select 'Child Items' in the field list to fetch the list of all the containers in the ADLS account.
2. Use ForEach activity to iterate through the output of get metadata activity using the expression :

   @activity('Get Metadata1').output.childItems

1. Use Another GetMetadata activity inside ForEach and make the dataset parameterized to get all the file names within each of the folders.
2. Use If Block to check if the step3 output is empty , which means if the folder doesn't contain any file, then use delete activity inside If block to delete that particular folder. Use the following expression in If condition :

   @empty(activity('Get Metadata2').output.childItems)
   

---

Hope it helps. Kindly accept the answer and mark it as helpful.