Azure Blob storage does not allow to delete the non empty folders using Java hdfs code

Question

I have an application which delete the folder inside the blob storage and the folders have some sub folders and files inside it using the Java hadoop/hdfs api .But as azure is not supporting the non empty folder to be deleted getting an exception as below
org.apache.hadoop.fs.azure.AzureException: com.microsoft.azure.storage.StorageException: This operation is not permitted on a non-empty directory.

I am using containers type of storage with hierarchy enabled

Answer 1

@SG Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

Note: Delete the file or directory (optionally recursively delete all files in the directory). This call throws an exception with IO_ERROR if the path is a non-empty directory and recursive is set to false or on other similar errors.

• You can use Azure Storage Explorer(Please refer to this article about how to install it and use it.), then nav to your Blob->Container-> right click the folder -> select delete. This can delete a non-empty folder.

or you can use AzCopy(see here for more details about this tool) with azcopy remove command and --recursive parameter.

Additional information : If you use the Delete Blob API to delete a directory, that directory will be deleted only if it's empty. This means that you can't use the Blob API delete directories recursively.

Based on the error message: You use the Azure Blob Storage as the staging linked service to link to a storage account that has the enabled hierarchical namespace, and that account uses key authentication in the linked service.

Recommendation: Create an Azure Data Lake Gen2 linked service for the storage, and select the Gen2 storage as the staging linked service in data flow activities.

If the issue still persist, I would I like wo work closer on this issue can you please share the code and screenshot of the error message with detailed information of the issue

Please let us know if you have any further queries. I’m happy to assist you further.

----------

Please do not forget to and wherever the information provided helps you, this can be beneficial to other community members.