How does blob lifecycle management interact with directories in Azure Data Lake Storage Gen2?

Brandon Vincent 21 Reputation points
2021-02-04T18:39:47.14+00:00

My understanding is that in ADLS Gen 2, a directory appears like a 0-byte blob when the container is queried through the blob storage APIs.

However, in ADLS Gen 2 sometimes it is desirable to have a directory be permanent, even if it is never modified, because you can attach ACLs to directories. Imagine a directory got cleaned up by a lifecycle management policy, then you would have to re-create the directory and re-attach the desired ACLs.

However, also in a filesystem like ADLS Gen 2, it may be desirable to automatically clean up empty directories over time.

Therefore, I am curious how management policies interact with directories in ADLS Gen 2. Do lifecycle management policies have special consideration for blobs that are actually directories?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,376 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,770 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,498 questions
{count} votes

Accepted answer
  1. deherman-MSFT 34,026 Reputation points Microsoft Employee
    2021-02-08T20:03:38.61+00:00

    @Brandon Vincent
    Lifecycle Management only cleans up empty directories that match the policy defined for the account. There is an open issue to improve our documentation regarding ADLS Gen 2 and lifecycle management. You might be interested in following this to see if further information gets added.

    Hope this helps. Let me know if there are still questions you have and we will be happy to help.

    -------------------------------

    Please don’t forget to "Accept the answer" and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

    2 people found this answer helpful.
    0 comments No comments

0 additional answers

Sort by: Most helpful