How to find the size of a folder in azure data lake Gen2

Ravi Sai Somasekhar Mahasiva 21 Reputation points
2023-01-19T16:09:45.56+00:00

Hi Team,

I have a requirement to check the folder size in a container.
Eg: we have 5 folders in a container, we need to check size of each folder.

Is it possible to check size of folder in data lake Gen2 using blob fuse (Linux command), we are using command du -h --max-depth=1 (but we are getting size as 0), how to check the folder size using blob fuse. Could you please share your thoughts on it.

Thanks

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,315 questions
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 75,286 Reputation points Microsoft Employee
    2023-01-20T05:23:25.5866667+00:00

    Hello @Ravi Sai Somasekhar Mahasiva,

    Thanks for the question and using MS Q&A platform.

    You can easily get the size of a folder in ADLS Gen2 using Azure Storage Explorer

    Step1: Download the Azure Storage Explorer: https://azure.microsoft.com/en-us/products/storage/storage-explorer

    Step2: Install the storage explorer and then Sign-in with your Azure Account.

    Step3: Open the folder which you would like to view the size details.

    On the top bar menu choose More -> Folder Statistics will help you get the details of the Directory including the size in bytes.

    User's image

    You can also get this details programatically using PowerShell or Python

    Powershell: https://azuredude.blogspot.com/2020/10/get-azure-data-lake-gen2-container.html

    Python: https://cloudarchitected.com/2019/05/computing-total-storage-size-of-a-folder-in-azure-data-lake-storage-gen2/

    Hope this helps. Do let us know if you any further queries.


    Please don’t forget to Accept Answer wherever the information provided helps you, this can be beneficial to other community members.

    2 people found this answer helpful.