How to find the size of a folder in azure data lake Gen2

asked 2023-01-19T16:09:45.56+00:00
Ravi Sai Somasekhar Mahasiva 21 Reputation points

Hi Team,

I have a requirement to check the folder size in a container.
Eg: we have 5 folders in a container, we need to check size of each folder.

Is it possible to check size of folder in data lake Gen2 using blob fuse (Linux command), we are using command du -h --max-depth=1 (but we are getting size as 0), how to check the folder size using blob fuse. Could you please share your thoughts on it.

Thanks

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
891 questions
No comments
{count} votes

1 answer

Sort by: Most helpful
  1. answered 2023-01-20T05:23:25.5866667+00:00
    PRADEEPCHEEKATLA-MSFT 52,956 Reputation points Microsoft Employee

    Hello @Ravi Sai Somasekhar Mahasiva,

    Thanks for the question and using MS Q&A platform.

    You can easily get the size of a folder in ADLS Gen2 using Azure Storage Explorer

    Step1: Download the Azure Storage Explorer: https://azure.microsoft.com/en-us/products/storage/storage-explorer

    Step2: Install the storage explorer and then Sign-in with your Azure Account.

    Step3: Open the folder which you would like to view the size details.

    On the top bar menu choose More -> Folder Statistics will help you get the details of the Directory including the size in bytes.

    User's image

    You can also get this details programatically using PowerShell or Python

    Powershell: https://azuredude.blogspot.com/2020/10/get-azure-data-lake-gen2-container.html

    Python: https://cloudarchitected.com/2019/05/computing-total-storage-size-of-a-folder-in-azure-data-lake-storage-gen2/

    Hope this helps. Do let us know if you any further queries.


    Please don’t forget to Accept Answer wherever the information provided helps you, this can be beneficial to other community members.