Getting the size of parquet files from azure blob storage

Question

I have a blob container abcd

The folder structure is like below:

abcd/Folder1/Folder a, Folder b…..Folder z

Inside a particular Folder a/v1/full/20230505/part12344.parquet

Similarly Folder b/v1/full/20230505/part9385795.parquet

Scenario is I need to get the size of each parquet files present in each folders a to z. I can see that there’s no get data size anymore in metadata activity in ADF. What else can be done here using ADF or ADB code?

Answer 1

Hi KEERTHANA JAYADEVAN - Thanks for reaching out.

I would recommend enabling blob inventory report for the scenario. You need to ensure that you have "Content Length" field in the same.

https://learn.microsoft.com/en-us/azure/storage/blobs/blob-inventory

You can then further leverage synapse or databricks to parse the report further and get the required details.

https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-inventory-report-analytics

https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-calculate-container-statistics-databricks

Hope that helps!

Let me know if there are any further queries/concerns, will be glad to assist.

---

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.