Deleteing Files from hot Storage Tier

Question

Hi Team ,

I have around 100Million files in Hot tier, I want to delete all the blob files except my last 30 days ones .

What transcations will it take to Delete the files .? Will it take read iterative opertions only & Delete is free .

Correct me if i am wrong .

Answer 1

Hope doing good!

To address or answer or query , please check below

Yes delete is free ,to explain better please check below

To delete files from Azure Blob Storage, you typically need to perform the following operations:

Listing Files: To identify the files you want to delete (i.e., all files except the last 30 days' ones), you'll need to list all the files in your Blob Storage container. This listing operation incurs costs based on the number of transactions, specifically READ operations.

Identify Files to Delete: After listing the files, you need to identify which files are older than 30 days based on their timestamps or metadata. This step does not involve additional transactions as it's part of your application logic.

Deleting Files: Once you've identified the files to delete, you can then delete them. The good news is that blob deletion operations in Azure Blob Storage are generally free. You don't incur additional charges when you delete blobs.

The most significant cost associated with your task is the initial listing operation to identify the files. With 100 million files in your Blob Storage, this listing operation can be substantial in terms of transaction costs. However, deleting the files itself is not a transaction cost.

Here are some considerations to optimize your process:

Prefix-Based Listing: If your files are organized with a consistent naming convention or prefix structure, you can use prefix-based listing to reduce the number of files you need to list.

Use Metadata: If your blobs have custom metadata that includes timestamps or other relevant information, you can filter and identify files based on metadata without listing all files.

Batch Processing: To manage the cost of listing and deleting files, consider processing and deleting files in smaller batches instead of trying to list and delete all files in a single operation.

Archiving: If these files have historical data that you don't need immediately, consider moving older files to a different storage tier like Cool or Archive to reduce costs. This might be more cost-effective in the long run.

Keep in mind that Azure's pricing and features may change over time, so it's important to consult the latest Azure Blob Storage pricing documentation to understand the exact costs associated with your specific scenario. Additionally, monitoring your Azure usage and costs through the Azure portal can help you stay informed about the financial aspects of your storage operations.