How can I detect deleted blobs

Question

Azure Storage Explorer tells me I have "Deleted blobs: 38,062 blobs, 18.53 GiB (19,896,279,242 bytes)". However when I execute my program to list blobs where BlobItem.Deleted equals true it finds 0. when I asked it to list blobs where BlobItem.VersionId = null it also returns 0.

I have over 200k blob items. How can I find AND eliminate them to recover this space?

            var blobEntities = container.GetBlobs(traits: BlobTraits.All, states: BlobStates.DeletedWithVersions);

            foreach (BlobItem blobItem in blobEntities)

            {

                totalBlobs++;

                if (blobItem.Deleted || blobItem.VersionId == null)

                {

                    deletedBlobs++;

                    string name = blobItem.Name;

                    long? size = blobItem.Properties.ContentLength;

                    AccessTier? accessTier = blobItem.Properties.AccessTier;

                    DateTimeOffset lastModified = (DateTimeOffset)blobItem.Properties.LastModified;

                    DateTime lastModifiedUtc = lastModified.UtcDateTime;

                    DateTime softDeleteOn = lastModifiedUtc.AddDays(90);

                    DateTime hardDeleteOn = lastModifiedUtc.AddDays(120);

                    outputFile.WriteLine($"{name}, {size}, {accessTier}, {lastModifiedUtc}, {softDeleteOn}, {hardDeleteOn}");

                }

            }

Answer 1

Hello Adrian Griffith,

Thanks for your query !

Adding to above response, Did you try the azure power shell commands mentioned in the below article?

https://learn.microsoft.com/en-us/powershell/module/az.storage/get-AzStorageblob?view=azps-5.9.0

Get-AzStorageContainer -Name container* | Get-AzStorageBlob -IncludeDeleted

Container Uri: https://storageaccountname.blob.core.windows.net/container1

Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDeleted ---- -------- ------ ----------- ------------ ---------- ------------ --------- test1 BlockBlob 403116 application/octet-stream 2017-11-08 07:53:19Z 2017-11-08 08:19:32Z True test1 BlockBlob 403116 application/octet-stream 2017-11-08 09:00:29Z True test2 BlockBlob 403116 application/octet-stream 2017-11-08 07:53:00Z False

You can also use az cli command https://learn.microsoft.com/en-us/cli/azure/storage/blob?view=azure-cli-latest#az_storage_blob_list

az storage blob list -c mycontainer --prefix foo --include deleted

Note: Use --include v instead of d If versioning is enabled for the blobs to see previous version/deleted blobs.

refer - https://learn.microsoft.com/en-us/azure/storage/blobs/blob-containers-cli

Customize the above commands accordingly to list only deleted ones.

To permanently delete soft-deleted blobs in your Azure Storage account and free up capacity, you have a few options:

1.Using Azure CLI: If you have permanent delete enabled for your storage account, you can use the deletetype=permanent query parameter to permanently delete a soft-deleted snapshot or deleted blob version.

Here’s an example of how to do it using Azure CLI:

az storage blob delete --container-name <container-name> --name <blob-name> --deletetype permanent

Replace <container-name> with the actual container name and <blob-name> with the blob you want to permanently delete

2.PowerShell Script:You can use the PowerShell script provided in the Microsoft Community Hub to permanently delete soft-deleted objects.

The script allows you to specify parameters such as container name, blob prefix, tier, and last modified date. It’s essential to understand the script’s limitations and permissions before running it.

refer - https://techcommunity.microsoft.com/t5/azure-paas-blog/azure-storage-permanent-delete-soft-deleted-objects/ba-p/3600967

Disable soft delete feature on your storage account before running the script. Otherwise, the soft-deleted snapshots remain in a soft-deleted state.

After running the script, you can re-enable the Soft Delete feature if needed.

Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

Hi @Adrian Griffith - Thanks for reaching out.

If the soft delete was enabled during the time when the deletion was performed, then the blobs will go into soft deleted state and will remain in that state until their retention period expires. If not, an you enabled it to post the deletion, then it will not have any effect on the blobs deleted.

The same is with versioning, if you have enabled versioning then the deleted blobs become the older version and then have to be removed. If not enabled then even this will not be affecting the same.

Lastly, considering this is a bulk deletion, it might take some time for garbage collection to happen at the backend. Post that the capacity aggregation will take place and you shall then see the actual capacity.

Below is a sample PS script that will help you in identifying if there are any soft deleted or versioned blobs present in your account:
https://github.com/Azure-Developer-Support/CodeRepository/blob/master/Storage/Powershell/GetContainerUsagealongwithSoftdeleteandversion.ps1

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.