Local storage usage in Azure Functions

Question

Hi,
We have an Queue storage trigger Azure function that is doing image processing and generate lots of files to the temp folder during its execution (it can gen up to 10 gigs per call).
Our concern is that as the workload increases and multiple instances are running in parallel on the same host we will reach the 250 GB max storage limit of the Premium plan.
What configuration options /design approach we can implement to avoid this?
Thanks

Answer 1

Hi @Eran David

Blob storage can certainly help you out. By using Blob storage, you can easily bind to write the files to a blob container and configure the lifecycle management policy to automatically delete the files after a certain period of time.

To implement this solution, you will need to modify your Azure function code to use the Blob storage output binding.

https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob-output?tabs=in-process&pivots=programming-language-csharp#usage

https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-policy-configure?tabs=azure-portal#create-or-manage-a-policy

Hope that helps.

If you need further help on this, tag me in a comment.

If the suggested response helped you resolve your issue, please 'Accept as answer', so that it can help others in the community looking for help on similar topics.

Answer 2

Hi,

You could use one of the Isolated v2 Service Plans which would give you 1TB storage. This runs in App Service Environment v3.

https://learn.microsoft.com/en-us/azure/app-service/environment/overview

If the above was helpful please click Accept Answer.

Thanks.

-TP

Answer 3

What we did is to set the max batchSize in host.json to 4, so only 4 queue triggered processes can run on a single VM in parallel.

{
    "version": "2.0",
    "extensions": {
        "queues": {
            "batchSize": 4,
            "newBatchThreshold": 1,
        }
    }
}