Function app - accessing transformer models and/or large ML models

Question

In my function app, I need to load a pre-trained BERT model which is nearly 1GB. Another process of my code needs to use 'sentence transformer' models, which it will attempt to download automatically if not already on the local storage available to the function.

I want to ask what is the best way to approach this. Since functions are stateless, my assumption is that

• I need to 'host' these models somewhere, such as in a Blob or a File Share
• Everytime my function starts, it should download these models from the above to local storage, e.g., '/tmp
• Load the locally saved models

Is this correct?

Is there other better ways, as re-downloading every time the function is called for the first time after a redeployment can take some time.

Also, I read here https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale, which seems to suggest that to host local data files beyond 1GB, my only option is **Premium plan?
**
Many thanks for your advice.

Answer 1

@ZZ ,

Thank you for reaching out and sharing your concerns. Based on the conversation, here is a summarized and concluded answer for accessing transformer models and/or large ML models using Azure Function App:

Hosting Models Externally: You are correct in hosting the pre-trained BERT model and Sentence Transformer models externally, such as in Azure Blob Storage or Azure File Share. This ensures that the models are centrally managed and readily available for download.

Downloading Models to Local Storage: When the function is triggered, the models can be downloaded to the local /tmp directory. Subsequent requests on the same instance will reuse the cached models, minimizing redundant downloads.

Temporary Storage Limits: For Azure Functions, the temporary storage limit for the Consumption plan is less than 1GB. Specifically, it is 1.5 GB for Linux Consumption plans. If your application requires more temporary storage, consider using the Premium plan, which offers more flexibility and higher resource limits.

Premium Plan Benefits: The Premium plan not only provides more storage but also offers other benefits like reduced cold starts, virtual network connectivity, and the ability to run continuously. This makes it ideal for hosting and managing large models efficiently.

Ephemeral Nature of /tmp Storage: The /tmp storage in Azure Functions is ephemeral and subject to the lifecycle of the function instance. It is typically cleared after a function instance has been idle for a period of time, during redeployment, or scaling events.

Suitability for Long-Running Processes: Azure Functions are generally designed for short-lived, stateless executions. Managing large ML models with temporary storage limitations can be cumbersome, especially for tasks requiring long execution times and large model files.

In conclusion, while Azure Functions can be used for accessing transformer models and large ML models, it may not be the most suitable option for long-running processes due to the temporary storage limitations and the stateless nature of functions. The Premium plan can help mitigate some of these issues by providing more storage and other benefits.
Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.