Setting up a computer vision token based API on Azure

Question

We're trying to develop a platform like OpenAi's text to image website (https://deepai.org/machine-learning-model/text2img), where the user inserts a prompt, presses the generate and the promp is sent to an endpoint that performs the machine learning operations and returns the result.

We have everything ready when it comes to functionality, we're just looking for the best way to host this on Azure as a way of only paying as we use it, to reduce costs.

Our current idea is the following:

• A HTTP trigger function (Premium Plan) with the endpoint that receives the data from the web app (image or video), processes it and sends the response back (json).
• A web application for the user interface that sends a request with the image or video to the azure function trigge endpoint.

The main problem so far has been that azure functions doesn't seem to work with GPUs.

We've also had a look on this topic -- https://github.com/puthurr/python-azure-function-gpu -- but if we're paying for the allocation of a kubernetes GPU cluster + the execution of the function itself, wouldn't it be adittional costs?

We're a little lost on this part, so we would appreciate any help or suggestions on the best way to do this.

Answer 1

@Pedrojfb Thanks for the question. Azure Functions currently does not support GPUs. One option could be to use Azure Kubernetes Service (AKS) with GPU-enabled nodes to run your machine learning workloads. This would allow you to take advantage of the scalability and flexibility of AKS while still being able to use GPUs for your computations.

However, it’s important to note that using AKS with GPU-enabled nodes will incur additional costs compared to using Azure Functions alone. It’s up to you to weigh the benefits of using GPUs for your workloads against the additional costs.