Deploying LLM to website for public use

Question

Hi,

I have a LLM (I haver a llama2 version and gpt version) that was fed some information (rag). I want to deploy the model to my website (made with flask). What is the best way to do this? (with regard to both cost & latency)

The website/chatbot works well on my local server, I want to make it available for public access

Answer 1

@VT You have an option to use Azure Machine Learning service and register the model on your Azure ML workspace and you can deploy it as an online endpoint. However, if your model has dependency on the platform or the web server to work as expected, you will have to use an app service deployment or a VM to deploy the same on a server and host it.

With Azure ML workspace you only pay for the compute & storage & other networking configurations used and it can be scaled if you use compute clusters as compute type. I hope this helps!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.