Azure Machine Learning
An Azure machine learning service for building and deploying models.
3,340 questions
How much does it cost to run Llama 2 on azure
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 74%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Anas Syed
20
Reputation points
I am trying to deploy Llama 2 instance on azure and the minimum vm it is showing is "Standard_NC12s_v3" with 12 cores, 224GB RAM, 672GB storage. It costs 6.5$/h and 4K+ to run a month is it the only option to run llama 2 on azure.
Azure Machine Learning
Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
3,631 questions
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 11%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERM%3C/text%3E%3C/svg%3E)
Ramr-msft 17,826 Reputation points2023-08-25T10:26:18.3266667+00:00 @Anas Syed Thanks for the question, Falcon LLMs models need Nvidia A100 GPUs to run. You will need quota for one of the following Azure VM instance types that have the A100 GPU: "Standard_NC48ads_A100_v4", "Standard_NC96ads_A100_v4", "Standard_ND96asr_v4" or "Standard_ND96amsr_A100_v4".
With AOAI you pay per use (tokens), where with other foundation models you need to host them yourselves and thus pay based on the compute required.
Sign in to comment
Sign in to answer