Is my understanding of the new Azure OpenAI Fine-Tuning pricing model accurate?

Question

Is my understanding of the new Azure OpenAI Fine-Tuning pricing model accurate?

Clay Casper 176

Reading through some of the documentation for the new fine-tuning capability for Azure OpenAI models (https://techcommunity.microsoft.com/t5/azure-ai-services-blog/fine-tuning-now-available-with-azure-openai-service/ba-p/3954693), it says there is a $7/hour hosting charge for GPT3.5-turbo. This comes out to over $5000 per month just for one fine-tuned model to be usable through Azure OpenAI. This seems like a lot. Is this an accurate understanding of the pricing model and if so, why is the hosting for a fine-tuned model so expensive? Maybe there's some complex storage mechanism for fine-tuned models, idk I'm not an AI expert.

AshokPeddakotla-MSFT 35,971 Reputation points Moderator

2023-10-28T14:20:26+00:00

Clay Casper Just checking if we are still connected on this discussion? Please let us know if you need further help.

If the response is helpful, please click Accept Answer and Click Yes so that we can close this thread.
AshokPeddakotla-MSFT 35,971 Reputation points Moderator

2023-11-02T17:15:20.21+00:00

Clay Casper Just checking if we are still connected on this discussion? Please let us know if you need further help.

If the response is helpful, please click Accept Answer and Click Yes So that we can close this thread.

1 answer

Your answer

AshokPeddakotla-MSFT 35,971 Reputation points Moderator

2023-10-28T14:20:26+00:00

Clay Casper Just checking if we are still connected on this discussion? Please let us know if you need further help.

If the response is helpful, please click Accept Answer and Click Yes so that we can close this thread.
AshokPeddakotla-MSFT 35,971 Reputation points Moderator

2023-11-02T17:15:20.21+00:00

Clay Casper Just checking if we are still connected on this discussion? Please let us know if you need further help.

If the response is helpful, please click Accept Answer and Click Yes So that we can close this thread.

Answer 1

Clay Casper Greetings!

Is this an accurate understanding of the pricing model and if so, why is the hosting for a fine-tuned model so expensive? Maybe there's some complex storage mechanism for fine-tuned models, idk I'm not an AI expert.

Your understanding is correct. As mentioned in the blogpost, Fine tuned models are subject to an hourly hosting charge, as well as token based pricing for input and output data.

User's image

There is a $7/hour hosting charge for GPT3.5-turbo fine-tuned model in Azure OpenAI Service. This hourly cost is for hosting the fine-tuned model on Azure infrastructure, which includes the cost of compute, storage, and networking resources.

The reason for the high cost of hosting a fine-tuned model is due to the complexity of the model and the resources required to run it. Fine-tuning a model involves training it on a large dataset to improve its accuracy for a specific task. This requires a lot of computational power and storage resources. Additionally, the fine-tuned model needs to be hosted on a powerful server to handle the increased workload. All of these factors contribute to the high cost of hosting a fine-tuned model.

We strongly recommend reviewing the pricing information for fine-tuning prior to make sure you are comfortable with the associated costs.

Also, please see Customize a model with fine-tuning (preview) and Plan to manage costs for Azure OpenAI Service for more details.

Do let me know if that helps or have any further queries.

If the response helped, please do click Accept Answer and Yes for was this answer helpful.

Doing so would help other community members with similar issue identify the solution. I highly appreciate your contribution to the community.

Ruben Aster 1 Reputation point

2024-10-24T06:53:00.5166667+00:00

Hosting fine-tuned OpenAI models is very expensive. In fact, hosting a fine-tuned model costs about 1.000€ per month without even using it.

In contrast, fine-tuning models directly with OpenAI (without Azure) has no hosting costs at all. Of course, this is no option for most of our customers.

The problem I see here is that fine-tuned models are often used for specific automation tasks (text entity extraction, finding semantic differences, etc.), sometimes requiring multiple fine-tuned models per task. Usually you have dozens of the automation tasks for that specialized fine-tuned models would be a perfect fit.

But often, 1.000€ per months is not worth the price for a small automation task. Especially if you have dozens and each of these requiring another fine-tuned model.

Share via

Is my understanding of the new Azure OpenAI Fine-Tuning pricing model accurate?

1 answer

Your answer