pricing fine-tuning GPT 3.5

Jan Soukeník 20 Reputation points


I am now interested in fine-tuning GPT 3.5.

And I'm not sure how the price of running the model is handled.

I understand the cost of tokens in and out.

But I am not sure about the hourly price that is listed on the website. Is it the price per hour when the model calculates ? Or is it calculated from the time the model is available for request ?

If it is the second option. Are there any APIs available to turn the model on and off ? And if so how long does it take to start ?

Azure Cost Management
Azure Cost Management
A Microsoft offering that enables tracking of cloud usage and expenditures for Azure and other cloud providers.
2,100 questions
Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,250 questions
0 comments No comments
{count} votes

Accepted answer
  1. Jay Woodhill 80 Reputation points

    Hi Jan,

    Cheers for the question.

    In your question you've specified time when the model 'calculates' as well as model 'availability'.
    These words can get a bit vague in meaning, so I'll assume that calculation refers to time spent generating a response (this is formally referred to as compute time).

    Once you have trained your model, you can deploy it into an Azure environment and access it as per your entity's needs. If you are designing an enterprise solution with live requirements, then this might be an ongoing cost. Comparatively, you might elect to use selected Azure services to activate the environments where your model is being accessed from. For various services, there are indeed API methods of passing commands to do things like 'switching off' or deactivating a deployed model for the sake of cost management. This is complex and contextual, generally.

    Your further question, relating the 'starting' of models, I am not sure I understand. If you are worried that redeploying a model will incur time delays, this is probably not an issue. These processes can take varying times, but are generally not more than 10 minutes. Redeployment of fine-tuned models in Azure OpenAI is not the same as retraining, meaning it saves it's memory of the computationally intensive training process, and is 'reactivated' later according to your needs if desired.

    Hope that answers everything =)

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful