pricing fine-tuning GPT 3.5

Question

Hello,

I am now interested in fine-tuning GPT 3.5.

And I'm not sure how the price of running the model is handled.

I understand the cost of tokens in and out.

But I am not sure about the hourly price that is listed on the website. Is it the price per hour when the model calculates ? Or is it calculated from the time the model is available for request ?

If it is the second option. Are there any APIs available to turn the model on and off ? And if so how long does it take to start ?

Answer 1

Hi Jan,

Cheers for the question.

In your question you've specified time when the model 'calculates' as well as model 'availability'.
These words can get a bit vague in meaning, so I'll assume that calculation refers to time spent generating a response (this is formally referred to as compute time).

Once you have trained your model, you can deploy it into an Azure environment and access it as per your entity's needs. If you are designing an enterprise solution with live requirements, then this might be an ongoing cost. Comparatively, you might elect to use selected Azure services to activate the environments where your model is being accessed from. For various services, there are indeed API methods of passing commands to do things like 'switching off' or deactivating a deployed model for the sake of cost management. This is complex and contextual, generally.

Your further question, relating the 'starting' of models, I am not sure I understand. If you are worried that redeploying a model will incur time delays, this is probably not an issue. These processes can take varying times, but are generally not more than 10 minutes. Redeployment of fine-tuned models in Azure OpenAI is not the same as retraining, meaning it saves it's memory of the computationally intensive training process, and is 'reactivated' later according to your needs if desired.

Hope that answers everything =)