Why is Azure OpenAI Resource and models do not incur any costs as compute?

Question

Dear team,

I am trying to understand the pay-as-you-go model pricing for Azure openai resources.

Azure OpenAI service Resource created in a subscription with models deployed do not incur any costs until invoked (via API or playground). How does the compute costs get priced in? Can we really consider that the data we pass to the models are within the boundaries of the corporate netowrk? How does this differ from the model offered by OpenAI? Could you please help me understand this better?

Thanks,

Anitha

Answer 1

Hello @Ramaswamy, Anitha (AllianzGI)

Thanks for reaching out to us, for all Language models, images models and embedding models, Azure OpenAI service resources and models do not incur any costs as compute until they are invoked via API or playground. This means that you can create and deploy models without incurring any compute costs until they are actually used. It is only billed by tokens/ completions.

Please refer to the pricing page - https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/

For the fine-tuning model (which is not available for new customer now), you may be billed by compute, training and also the tokens you passed -

Regarding the data passed to the models, it is important to ensure that you are complying with any relevant data privacy and security regulations. You should also ensure that you have the necessary permissions and consent to use the data.

For your question 2 -

Azure OpenAI is a collaboration between Microsoft Azure and OpenAI¹. With Azure OpenAI, customers get the security capabilities of Microsoft Azure while running the same models as OpenAI. Azure OpenAI offers private networking, regional availability, and responsible AI content filtering. The data submitted to the Azure OpenAI Service remains within Microsoft Azure and is not passed to OpenAI (the company) for model predictions². The Azure OpenAI service automatically encrypts any data that is persisted to the cloud, including training data and fine-tuned models².The Azure OpenAI service differs from the model offered by OpenAI in that it is a cloud-based service that provides access to pre-trained models and allows you to deploy your own custom models.

I hope this helps clarify the pricing and differences between Azure OpenAI service and the OpenAI API. Let me know if you have any further questions!

Regards,

Yutong

-Please kindly accept the answer if you feel helpful to support the community, thanks a lot.