Does Azure Open AI Charge for Ratelimit error or any other 40x error?

Question

Does Azure OpenAI charges cost, even when request failed, for example because of rate limit or token limit?

I am trying to implement exponential backlog to mitigate the problem of rate limit, which means some request may sometimes fail if the rate limit or Request per Minutes/ Token Per Minute limit is exceeded.

So the question is: If I retry 10 times and the request still fail, will I be charged for these 10 time trail or only for the trial which succeeded.

Thanks!

Answer 1

Hello @CarelleD , Thank you again for your time and patience throughout this issue.

The charges are applied depending on the error type. In general, calls that resulted in a 200 or 400 code should contribute towards the billed token count.  None of the calls which caused a 429 error should not contribute to the billed tokens. 

Regarding your question about 429 errors, these are triggered when the requests exceed your limit of Requests per Minute or Tokens per Minute and do not incur any charges. Since Azure OpenAI generates no response and the input tokens are not used, you are not billed for these errors.

I hope this helps.

Regards,

Vasavi

Please remember to "Accept Answer" if any answer/reply helped, so that others in the community facing similar issues can easily find the solution.