When making requests to Azure OpenAI's GPT-3.5-turbo with streaming enabled, I received a response where the usage field, which represents the usage statistics, was null. Why is this the case?

Question

When making requests to Azure OpenAI's GPT-3.5-turbo with streaming enabled, I received a response where the usage field, which represents the usage statistics, was null. Why is this the case?

劉亞坤 30

User's image

romungi-MSFT 48,911 Reputation points Microsoft Employee Moderator

2023-08-15T15:45:05.7466667+00:00

@劉亞坤 Is this a case with each of your request for chat completions? Does it display the usage information if streaming is set to false? Also, what API versions are you using with the above requests?
劉亞坤 30 Reputation points

2023-08-16T01:22:27.53+00:00

Based on my testing, I can confirm that when the stream is open, I indeed cannot receive the usage field. However, when I close it, I can receive it normally. I noticed another technician's response in the thread stating that the usage field is not currently supported when the stream is open. It seems to be true.

Accepted answer

0 additional answers

Your answer

romungi-MSFT 48,911 Reputation points Microsoft Employee Moderator

2023-08-15T15:45:05.7466667+00:00

@劉亞坤 Is this a case with each of your request for chat completions? Does it display the usage information if streaming is set to false? Also, what API versions are you using with the above requests?
劉亞坤 30 Reputation points

2023-08-16T01:22:27.53+00:00

Based on my testing, I can confirm that when the stream is open, I indeed cannot receive the usage field. However, when I close it, I can receive it normally. I noticed another technician's response in the thread stating that the usage field is not currently supported when the stream is open. It seems to be true.

Answer 1

John Sanders 176 Microsoft Employee

Stream = On does not support usage tokens at the end of the request at this time; this is not a supported capability. It is in our backlog, but no ETA at this time.

劉亞坤 30 Reputation points

2023-08-16T01:26:29.93+00:00

This is really heartbreaking. I hope for early support.
Bentley, Thomas (Cognizant) 0 Reputation points

2023-12-19T09:35:59.5466667+00:00

Seems like a much needed feature..

Share via

When making requests to Azure OpenAI's GPT-3.5-turbo with streaming enabled, I received a response where the usage field, which represents the usage statistics, was null. Why is this the case?

0 additional answers

Your answer