Thanks for the question, Here is the sample for token count for stream enabled. Jupyter notebooks to calculate tokens usage with Tiktoken for scenarios with and without Token Streaming. https://github.com/LazaUK/AOAI-Streaming-TokenUsage/tree/main
Retrieving token usage in Azure OpenAI response when streaming is enabled
chaymr
181
Reputation points
I have an Azure OpenAI deployment used by multiple internal users that charges back based on token usage found in the "usage" field of the API response. However, users who stream the response with "stream=True" do not receive the "usage" field in the Azure OpenAI response. Is there any way to retrieve the token count even with "stream=True"?.