How do you calculate the token needed for Azure open ai chat interface and how is it calculated after using tools?

俊岚 吴 0 Reputation points
2024-04-28T06:30:39.92+00:00

How do you calculate the token needed for Azure open ai chat interface and how is it calculated after using tools?

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,267 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Charlie Wei 2,845 Reputation points
    2024-04-28T12:03:47.05+00:00

    Hello 俊岚 吴,

    If you are developing with Python, it is recommended to use the tiktoken package to calculate the token usage.

    Another simpler method is to check the token consumption for a request from the HTTP response. A straightforward example of a response can be found in the Microsoft Learn documentation.

    {
      "choices": [
        {
          "finish_reason": "stop",
          "index": 0,
          "message": {
            "content": "...",
            "role": "assistant"
          }
        }
      ],
      "created": 1679001781,
      "id": "chatcmpl-6upLpNYYOx2AhoOYxl9UgJvF4aPpR",
      "model": "gpt-3.5-turbo-0301",
      "object": "chat.completion",
      "usage": {
        "completion_tokens": 39,
        "prompt_tokens": 58,
        "total_tokens": 97
      }
    }
    

    Best regards,
    Charlie


    If you find my response helpful, please consider accepting this answer and voting 'yes' to support the community. Thank you!