Share via

The maximum batch size and Input Size of Azure OpenAI for textembeddinglarge model

Sarvi 0 Reputation points
2024-05-08T12:32:22.04+00:00

I am using textembeddinglarge model from Azure OpenAI. What is the maximum batch size and token limit / batch for this model.

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
4,086 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Charlie Wei 3,335 Reputation points
    2024-05-08T15:00:35.0766667+00:00

    Hello Sarvi,

    The text-embedding-3-large model has a maximum request size of 8,191 tokens per single call, as referenced in this document.

    Additionally, it has a request limit of 350K Tokens-Per-Minute (TPM), as further detailed in this documentation.

    Best regards,
    Charlie

    If you find my response helpful, please consider accepting this answer and voting yes to support the community. Thank you!

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.