Share via

The maximum batch size and Input Size of Azure OpenAI for textembeddinglarge model

Sarvi 0 Reputation points
2024-05-08T12:32:22.04+00:00

I am using textembeddinglarge model from Azure OpenAI. What is the maximum batch size and token limit / batch for this model.

Azure OpenAI in Foundry Models
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Charlie Wei 3,335 Reputation points
    2024-05-08T15:00:35.0766667+00:00

    Hello Sarvi,

    The text-embedding-3-large model has a maximum request size of 8,191 tokens per single call, as referenced in this document.

    Additionally, it has a request limit of 350K Tokens-Per-Minute (TPM), as further detailed in this documentation.

    Best regards,
    Charlie

    If you find my response helpful, please consider accepting this answer and voting yes to support the community. Thank you!

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.