What is the token limit for input documents in the "Bring your own data" service?

AMROUN Lysa 241 Reputation points
2023-11-21T16:50:21.94+00:00

Hello,

I would like to know the maximum number of tokens that chatgpt3.5 can ingest from my documents. I use chatgpt3.5 in azure open ai studio for question and answer on my documents through the Bring Your own data" service and when I ingest 10 files (each file has several pages) I get the following warning:

User's image

What is the limit and is it possible to increase the limit of tkens ingested by the model?

thank you

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,708 questions
Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,577 questions
Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
2,607 questions
0 comments No comments
{count} votes

Accepted answer
  1. YutongTie-MSFT 48,166 Reputation points
    2023-11-21T23:25:13.7866667+00:00

    @AMROUN Lysa

    Thanks for reaching out to us, please see below table for the max request since the limit depends on the model -

    Reference document - https://learn.microsoft.com/en-us/azure/ai-services/openai/quotas-limits

    GPT 3.5

    User's image

    GPT 3.5 Turbo

    User's image

    More information you may need is here -
    System message -

    https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data?tabs=ai-search

    Give the model instructions about how it should behave and any context it should reference when generating a response. You can describe the assistant's personality, what it should and shouldn't answer, and how to format responses. There's no token limit for the system message, but will be included with every API call and counted against the overall token limit. The system message will be truncated if it's greater than 400 tokens.

    Can I use quota to increase the max token limit of a model?

    You can refer to the document - https://learn.microsoft.com/en-us/azure/ai-services/openai/faq#can-i-use-quota-to-increase-the-max-token-limit-of-a-model-

    No, quota Tokens-Per-Minute (TPM) allocation isn't related to the max input token limit of a model. Model input token limits are defined in the models table and aren't impacted by changes made to TPM.

    Please let me know if you need any more information, I hope it helps.

    Regards,

    Yutong

    -Please kindly accept the answer and vote 'Yes' if you feel helpful to support the community, thanks a lot.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful