What is the max amount of data an Azure OpenAI Gpt-4 model can use?

Question

Hello, I am interested in connecting a gpt-4 model to my own data through Azure OpenAI and I am trying to understand what the current limitations are in terms on how much data a single gpt-4 model can process/access.

I am aware of the quotas and limits tables but need to understand them better (https://learn.microsoft.com/en-us/azure/ai-services/openai/quotas-limits)

The site says max size of files per upload is limited to 16 MB and max file size per Assistants is limited to 512 MB and Assistants token limit is 2M tokens. Does this mean that one model can at maximum have access to 512 MB of data or 2 million tokens (minus system message and model response), whatever is smallest regardless of which data source I have used? (Azure AI Search, Blob storage, manual upload...) Am I understanding this correctly? I am trying to get an estimate on how much data in form of text one model can process/access.

Answer 1

Hello Karl-Johan Olofsson,

The information you provided regarding the Assistants API is accurate; it is a new feature that was just released this month.

If your considerations are focused on "connecting to my own data" and "how much data", then the On Your Data feature might be a more suitable option for you. The data for this feature is stored on Azure Blob and Azure AI Search, which essentially allows you to regard it as supporting an unlimited amount of data.

Best regards,
Charlie

---

If you find my response helpful, please consider accepting this answer and voting 'yes' to support the community. Thank you!