I want to know limitation related to OpenAI service

Question

Hello. I'm working on a feature of summarizing massive documents using OPENAI Service, and I found out that there are some limitaion using OPENAI features. so I have few questions to confirm if I'm understanding the documents correctly.

1. on this document(https://learn.microsoft.com/en-us/azure/ai-services/openai/quotas-limits) it says, "Max size of all files per upload" is limited to 16MB. does it mean that the max size of index I can use for BYOD is 16MB? I'm kinda confused because when I used 19.15 MB stoage size of index as a datasource for BYOD it didn't commit any error. but at the same time, it seems like it cannot find certain information from retrieved documents.
2. if the limitation is 16MB, is there any error message I can get through the response of API? and is there any plan to increase the limitation of size I can upload to OPENAI in the near future?
3. if the limitation is not 16MB, what is exactly the limitaion of the size I can upload?
4. I also found that in Cognitive Search Service(https://learn.microsoft.com/en-us/azure/search/search-limits-quotas-capacity) the max index/indexer numbers I can make for S3 is only 200. is there any plan to increase the maximum size in the future?
5. Can BYOD feature of OPENAI Sevice handle datasource(index) containing 1000 files having 1000 pages each? What would be the best practice to use such size of datasource for BYOD?
6. Also, is it possible to query datasource(index) for BYOD so that I can use only certain data in datasource? this question is related to 4. for I don't wanna make numerous indexes to filter data I'm using for BYOD REST API. I found out that there is 'filter' parameter(https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data#document-level-access-control), but it seems like I cannot use it for purposes like above. or am I understanding wrong?
7. I just confirmed that I can use 'filter' parameter to use only the data I wanna use from the index for BYOD REST API. if there is upload size limit for BYOD, and let's say that the data size filtered(by 'filter' parameter) from the index(which exceeds 16MB) is lower than 16MB can I still use the filtered data without any loss for BYOD feature?

thank you for the answer in advance!

Answer 1

@AGT 金恩哲 Thanks for reaching here! The 16MB limit applies to file upload, the files can be larger from Blob and the offline ingestion script can be used to work around any limitation implied by uploading content through the studio.