OpenAI sentence embedding input array limit

Question

The documentation for Azure OpenAI Sentence Embedding says

Currently we accept a max array of 1.

Microsoft should know that this restriction renders the sentence embedding service quite useless. I'm forced to turn to another solution.

I need to build an initial catalog of about 310k sentence embeddings. I've been running for a couple of hours now but am only at 10%, and am receiving tons of messages like this:

Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 4 seconds. Please contact Azure support service if you would like to further increase the default rate limit.

I don't mind being rate-limited if I could only chunkify my requests. This same job only takes a few minutes with OpenAI (as in platform.openai.com) even though OpenAI restricts you to 1/5 the request rate. But alas Microsoft has decided to make life difficult by artificially preventing chunking.

This is not feasible.

Answer 1

Matthew Thomas I think there is an option to increase your quota limits through a request submitted through this form. Depending on the model used you can increase requests per minute (RPM) throughput & total tokens (prompt + completion). The requests are similar to the original request to gain access to Azure OpenAI but it can help you improve the current speed if request is approved. Thanks!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 2

+1, Azure is a horrible experience.