Batch Support For Azure OpenAI Embedding

Question

Batch Support For Azure OpenAI Embedding

Sicong Yang 30 Microsoft Employee

According to Azure OpenAI Service REST API reference - Azure OpenAI | Microsoft Learn, currently AOAI's embedding API only accept a max array of 1. I'm just curious about why it doesn’t allow batched embeddings. When would you support batch embeddings?

Shahar Keren 0 Reputation points Microsoft Employee

2023-07-03T08:34:24.35+00:00

Same here. The answer did not answer the question which is for the specific limitation of batch call for embedding.
We are 3 months after the original question, and still not estimation in documentation on when this is going to be supported.
Chris Rickard 5 Reputation points

2023-07-19T03:22:16.9133333+00:00

Agree, would be good to get an idea of if and when batch embeddings with be supported on Azure OpenAI.
Nikhil Ashok Patange 0 Reputation points

2023-08-10T23:29:04.43+00:00

Here is an article to solve this issue,

https://www.linkedin.com/posts/nikhil-patange-81823858_azureopenai-huggingface-langchain-activity-7095544547211976704-c6IT?utm_source=share&utm_medium=member_desktop

Accepted answer

0 additional answers

Your answer

Shahar Keren 0 Reputation points Microsoft Employee

2023-07-03T08:34:24.35+00:00

Same here. The answer did not answer the question which is for the specific limitation of batch call for embedding.
We are 3 months after the original question, and still not estimation in documentation on when this is going to be supported.
Chris Rickard 5 Reputation points

2023-07-19T03:22:16.9133333+00:00

Agree, would be good to get an idea of if and when batch embeddings with be supported on Azure OpenAI.
Nikhil Ashok Patange 0 Reputation points

2023-08-10T23:29:04.43+00:00

Here is an article to solve this issue,

https://www.linkedin.com/posts/nikhil-patange-81823858_azureopenai-huggingface-langchain-activity-7095544547211976704-c6IT?utm_source=share&utm_medium=member_desktop

Answer 1

romungi-MSFT 48,916 Microsoft Employee Moderator

@Sicong Yang I think due to the current demand for the service and compute limitations there are certain limits on the rate and token sizes that can be accepted by the API but there is an option to increase your quota limits through a request submitted through this form. Depending on the model used you can increase requests per minute (RPM) throughput & total tokens (prompt + completion). The requests are similar to the original request to gain access to Azure OpenAI but it can help you improve the current speed if request is approved. Thanks!! If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

SZETO TAI SHUN 0 Reputation points

2023-04-21T09:59:36.2733333+00:00

The application form seems have no box to request for increase the array size to larger than 1.

Share via

Batch Support For Azure OpenAI Embedding

0 additional answers

Your answer