How many concurrent threads can be ran using one assistant.

Question

I have 100+ applications. Each application can have many user. I am considering to use one assistant for one application. I want to know the number of threads each assistant can use.

Answer 1

Hi @Giridhar Vinukonda,

Thank you for reaching out to Microsoft Q&A forum!

If you're using the Azure OpenAI Assistants platform, the number of concurrent threads that can be run using one assistant depends on the specific model and configuration. Azure OpenAI has certain limitations and quotas based on your subscription plan and the model's capacity.

A chat session, also known as a thread within the Assistant's API, is where the conversation between the user and the assistant takes place. There is no limit to the number of messages in a thread, as the assistant automatically compresses requests to fit within the model's input token limit. Token management is fully abstracted and handled by the Assistant's API, meaning you don't control how many tokens are passed during each turn.

For more info:

• OpenAI Assistants.
• Quotas and limits.

I hope this helps. And, if you have any further query do let us know.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful.