Is there a tenant limit for Azure OpenAI multi-tenancy?

Question

I am implementing a Chatbot widget with Azure OpenAI integration, with hope that it would work as a new feature for my company's web product. Our product's purpose is to manage, store and utilize private information for companies. And our chatbot wishes to assist navigation and usage of that web.

To which we enter a more technical question, what model should I use if I were to commercialize our resources. My first and current option is to have 'Tenants' all have access to a provider's subscription with a secured dataset trained AI only accessible by them, they all should have their own trained product which will go through our resources. The other option is to have them carry their own subscriptions using their own datasets, pretty much handling everything by themselves. This will lift a lot of burden from our back but put a lot on theirs so less consumer engagement, we do not want that.

The issue with option 1 is that, the process is quite confusing, I have a lot of questions. I know that Azure have a multi-tenancy function, I have not tested it on OpenAI yet but I was informed of its possibility. And even if there is that option, is there a limit on the amount of tenants we have connected that subscription? and the customization of the Ai will also needs to be searched up on. Besides, the pricing of the whole subscription + quotas limit increase would also be a question in concern. Or should I go with pay-as-you-go option? And would large amount of tenancy be a cause for the Noisy Neighbour issue?

Any questions answered or informed would be massively appreciated.

Thanks,
An

Answer 1

Hi An Phan Son,

Thanks for reaching out to Microsoft Q&A.

When implementing a multitenant arch using azure OpenAI, several factors must be considered, including tenant limits, isolation models, and potential performance issues.

Tenant limitations and isolation models:

Azure OpenAI supports various multitenancy models, and while there is no explicit limit on the number of tenants you can connect to a single subscription, practical considerations such as resource management and performance may impose effective limits. The choice of isolation model is crucial:

1. Azure OpenAI for each tenant in provider's sub:
   • Data Isolation: High
   • Performance Isolation: High
   • Deployment Complexity: Low to medium
   • This model allows each tenant to have a dedicated instance, providing strong data and performance isolation, but requires managing multiple resources as the number of tenants grows.
2. Azure OpenAI for Each Tenant in Their own subscription:
   • Data Isolation: Very high
   • Performance Isolation: High
   • Deployment Complexity: High
   • Each tenant manages their own instance, which can alleviate the provider's burden but may lead to lower consumer engagement due to increased complexity for tenants.
3. Shared Azure OpenAI Instance:
   • Data Isolation: Low
   • Performance Isolation: Low to medium
   • This is the easiest to implement but can lead to the "Noisy Neighbor" problem, where one tenant's usage negatively impacts others. It also complicates data security, especially with sensitive information.

Customization and Pricing Considerations:

Customization of AI models for each tenant is essential, especially if different tenants have unique compliance or operational needs. You may need to implement mechanisms to allow tenants to customize their models while maintaining data security and privacy.

In terms of pricing, azure offers both pay-as-you-go and Provisioned Throughput Units (PTU's), which allow for reserved capacity at a monthly commitment. The choice between these options depends on your expected usage patterns and budget considerations.

Performance and Resource Management:

To mitigate issues like the Noisy Neighbor effect, your application should be multitenancy-aware, tracking resource usage (e.g., tokens consumed) for each tenant. This ensures fair resource allocation and can help manage costs effectively.

To sum up, while azure OpenAI does not impose a strict tenant limit, the arch. you choose will significantly impact performance, security, and operational complexity. It is critical to carefully evaluate the isolation models and their implications for your specific use case, especially when dealing with sensitive data and compliance requirements.

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.