Determine whether Azure OpenAI model deployments are utilizing CPU or GPU resources?

Question

1. Are Azure OpenAI models individually hosted in dedicated infrastructure or do they share a common infrastructure within Microsoft?
2. How can we ensure consistent performance and responsiveness for Azure OpenAI-based applications, given that we lack control over the selection of the hosting hardware?

Thanks in advance!

Answer 1

@Chakaravarthi Rangarajan Bhargavi Thanks for the question, We cannot provide more details around this. Our architecture is protected/sensitive property of Microsoft. We don't disclose technical implementations of our Azure services as we could inadvertently introduce vulnerabilities and potential exploitation.