How is the open ai service managed for use in Copilot studio?

Question

At the global level (non-topic) the generative AI settings only allow you to choose a model. There has to be an openai service backing this model with inference but it seems it is purely managed and a black box.

If an agent is deployed to teams or to an intern website, does its inference endpoint get set to this managed open ai service and will there ever be an issue with connectivity depending on what channel it is published to?

Do you have any architecture diagrams showing this relationship so I can understand? I assume the company O365/power platform wraps it all so there is no need to worry about connectivity/security?

Thanks so much!

Answer 1

Hello Toby,

Thank you for your question about OpenAI service management in Microsoft Copilot Studio. Let me address your concerns about the managed OpenAI service, inference endpoints, and connectivity.

OpenAI Service Management in Copilot Studio

You are correct that Microsoft Copilot Studio uses a managed OpenAI service that operates as a "black box" from the user perspective. Here's how it works:

1. Managed Infrastructure

Microsoft handles the entire OpenAI service infrastructure through Azure OpenAI Service, which is integrated into Copilot Studio. This includes:

• Model hosting and management
• Inference endpoints
• Scaling and availability
• Security and compliance

Reference: What is Azure OpenAI Service?

2. Model Selection and Configuration

As you noted, at the global level, you can only select models through the generative AI settings. The underlying infrastructure is completely managed by Microsoft.

Deployment Channels and Connectivity

Teams Deployment

When your agent is deployed to Microsoft Teams:

• The inference endpoint uses Microsoft's managed Azure OpenAI Service
• Connectivity goes through Microsoft's secure infrastructure
• No external OpenAI API calls are made directly

Website Deployment

For website deployments:

• Same managed Azure OpenAI Service is used
• Requests are routed through Microsoft's secure channels
• No direct connectivity issues with OpenAI's public endpoints

Reference: Publish your copilot

Architecture and Security

Microsoft 365/Power Platform Integration

You are absolutely correct that Microsoft 365 and Power Platform wrap the entire infrastructure:

1. Security: All communications are encrypted and go through Microsoft's secure channels
2. Compliance: Adheres to Microsoft's enterprise compliance standards
3. Connectivity: No need to worry about external connectivity issues to OpenAI

Architecture Overview

While detailed architecture diagrams aren't publicly available for security reasons, the flow is:

• Copilot Studio → Azure OpenAI Service → Managed OpenAI Models

References:

• Security and governance in Copilot Studio
• Azure OpenAI Service security

Key Benefits of the Managed Approach

1. No Infrastructure Management: Microsoft handles all OpenAI service management
2. Enterprise Security: Built-in security and compliance
3. Reliable Connectivity: No dependency on external OpenAI API availability
4. Integrated Billing: Included in your Microsoft licensing

Additional Resources

• Microsoft Copilot Studio documentation
• Power Platform security and governance

In summary, you don't need to worry about connectivity or security issues as Microsoft manages the entire OpenAI service stack through their enterprise-grade infrastructure.

Best regards,

Karan Shewale.

*************************************************************************  

If the response is helpful, please click "Accept Answer" and upvote it. You can share your feedback via Microsoft Teams Developer Feedback link. Click here to escalate. 

Answer 2

Microsoft Q&A(en-us) does not support Power Platform.
For questions about Power Platform (PowerApps, PowerAutomate, etc.), please ask in the official dedicated community below.
Power Platform Community
　├ Copilot Studio
　├ Power Apps
　├ Power Automate
　│　└ Power Automate Desktop
　└ Power Pages