Copilot Studio Agent Experiencing High Response Latency

Question

Hi,

I am working with an agent in Microsoft Copilot Studio and am experiencing noticeable latency when interacting with the agent.

The agent takes several seconds to respond to user queries, even for relatively simple questions. The delay is affecting the overall user experience.

The agent configuration includes knowledge sources and generative responses, but I would like to understand whether the latency is related to the agent configuration, knowledge retrieval, model processing, or other Copilot Studio services.

I would like to understand:

1. What are the main factors that contribute to latency in Copilot Studio agents?
2. What would be considered normal/expected response latency?
3. How can I identify which part of the agent execution is causing the delay?
4. Does the number and size of knowledge sources affect overall agent response time?
5. Can topics, generative answers, actions, or tools increase latency?
6. Are there recommended ways to optimize a Copilot Studio agent for faster responses?
7. Is there any built-in monitoring or diagnostic capability to measure the latency of individual steps/components?

If anyone has experienced similar latency with Copilot Studio agents, I would appreciate any recommendations for troubleshooting and optimization.

Thanks!

Answer 1

Hello Sonalika •,
Latency in Copilot Studio can come from multiple stages of the request pipeline, including knowledge retrieval, generative AI processing, topic orchestration, actions/flows, external connectors, and tool execution. The number and size of knowledge sources can impact retrieval time, while generative answers, Power Automate flows, external APIs, and custom tools can add additional delay. Response times vary depending on complexity, but simple queries are typically faster than requests that require multiple retrieval and reasoning steps.

To troubleshoot, review conversation diagnostics, test the agent with and without knowledge sources, actions, or tools enabled, and compare response times to isolate the component introducing latency. Monitoring individual execution steps through analytics, run history, and diagnostic logs can help identify bottlenecks. Optimizing knowledge sources, reducing unnecessary actions, and minimizing external dependencies are common approaches to improving response performance.

Answer 2

The main latency contributors in Copilot Studio agents are:

1. External operations
   • Calls to external systems, connector invocations, and Power Automate flows can add noticeable delay.
   • Copilot Studio off-agent calls such as Power Automate are synchronous, so the conversation waits until the flow finishes.
2. Agent design
   • Placing multiple API calls or connector calls in a conversation path increases wait time.
   • Repeating the same retrieval instead of caching results in variables adds unnecessary latency.
3. Model choice
   • Model category affects speed:
     • General is optimized for speed and lowest latency.
     • Auto has variable latency.
     • Deep has the highest latency and credit consumption.
   • Generative orchestration can handle more complex queries, but it can introduce latency compared to classic NLU for specific intents.
4. Knowledge and generative features
   • Generative AI responses and topic orchestration are subject to quotas and limits.
   • The context does not provide a fixed rule that knowledge source count or size directly determines latency, but retrieval and grounding operations can add processing overhead.

What is considered normal latency is not defined as a single expected number in the available guidance. The documented guidance instead focuses on measuring latency by component and reducing avoidable delays.

To identify where the delay occurs, use these approaches:

1. Measure end-to-end response time
   • For Direct Line testing, track:
     • Generate Token
     • Start Conversation
     • Send Activity
     • Receive Activities/Get Activities
   • The user-perceived response time is the time between the user message and the last agent message for that turn.
2. Inspect Power Automate duration
   • Use the Power Automate dashboard to review request durations, success rates, and failure rates for flows called by the agent.
   • This is important because Copilot Studio itself does not provide insight into how long those off-agent operations take.
3. For real-time voice agents
   • In Application Insights, Duration_Ms on LlmInvocation measures caller-perceived latency as Time to First Audio Back.
   • For multi-step turns, group by correlationId and use the maximum Duration_Ms per turn.
   • Tool duration is reported separately on ToolExecution rows.

Yes, topics, generative answers, actions, and tools can increase latency:

• Topics and orchestration: generative orchestration can be slower than classic NLU.
• Actions/tools/connectors: each external call can add delay.
• Power Automate flows: can introduce considerable overhead.
• Multi-step interactions: each step can contribute to the total user-perceived wait.

Recommended optimization steps:

1. Reduce external call overhead
   • Minimize long off-agent operations.
   • If only a single lookup is needed, prefer the HTTP Request node over a Power Automate flow.
2. Place calls strategically
   • Avoid making users wait for multiple API or connector completions in the same path.
3. Cache retrieved data
   • Store reused information in variables instead of calling APIs repeatedly.
4. Choose the right model category
   • Use General for high-volume FAQ-style and lightweight scenarios where low latency matters.
   • Avoid moving simple, high-volume workloads to Deep, because latency can rise sharply.
5. Turn on express mode
   • This is listed as a performance best practice.
6. Set timeout for externally populated global variables
   • Configure the optional timeout so the agent does not hang longer than necessary when waiting on external context.
7. Improve perceived responsiveness
   • If an operation may take more than a couple of seconds, send a brief message indicating the agent is looking up information.

Built-in monitoring and diagnostics are limited by component:

• Power Automate dashboard: use for flow duration and reliability.
• Direct Line performance testing: use to measure request and response timing across conversation steps.
• Application Insights for real-time voice agents: use for latency and tool execution analysis.
• The available guidance does not describe a single built-in Copilot Studio view that breaks down latency for every individual internal step of a text agent turn.

A practical troubleshooting path is:

1. Check whether the slow turns involve Power Automate, connectors, or HTTP calls.
2. Replace simple flow-based lookups with an HTTP Request node where possible.
3. Cache repeated retrievals in variables.
4. Review whether the agent is using a higher-latency model category than needed.
5. If using generative orchestration for simple intent-driven scenarios, evaluate whether classic NLU is more appropriate.
6. Measure end-to-end response time with Direct Line, and for voice agents use Application Insights to isolate tool versus model time.

---

• Optimize agents to minimize latency
• Best practices for improving conversational agent performance
• Manage the AI model lifecycle for Copilot Studio agents
• Test conversational agents using Direct Line
• Monitor real-time voice agents by using Application Insights
• Monitor real-time voice agents by using Application Insights