Diagnose issues from session evidence
Choosing the right fix for a published agent depends on finding the cause behind a signal on the Monitor tab. In this unit, you learn how to trace a symptom through session transcripts and the activity trace, tell a quality defect from a runtime failure, and choose where to make the fix.
Follow a signal into the session transcript
Diagnosis narrows the signal you chose on the Monitor tab to specific sessions and responses. Filter the sessions list by Failed or Rejected status to find the problematic interactions. The Monitor tab also helps you check tool connection status and authentication errors.
Treat each negative reaction as a lead to investigate, not as a score. The thumbs-down shows that a user rated a response poorly. The comment, when present, can explain what the user expected the agent to do.
Select a session to open its transcript. Read it for three facts that the rest of the diagnosis depends on:
- The request: The exact user message that led to the problem.
- What the agent used: The tools it invoked and the knowledge sources it drew on.
- The outcome: The response the user rated poorly, or the part of the conversation that failed.
One transcript shows what happened in one conversation. Compare several failed sessions for a shared tool or a shared kind of request. Patterns in failed sessions can indicate a configuration problem.
Read the activity trace for a reproduced request
A transcript records what the agent did in production. To see how the agent processed each step, reproduce the request and read the activity trace. The activity trace is a chain-of-thought view that shows how your agent processes each message, including the knowledge sources it consults, the tools it invokes, and the reasoning steps it follows. The activity trace is available on the Preview tab, so reproduce the request there to see each step.
To reproduce a request:
- Open your agent in Copilot Studio and select the Preview tab.
- Make sure the End user preview toggle is off. This default maker view shows the activity trace alongside the chat.
- Select New chat to start a conversation with no prior context.
- Send the same request that the user sent in the production transcript.
- Select any node in the trace to expand its details.
To return to an earlier test conversation, select History in the preview header. In maker view, a past conversation shows the activity trace for each message.
Each step in the trace appears as a node. Read the node types in four groups:
- Messages: User message and Agent response nodes show what you sent and what the agent generated.
- Grounding: A Knowledge node shows the knowledge sources the agent searched, with citation links when available. Select a source to see the specific content retrieved.
- Actions: Ran action, Tool, Connector, Flow, and Skill nodes show the tools, connectors, agent flows, and skills the agent invoked. A Ran action card shows the connector, the filled input parameters, and the delivery duration.
- Outcome: An Error node marks a problem during processing. The Complete indicator, a green checkmark at the bottom of the trace, shows that all tool calls finished.
Node details can include the input the agent sent to the step, the output returned, processing time and metrics, and error details if the step failed.
Compare the reproduction with the production transcript. When the trace shows the same tool, inputs, or error, the reproduction confirms where the problem occurs. When the symptom doesn't appear, keep the production transcript as your main evidence.
Separate quality defects from runtime failures
The transcript and the trace together show which kind of problem you have. A quality defect means the agent ran but gave a wrong or incomplete response. A runtime failure means a step failed while the agent processed the request.
Evidence points to a quality defect when:
- The session completes, and the trace shows no Error node.
- The Knowledge node shows the wrong source, no source, or retrieved content that doesn't match the request.
- The agent invoked the wrong tool, skipped the expected tool, or passed the wrong inputs.
- A negative reaction's comment describes an answer the user expected but didn't get.
Evidence points to a runtime failure when:
- The session status is Failed, or the trace contains an Error node.
- The error details show a common cause: an authentication failure for a tool or connector, a knowledge source access error, a timeout on a long-running tool call, or a content moderation flag.
Read the whole trace before you classify the problem. The GitHub Copilot harness can adjust when a step fails, so a response on its own doesn't show that every step succeeded.
For example, suppose users rate answers from a status-lookup tool poorly. If the Tool node shows that the agent passed a customer name where the tool expects an order number, the cause is a quality defect in the tool's inputs. If the trace shows an authentication failure for that tool instead, the cause is a runtime failure in the tool's connection. The symptom is the same, and the fixes are in different places.
Match the fix to the cause
Each finding points to a specific place in the agent to change. The following table pairs common findings with where to fix them.
| Finding in the trace | Where to fix | What to change |
|---|---|---|
| Wrong source, no source, or irrelevant retrieved content | Knowledge configuration on the Build tab | The knowledge sources the agent can use |
| Knowledge source access error | Knowledge configuration on the Build tab, or Settings | The source that failed, or the setting that blocks access to it |
| Expected tool not invoked, or wrong tool invoked | Tool details and the agent's instructions on the Build tab | The tool's name and description, and how the instructions guide tool use |
| Tool invoked with wrong inputs, or incorrect results | The Inputs panel in Tool details | The values passed to the tool, its parameters, or its selected actions |
| Authentication failure or connection error | Tool details on the Build tab | Expired authentication credentials |
| Content moderation flag | The AI & behavior tab in Settings | The Moderation level, only when the flagged request is within the agent's intended scope |
A tool's name and description guide when the agent invokes it. A vague description can keep the agent from selecting the tool. The agent's instructions also influence when tools are used. For a connection error, check that the service is available and that the tool's credentials are current. Changes in Settings take effect after you save and publish the agent.
To reach the right component quickly, select any source, tool, or component in the trace. Copilot Studio opens the matching component on the Build tab. Make your change, return to the Preview tab, and send the same request again. Then check the node where the problem appeared. A fix that matches the cause changes that node, for example with the right source in the Knowledge node or no Error node for the tool.
When a knowledge source returns outdated content, update the source content or the knowledge configuration. Matching each fix to its evidence keeps each change small and easy to retest.
Reflect: Pick one tool or knowledge source that an agent you manage or plan to publish depends on. If users rate its answers poorly, what would you look for in the transcript and the activity trace to tell a quality defect from a runtime failure?