Evaluate agents in Microsoft Copilot Studio (GitHub Copilot)
Intermediate
Developer
App Maker
Microsoft Copilot Studio
Use repeatable evaluations in Microsoft Copilot Studio to measure whether each change to an agent powered by the GitHub Copilot harness improves or degrades its quality, and to decide whether to move the agent to a new AI model.
Learning objectives
After completing this module, you'll be able to:
- Create a test set that represents an agent's real scenarios.
- Run an evaluation on the Evaluate tab.
- Verify agent behaviors that the General quality test method doesn't check, such as tool use.
- Triage failed test cases to a root cause.
- Validate an AI model migration by comparing evaluation results against acceptance gates.
Prerequisites
- Experience building an agent in Microsoft Copilot Studio with the GitHub Copilot harness, including instructions, knowledge sources, and tools.
- Experience testing an agent on the Preview tab and reading its activity trace.
- Understanding of the difference between interactive testing and repeatable evaluation.
- Awareness that building, testing, and evaluating agents consume Copilot Credits.