Preview and refine an agent
The Preview tab provides an interactive environment for testing an agent before publication. Use it to establish a baseline, observe how the agent applies instructions and knowledge, and determine what to refine.
Build a baseline test set
Start with a small set of prompts that represent different behavior classes:
| Test type | Purpose | Benefits agent example |
|---|---|---|
| In scope | Confirm that the agent answers a supported request from approved knowledge | When can I change my medical plan? |
| Paraphrase | Confirm that different wording produces consistent behavior | Is there a time each year when I can switch health plans? |
| Ambiguous | Confirm that the agent requests information it needs | Tell me about coverage. |
| Missing information | Confirm that the agent doesn't invent unsupported details | What is the exact employee premium next year? |
| Out of scope | Confirm that the agent declines or redirects unrelated requests | Write a sales proposal for me. |
| Escalation | Confirm that the agent refers decisions to the correct destination | Am I eligible to add my partner to my plan? |
Record the expected result for each prompt before testing. An expected result describes the required behavior rather than exact wording. This approach allows natural variation while still identifying failures.
Observe the result
Open the agent and select Preview. Start a new conversation for each baseline prompt so earlier responses don't affect later tests. Review:
- Whether the response follows the agent's scope and tone.
- Whether factual claims match the approved knowledge.
- Whether the response acknowledges missing information.
- Whether the agent asks useful clarification questions.
- Whether boundaries and escalation instructions are followed.
- Whether source information or citations support the answer.
Use the activity information in Preview to check which knowledge source the agent used. A response that sounds correct can still be a problem if the agent uses the wrong source or doesn't follow its limits.
Refine one behavior at a time
When a test fails, identify the smallest instruction or source change that addresses the observed behavior. Make the rule more specific, save the agent, start a new test conversation, and rerun the same prompt. Then rerun nearby baseline cases to check that the change didn't introduce a regression.
For example, if the agent estimates eligibility, change Don't make eligibility decisions to Don't state or imply that an employee qualifies for a benefit. Explain only the published criteria and direct the employee to Human Resources for a personal decision. The revised instruction defines the prohibited behavior and the required alternative.
Preview testing is iterative. Test a prompt, compare the response with the expected result, make one focused change, and test the prompt again. Repeat this process until the agent responds consistently without affecting behavior that already works as expected.
For more information, see Test an agent.