This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
A maker needs an evaluation for an IT help agent. The help desk already keeps a list of real employee questions, and past incidents show that ticket status lookups sometimes fail. Which approach gives the test set the most representative coverage?
Generate a quick conversation set from the agent's configuration and run it as the only test set
Upload the real questions as a CSV file, and then write targeted conversations for ticket status lookups
Write a small set of carefully worded questions that cover only the most common policy topics
A maker writes an expected response for every conversation in a test set and runs the evaluation with the General quality test method. Several responses pass even though they differ from the expected responses. What explains this result?
The evaluation used a user profile that can't see the expected responses
Expected responses only affect the score after the evaluation runs a second time
General quality assesses relevance and completeness and doesn't compare responses to expected responses
A ticket status conversation passes General quality with a response that reads as relevant and complete. The maker needs to confirm that the agent actually ran the ticket lookup tool. What should the maker check?
The tool nodes in the Preview activity trace after sending the same user message
The Score field in the evaluation summary for the run
The credits used for the test case in the test case details
After an evaluation run, several ticket status test cases fail. The maker finds that the connection the lookup tool needs is expired for the account that ran the evaluation. How should the maker classify and handle these failures?
As an agent configuration issue, by rewriting the tool description and instructions
As a platform limitation, by documenting the failure and escalating it
As an evaluation setup issue, by restoring the connection and rerunning the evaluation
A team plans to move an agent to a candidate AI model. They defined acceptance gates and have a regression set ready. What should they do before they change the model?
Add new conversations to the regression set so the candidate faces more scenarios than the current model did
Run the regression set against the current model and export the test results
Select the candidate model on the Build tab of the production agent and save it
You must answer all questions before checking your work.
Was this page helpful?
Need help with this topic?
Want to try using Ask Learn to clarify or guide you through this topic?