Modify test cases within a test set

[This article is prerelease documentation and is subject to change.]

During or after creating a test set, you can edit the test cases by changing the wording of questions, choosing different test methods, or modifying the expected responses as needed. You can select multiple test cases to edit them in bulk by selecting the checkboxes beside each test case.

You have a choice of three test methods, also referred to as graders, to evaluate agent responses: quality, similarity, and text match. For more information about the different test methods, see Test methods.

Edit test cases within a test set

1. In your agent's Evaluation page, select the test set.

2. In the test set, select the test case you want to edit.

1. In the right pane, change the wording of a question by editing the text in the Question field.

1. Select the test method that you want to use.

• Quality:

  • Select General quality to evaluate the answer based on relevance, groundedness, and completeness.

  • Select Compare meaning to evaluate the answer based on how well it captures the meaning of the expected response. Under Passing score, you can set the threshold for what constitutes a passing score for an answer.

    In the Expected response box, provide the response against which the test method evaluates the agent's answer.

• Similarity: uses a Cosine similarity metric to assess how similar the agent's answer is to the wording and meaning of the expected response. It determines a score between 0 and 1, where 1 means it matches closely and 0 means it doesn't match at all. Under Passing score, you can set the threshold for what constitutes a passing score for an answer.

  In the Expected response box, provide the response against which the test method evaluates the agent's answer.

• Text match:

  • Select Exact match to evaluate the agent's answer against the expected response, where a passing score means the agent's answer exactly matched the defined expected response.

    In the Expected response box, provide the response against which the test method evaluates the agent's answer.

  • Select Keyword match to evaluate the agent's answer against the expected response, where a passing score means the agent's answer contained some of the words or phrases from the defined expected response.

    In the Expected response box, provide a phrase or keyword against which the test method evaluates the agent's answer. To add multiple keywords or phrases, select Add, select the operator and or or between the boxes, and provide the keyword or phrase.

    

1. Select Apply.

2. When you're finished with your changes, select Save to save your test set or Evaluate to run the test.

Add more test cases to a test set

While creating or editing a test set, you can add more test cases.

1. In your agent's Evaluation page, select the test set.

2. Generate or manually add more test cases.

1. Select and edit any new test cases to add details.