This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 3%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EF%3C/text%3E%3C/svg%3E)
Dear Microsoft community,
I am following step by step this official tutorial: https://learn.microsoft.com/en-us/azure/ai-studio/tutorials/deploy-copilot-ai-studio#customize-prompt-flow-with-multiple-data-sources
When I run the evaluation I get a result with the questions and answers I prepared on the test dataset. This is right:
However, the result with evaluation metrics is wrong. The questions of the test dataset are not considered. Instead I get always the same default question and answer pair:
How can I solve this problem? I used the custom evaluation instead of the built-in evaluation. When I use the built-in evaluation I get this other issue that I posted here https://learn.microsoft.com/en-us/answers/questions/1598940/error-flow-runtime-not-found-llms-built-in-evaluat.
Thanks in advance for your kind support.
@ferrand I just ran through the document to setup my prompt flow and ran built in evaluation, which seems to have run as expected except for some errors in metrics since some of them timed out with token limit.
In your case I think you used custom evaluation. I do not have a customized evaluation to select in this case to check for similar behavior, I think you can check the data tab under components of your project to check if your test set of questions/answers(truth) are correctly uploaded since you are seeing only one question for all indexes as per screen shot.
Also, you can check the logs tab of the evaluation details to check if there is any timeout in processing all the metrics. This tab helped me to understand failures in some of the metrics in my case. I hope this helps!!
@romungi-MSFT I changed the model to GPT4-32k and used a jsonl format instead of csv. I made it. Thanks a lot for your support.