Prompt Flow Evaluation Failed with "The input for batch run is incorrect."

Question

Prompt Flow Evaluation Failed with "The input for batch run is incorrect."

杉山慎伍 0

Hello, I am currently doing research on Prompt Flow.

https://learn.microsoft.com/ja-jp/azure/machine-learning/prompt-flow/how-to-bulk-test-evaluate-flow?view=azureml-api-2

When trying to use the above manual to evaluate the performance of a flow using built-in flows, I encountered an error that the batch execution input was incorrect.

Clone the built-in flow "Web Classification".
Open the "Evaluation" option in the upper right corner of the edit screen and specify ${data.url} for the url input.
Select the built-in "Classification Accuracy Evaluation" in the "Evaluation Run" selection. 4) Select the "Evaluation Input Mapping" option.
Specify "groundtruth":${data.category} and "prediction":${run.outputs.category} for the evaluation input mapping.
Job execution of Web Classification-<date>-variant_0-<date>-Classification Accuracy Evaluation-<date> fails.

The error message is as follows:

UserError: The input for batch run is incorrect. Couldn't find these mapping relations: ${data.category}.

The test data is a csv file in the following format.

url, category, evidence

The link listed in the error message was 404 and no solution could be found. Could you please tell me which part is wrong?

Log file is here: Web Classification-10-26-2023-10-05-00-variant_0-202311060519-Classification Accuracy Evaluation-202311060519__.log.

Ramr-msft 17,826 Reputation points

2023-11-06T16:05:10.4633333+00:00

杉山慎伍 Thanks for the question, Can you please share the input file to check. Here is the blog for prompt flow.

https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/harness-the-power-of-large-language-models-with-azure-machine/ba-p/3828459
杉山慎伍 0 Reputation points

2023-11-07T00:48:00.7766667+00:00

classifytest.csv.txt
Here is test file but original format is csv.
杉山慎伍 0 Reputation points

2023-11-09T01:16:25.9366667+00:00

@Ramr-msft I uploaded the dataset. Is not the solution found yet?
Ramr-msft 17,826 Reputation points

2023-11-23T07:20:01.3633333+00:00

Thanks for the details, The error message you’re seeing, “UserError: The input for batch run is incorrect. Couldn’t find these mapping relations: ${data.category}”, suggests that there might be an issue with the way the input data is being mapped in the “Evaluation Input Mapping” step.

You mentioned that you’re specifying “groundtruth”:data.categoryand"prediction":{run.outputs.category} for the evaluation input mapping. This error could occur if the ${data.category} variable is not found in the input data.

Here are a few things you could check:

CSV File Format: Make sure that your CSV file is correctly formatted and includes a ‘category’ column since you’re using ${data.category} in your input mapping. The CSV file should have the following format: url, category, evidence.

Variable Names: Ensure that the variable names you’re using in your input mapping (data.categoryand{run.outputs.category}) match exactly with the column names in your CSV file and the output of your flow respectively.

Flow Outputs: Check the outputs of your flow to make sure that ‘category’ is an output of the flow. You can do this by running a test on a single row of data and checking the outputs.

Your answer

Ramr-msft 17,826 Reputation points

2023-11-06T16:05:10.4633333+00:00

杉山慎伍 Thanks for the question, Can you please share the input file to check. Here is the blog for prompt flow.

https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/harness-the-power-of-large-language-models-with-azure-machine/ba-p/3828459
杉山慎伍 0 Reputation points

2023-11-07T00:48:00.7766667+00:00

classifytest.csv.txt
Here is test file but original format is csv.
杉山慎伍 0 Reputation points

2023-11-09T01:16:25.9366667+00:00

@Ramr-msft I uploaded the dataset. Is not the solution found yet?
Ramr-msft 17,826 Reputation points

2023-11-23T07:20:01.3633333+00:00

Thanks for the details, The error message you’re seeing, “UserError: The input for batch run is incorrect. Couldn’t find these mapping relations: ${data.category}”, suggests that there might be an issue with the way the input data is being mapped in the “Evaluation Input Mapping” step.

You mentioned that you’re specifying “groundtruth”:data.categoryand"prediction":{run.outputs.category} for the evaluation input mapping. This error could occur if the ${data.category} variable is not found in the input data.

Here are a few things you could check:

CSV File Format: Make sure that your CSV file is correctly formatted and includes a ‘category’ column since you’re using ${data.category} in your input mapping. The CSV file should have the following format: url, category, evidence.

Variable Names: Ensure that the variable names you’re using in your input mapping (data.categoryand{run.outputs.category}) match exactly with the column names in your CSV file and the output of your flow respectively.

Flow Outputs: Check the outputs of your flow to make sure that ‘category’ is an output of the flow. You can do this by running a test on a single row of data and checking the outputs.

Share via

Prompt Flow Evaluation Failed with "The input for batch run is incorrect."

Your answer