Help Needed with Dataset Mapping in Microsoft Foundry UI

Question

Help Needed with Dataset Mapping in Microsoft Foundry UI

Ipsita Dutta 0

Please let me know how we can provide tool definitions and tool calls data in the dataset and map the same in Foundry UI for creating evaluations. I am not able to understand how we need to provide the data and map for agent testing User's image

SRILAKSHMI C 19,730 Reputation points Microsoft External Staff Moderator

2026-07-14T11:27:11.72+00:00
Hello @Ipsita Dutta

Thank you for reaching out to Microsoft Q&A.

From your screenshot, it appears you're configuring an evaluation in the Azure AI Foundry UI, and you're unsure how to populate and map the Tool calls and Tool definitions fields in your evaluation dataset.

Based on the available Microsoft documentation, here's what we can confirm.

What are Tool calls and Tool definitions?

These fields are used when evaluating tool-using agents that is, agents that invoke functions, APIs, MCP tools, or other external tools during execution. If your agent does not use any tools, these fields are optional and can be left unmapped.

Tool calls represent the actual tool invocations made by the agent while responding to a query.

Tool definitions represent the available tool metadata or schemas (such as function names, descriptions, and parameters) that the agent can use.

How does the mapping work?

The evaluation wizard is asking you to map dataset fields to evaluator inputs such as:

Query

Response

Context

Ground truth

Tool calls

Tool definitions

This means your dataset should contain fields or columns corresponding to these values. During the mapping step, you simply bind the dataset fields to the appropriate evaluator inputs for example:

{{sample.tool_calls}} → Tool calls

{{sample.tool_definitions}} → Tool definitions

The purpose of this mapping is to allow the evaluator to understand:

Which tools were available to the agent.

Which tool invocations the agent actually produced during execution.

Example structure

If your agent uses function calling, your dataset could conceptually include fields similar to the following.

Tool calls

[ { "id": "call_1", "type": "function", "function": { "name": "search_documents", "arguments": "{\"query\":\"Azure OpenAI pricing\"}" } } ]

Tool definitions

[ { "type": "function", "function": { "name": "search_documents", "description": "Search enterprise documents", "parameters": { "type": "object", "properties": { "query": { "type": "string" } }, "required": [ "query" ] } } } ]

The Azure AI Foundry documentation confirms that:

Tool calls are represented by a FunctionToolCall object, which contains:

name – the function/tool name

arguments – a JSON string containing the arguments generated by the model.

Evaluation datasets can be supplied as datasets stored in Azure Storage for example, uri_file or uri_folder using a dataUri, and the evaluation UI maps fields from those datasets into the evaluator inputs.

At this time, the available Microsoft documentation does not provide:

An end-to-end sample evaluation dataset containing tool_calls and tool_definitions.

The exact JSON schema expected specifically for the Tool definitions field in the evaluation dataset.

A step-by-step example showing how to construct and upload a dataset with these fields for the Foundry evaluation UI.

Please refer this

Microsoft Foundry (Dataset / Tool call related API components): https://learn.microsoft.com/rest/api/microsoft-foundry/aiproject?wt.mc_id=knowledgesearch_inproduct_azure-cxp-community-insider#components

I Hope this helps. Do let me know if you have any further queries.

Thank you!
SRILAKSHMI C 19,730 Reputation points Microsoft External Staff Moderator

2026-07-15T15:44:02.1066667+00:00

Hi @Ipsita Dutta

Did you get any chance to review the above response. Do let me know if you have any further queries.

Thank you!
SRILAKSHMI C 19,730 Reputation points Microsoft External Staff Moderator

2026-07-19T01:10:17.31+00:00

Hi @Ipsita Dutta,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Thank you!
Ipsita Dutta 0 Reputation points

2026-07-20T12:30:06.04+00:00

Hi @SRILAKSHMI C Thanks for the reply, I will check and let you know
SRILAKSHMI C 19,730 Reputation points Microsoft External Staff Moderator

2026-08-05T07:13:25.3533333+00:00

Hi @Ipsita Dutta

Were you able to check? Please let me know.

Thank you!

1 answer

Your answer

SRILAKSHMI C 19,730 Reputation points Microsoft External Staff Moderator

2026-07-15T15:44:02.1066667+00:00

Hi @Ipsita Dutta

Did you get any chance to review the above response. Do let me know if you have any further queries.

Thank you!
SRILAKSHMI C 19,730 Reputation points Microsoft External Staff Moderator

2026-07-19T01:10:17.31+00:00

Hi @Ipsita Dutta,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Thank you!
Ipsita Dutta 0 Reputation points

2026-07-20T12:30:06.04+00:00

Hi @SRILAKSHMI C Thanks for the reply, I will check and let you know
SRILAKSHMI C 19,730 Reputation points Microsoft External Staff Moderator

2026-08-05T07:13:25.3533333+00:00

Hi @Ipsita Dutta

Were you able to check? Please let me know.

Thank you!

Answer 1

For agent evaluation in Microsoft Foundry, the dataset must contain the fields that the selected evaluator expects, and those fields must be mapped in the UI.

For individual turn evaluations, these fields are supported:

query — required
response — required
context — optional
ground_truth — optional
tool_calls — optional
tool_definitions — optional

For conversation evaluations, these fields are required:

messages
tool_definitions

How to provide tool data in the dataset:

Add a tool_definitions field to each dataset row when the evaluator needs tool information.
Add a tool_calls field when the evaluator supports or requires tool call data.
The dataset field names can be any names, but the mapping must point to the correct column using {{item.field_name}} syntax.

How to map in Foundry UI:

In the evaluation creation flow, go to Configure field mapping.
If Foundry does not auto-map a field, it appears as Unassigned.
Use the dropdown for each required evaluator input and select the dataset column that contains that data.
Required fields marked with * must be assigned, or the evaluator fails.

Evaluator requirements for tool-related agent testing:

Tool Call Accuracy: requires either
- query, response, tool_definitions, or
- query, tool_calls, tool_definitions
Tool Selection: requires either
- query, response, tool_definitions, or
- query, tool_calls, tool_definitions
Tool Input Accuracy: requires query, response, tool_definitions
Tool Output Utilization: requires query, response, tool_definitions
Tool Call Success: requires response

Important detail for the UI shown in the screenshot:

{{sample.tool_calls}} and {{sample.tool_definitions}} are not dataset fields.
Dataset field mapping uses {{item.field_name}}.
{{sample.output_items}} is used for the agent’s structured output in evaluator configuration scenarios that need full interaction context.
{{sample.output_text}} is used when an evaluator expects plain text response.

Recommended dataset shape for agent testing:

{"query":"What's the weather in Seattle?","response":"The weather in Seattle is rainy, 14°C.","tool_definitions":[{"type":"function","function":{"name":"get_weather","description":"Get weather by city"}}],"tool_calls":[{"name":"get_weather","arguments":"{\"city\":\"Seattle\"}"}]}

If the dataset uses different column names, map them explicitly. Example:

dataset column question_text → query using {{item.question_text}}
dataset column answer_text → response using {{item.answer_text}}
dataset column tools → tool_definitions using {{item.tools}}
dataset column calls → tool_calls using {{item.calls}}

If testing multi-turn conversations, use:

messages
tool_definitions

If testing single-turn agent evaluators, use query and response at minimum, then add tool_definitions and tool_calls based on the evaluator selected.