Tools and connectors in Frontier Tuning

A tool in Frontier Tuning is an action an agent can take in a tuning environment. Examples include searching mail, reading a SharePoint file, or looking up an issue. A connector groups tools for a service. You add, connect, and configure tools one connector at a time.

As an agent creator, consider the data tools use and the changes they can make. Tool behavior also varies across inference, evaluation, and training.

Why tool choice matters

A tuning environment uses tools in three stages:

Stage What happens
Inference. The agent or tuned model handles requests, for example in chat.
Evaluation. The agent completes sample tasks, and responses are assessed against your evaluation criteria.
Training. The model calls tools as it learns to complete your tasks.

The model learns to work with the tools it encounters during training, including their names, inputs, results, and errors. Differences in those tool details at inference can affect model behavior.

Training can also call tools far more often than inference. Service capacity and the effect of changes made by tools matter when you choose a connector.

Real, virtualized, and mock tools

The tool type determines whether calls use real services and how writes behave:

Tool type Data and behavior
Real tool. Calls the actual service and uses its data, for example Outlook Mail or SharePoint.
Virtualized tool. Reads real service data, but intercepts writes during evaluation and training. Reads after an intercepted write return the updated data, although the result might differ from what the real service would return.
Mock tool. Uses its own Microsoft-provided data instead of calling your services or using your data.

A read operation retrieves information. A write operation creates, changes, or deletes something, such as sending a message or updating a record.

The current v1 CLI can report a tool's kind as Mock or Live. A connector can contain mixed kinds, and an omitted kind means unknown. This metadata doesn't establish authorization, successful tool calls, or freedom from side effects.

Choose tools for each stage

Use real tools at inference for your business scenario. For evaluation and training, consider whether the service can handle the load, whether using the data meets your privacy requirements, and whether operations have side effects. For example, reading a message might mark it as read.

Tool behavior differs by stage. The following table isn't a list of supported connectors.

Tool behavior Inference Evaluation and training
Real tools with virtualization support. Calls the real service, including write operations. Reads real data and intercepts supported writes.
Real tools restricted to read-only evaluation and training. Calls the real service, including write operations. Runs tools marked as read-only. Calls to write tools, including tools without the read-only marking, fail.
Microsoft-provided mock tools. Not covered by this article. Uses supplied mock data for the sample scenario.

For evaluation or training with mock tools, choose tasks, samples, and evaluation criteria that fit the supplied data. Results from a mock scenario don't establish performance with your business data.

Connector connection lifecycle

Discovery, connector and tool enablement, authorization, stage selection, and runtime verification serve different purposes:

Activity Purpose
Discover a connector. Identify connectors available to your tenant, environment, and shared access.
Enable a connector. Enable its tools for use, subject to individual tool settings and catalog policy.
Enable an individual tool. Change one tool's setting in a supported Environment Connector. Both the connector and the tool must be enabled.
Authorize a connection. Connect to the service when the connector requires sign-in. Signing in to the service's website alone doesn't connect the connector.
Check the connector. Review saved enablement, sign-in state, tools, and stage labels. These aren't evidence of a successful tool call.
Choose stages. Select connector-wide labels: Inference, Evaluation, and Finetuning.
Verify a tool call. Inspect the selected tool's execution trace and result, not just a completed chat response.

In the current v1 CLI, a label update replaces the previous selection. No stored selection defaults to all three stages; an explicitly empty selection excludes the catalog connector from all three. A saved update isn't necessarily effective for every run yet: wait five minutes before starting a new training job. Running jobs keep their starting connector settings.

When a connection flow provides a sign-in link, open it in your regular browser and authorize access there. If a browser embedded in another app fails to connect, use your regular browser instead.

Direct custom MCP sources outside the connector catalog don't appear in catalog discovery or connector details in the current v1 CLI. For an already configured custom host, check service reachability, authentication, and behavior after restarts and under expected load. Execution traces and host logs provide evidence of its tool calls.

Mock tools and sample environments

Microsoft provides mock connectors with their own data. In the current v1 CLI, service-owned catalog mocks use the same discovery, inspection, and enablement commands as other catalog connectors. Mock connectors don't require sign-in to a real service.

The first enable saves a mock package for the environment. Disabling keeps the saved package and settings; deleting removes the environment's registration and settings, not the saved package. Deleting a registration isn't confirmation that test data was erased or the mock was reset.

A sample tuning environment can include skills, evaluation criteria, samples, and mock data for a scenario. Its contents depend on the template you choose. To explore the workflow, see Try Frontier Tuning with a sample tuning environment.

Check readiness and start small

Before evaluation or training, review these conditions:

  • The selected tools and data fit the tasks and evaluation criteria.
  • The service can handle the expected tool-call load.
  • Using real service data meets your privacy requirements.
  • Read and write operations have the expected side effects for the selected stage.

Saved settings and a connected sign-in state don't prove runtime callability. The current v1 quality check scores authored assets and starts or joins a service run; its report isn't proof that a selected tool can run successfully, and the CLI doesn't estimate its cost. Before larger runs, use an authorized, harmless short execution and inspect the intended tool's trace and result.

Start with a small evaluation and inspect the results for failed tool calls before running larger evaluations or training. Throttling and timeouts can affect evaluation scores and what the model learns.

Limitations

Connector settings and mock-data restrictions apply at different levels:

  • Stage labels apply to the whole catalog connector, including its tools and instances, subject to catalog policy and enablement. Labels don't restrict hosted mock bindings or built-in tools, so labels alone don't make a run mock-only.
  • The current v1 CLI supports individual tool toggles for supported Environment Connectors, not individual actions within built-in sources. Enabling a tool doesn't enable a disabled connector.
  • Whole-connector toggles aren't supported for connectors managed through instance configuration.
  • The current v1 CLI doesn't provide custom mock authoring or generation. Support for replacing Microsoft-provided mock data remains unconfirmed.