Frontier Tuning flow choices

A tuning flow in Frontier Tuning is an optional workflow for investigating an improvement to inference search, skill artifacts, or model weights. As an agent creator, you choose the mechanism that matches the gap in your evaluation evidence and compare its results against your requirements.

Quick, Light, and Deep describe different improvement choices, not a mandatory progression or single commands. A direct instruction edit, a standalone trial, or a directly requested training job can also fit your goal. If the current behavior meets your requirements, stopping without a change is a valid outcome.

Compare tuning flows

The flows differ in what they change and what evidence supports the decision.

Flow and target What changes What stays fixed for the test When to consider it and useful evidence
Quick: inference search. The strategy used to search for an answer. Skill instructions, model, and established evaluation criteria. Alternative answers might help; compare strategies on the same cases.
Light: artifact refinement. Instructions, evaluation criteria, or both. Model and other controls outside the chosen artifact change. Instructions or grading need revision; use reviewed development samples and inspect the proposed result.
Deep: model adaptation. Model weights through fine-tuning. Instructions, criteria, samples, and strategy in the model comparison. Reviewed eligible Training inputs support adaptation; compare the tuned model with its base model on held-out cases.

Choose from evaluation evidence

The observed gap determines the improvement to investigate, rather than the size or name of a flow.

Evidence Improvement to investigate
Missing instructions or a misunderstood procedure. Refine the instructions or edit them directly.
Incorrect or unclear grading. Revise the evaluation criteria.
A plausible benefit from alternative answers. Compare inference strategies.
Suitable training data and a task-specific quality goal. Test model fine-tuning.
Missing access, data, or tool capability. Investigate the operational gap before attributing failure to response quality.
Current behavior meets requirements. Keep it unchanged.

An access or tool failure isn't the same as an inadequate answer. Model training isn't a substitute for fixing missing access, data, or capabilities.

Quick tests whether a different strategy helps without changing instructions or training model weights. Simple runs a single execution; BestOfN selects among candidate responses; TreeSearch explores alternatives.

Establish and review evaluation criteria before the strategy comparison. Criteria generation saves immediately after authorization to generate; it isn't an unsaved proposal. Inspect the saved criteria before using them to score responses.

Trying an individual request can provide exploratory evidence without saved samples. That's distinct from a sample-based evaluation, which needs selected samples. Compare one model across strategies, or one strategy across models, rather than changing both and attributing the difference to one.

Light: artifact refinement

Light proposes changes to instructions, evaluation criteria, or both from reviewed eligible Training samples. Instructions guide the response; criteria define how it's assessed. An instruction-only refinement doesn't also revise the criteria.

Review the whole completed run and its chosen mode before application. Application saves the entire run's result, not individually selected suggestions. Criteria refinement replaces the entire criterion set. Inspect the saved instructions and criteria afterward.

This reviewed refinement differs from immediate-save criteria generation. When criteria change, scores measured under the old set aren't directly comparable: evaluate both the baseline and the candidate under the revised set.

Deep: model adaptation

Deep fine-tunes a base model using reviewed eligible Training inputs. Apply intended instruction, criterion, and sample changes before training. Reviewed, stable artifacts support an interpretable experiment; they don't imply that you must complete Light first.

Keep final validation cases and their derivatives out of generation, refinement, and training inputs. Sample labels alone don't enforce that separation.

Training-job success alone doesn't establish output readiness. In the inspected public v1 reference, evaluation readiness requires both job completion and successful model deployment-stage completion. Readiness doesn't publish the result to users. Later artifact edits don't retrain an already-produced model.

Quality, cost, and latency

Compare measured quality with latency and consumption, not an assumed benefit. BestOfN and TreeSearch can use more tokens and time without improving quality. Refinement runs, training, evaluation, runtime traffic, and model availability where applicable have different planning assumptions.

Larger criteria can take longer to grade. Evaluation token totals can include grading and orchestration: they aren't target-model-only usage, environment-wide consumption, or currency. Report latency for the measured operation and cases, not as a universal response-time promise. Missing usage is unmeasured, not free.

Fair comparisons

A useful comparison changes only the variable you're testing.

  • Keep prompts, instructions, criteria and graders, source and tool state, sample IDs and content, model/version, strategy options, and repeat conditions fixed except for that variable. Record drift or unavailable controls.
  • Compare base Simple with tuned Simple to test the model change. Compare tuned Simple with tuned TreeSearch to test search. These comparisons don't establish every model/strategy interaction.
  • Reserve untouched final cases for the final measurement; development evidence used to choose improvements isn't final evidence.
  • Record exact job IDs and content in place for each run. Inspect actual outputs, tool failures, and graded coverage alongside scores; job completion doesn't establish full grading.