Skip to main content

This browser is no longer supported.

Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.

Download Microsoft Edge More info about Internet Explorer and Microsoft Edge
microsoft-foundry.rest.security.o-auth2-auth

Fine-tuning

Validate a grader

POST /openai/v1/fine_tuning/alpha/graders/validate

Validates a grader.

Request Body

The request body.

Content-Type: application/json
grader OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderMulti required

The grader used for the fine-tuning job.

One of the following:
type enum required

The object type, which is always string_check.

string_check
name string required

The name of the grader.

input string required

The input text. This may include template strings.

reference string required

The reference text. This may include template strings.

operation enum required

The string check operation to perform. One of eq, ne, like, or ilike.

eqnelikeilike
type enum required

The type of grader.

text_similarity
name string required

The name of the grader.

input string required

The text being graded.

reference string required

The text being graded against.

evaluation_metric enum required

The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.

cosinefuzzy_matchbleugleumeteorrouge_1rouge_2rouge_3rouge_4rouge_5rouge_l
type enum required

The object type, which is always python.

python
name string required

The name of the grader.

source string required

The source code of the python script.

image_tag string

The image tag to use for the python script.

type enum required

The object type, which is always score_model.

score_model
name string required

The name of the grader.

model string required

The model to use for the evaluation.

sampling_params OpenAI.EvalGraderScoreModelSamplingParams
seed integer | null
top_p number | null
temperature number | null
max_completions_tokens integer | null
reasoning_effort string | enum | null

Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.

Any of the following:
string
enum
noneminimallowmediumhighxhighmax
input OpenAI.EvalItem[] required

The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.

range OpenAI.numeric[]

The service requires two numbers for the score range. Defaults to [0, 1].

type enum required

The object type, which is always multi.

multi
name string required

The name of the grader.

graders object required
[key: string] OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderLabelModel
Any of the following:
type enum required

The object type, which is always string_check.

string_check
name string required

The name of the grader.

input string required

The input text. This may include template strings.

reference string required

The reference text. This may include template strings.

operation enum required

The string check operation to perform. One of eq, ne, like, or ilike.

eqnelikeilike
type enum required

The type of grader.

text_similarity
name string required

The name of the grader.

input string required

The text being graded.

reference string required

The text being graded against.

evaluation_metric enum required

The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.

cosinefuzzy_matchbleugleumeteorrouge_1rouge_2rouge_3rouge_4rouge_5rouge_l
type enum required

The object type, which is always python.

python
name string required

The name of the grader.

source string required

The source code of the python script.

image_tag string

The image tag to use for the python script.

type enum required

The object type, which is always score_model.

score_model
name string required

The name of the grader.

model string required

The model to use for the evaluation.

sampling_params OpenAI.EvalGraderScoreModelSamplingParams
seed integer | null
top_p number | null
temperature number | null
max_completions_tokens integer | null
reasoning_effort string | enum | null

Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.

Any of the following:
string
enum
noneminimallowmediumhighxhighmax
input OpenAI.EvalItem[] required

The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.

range OpenAI.numeric[]

The service requires two numbers for the score range. Defaults to [0, 1].

type enum required

The object type, which is always label_model.

label_model
name string required

The name of the grader.

model string required

The model to use for the evaluation. Must support structured outputs.

input OpenAI.EvalItem[] required
labels string[] required

The labels to assign to each item in the evaluation.

passing_labels string[] required

The labels that indicate a passing result. Must be a subset of labels.

calculate_output string required

A formula to calculate the output based on grader results.

Responses

The request has succeeded.

Body

Content-Type: application/json
grader OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderMulti

The grader used for the fine-tuning job.

One of the following:
type enum required

The object type, which is always string_check.

string_check
name string required

The name of the grader.

input string required

The input text. This may include template strings.

reference string required

The reference text. This may include template strings.

operation enum required

The string check operation to perform. One of eq, ne, like, or ilike.

eqnelikeilike
type enum required

The type of grader.

text_similarity
name string required

The name of the grader.

input string required

The text being graded.

reference string required

The text being graded against.

evaluation_metric enum required

The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.

cosinefuzzy_matchbleugleumeteorrouge_1rouge_2rouge_3rouge_4rouge_5rouge_l
type enum required

The object type, which is always python.

python
name string required

The name of the grader.

source string required

The source code of the python script.

image_tag string

The image tag to use for the python script.

type enum required

The object type, which is always score_model.

score_model
name string required

The name of the grader.

model string required

The model to use for the evaluation.

sampling_params OpenAI.EvalGraderScoreModelSamplingParams
seed integer | null
top_p number | null
temperature number | null
max_completions_tokens integer | null
reasoning_effort string | enum | null

Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.

Any of the following:
string
enum
noneminimallowmediumhighxhighmax
input OpenAI.EvalItem[] required

The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.

range OpenAI.numeric[]

The service requires two numbers for the score range. Defaults to [0, 1].

type enum required

The object type, which is always multi.

multi
name string required

The name of the grader.

graders object required
[key: string] OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderLabelModel
Any of the following:
type enum required

The object type, which is always string_check.

string_check
name string required

The name of the grader.

input string required

The input text. This may include template strings.

reference string required

The reference text. This may include template strings.

operation enum required

The string check operation to perform. One of eq, ne, like, or ilike.

eqnelikeilike
type enum required

The type of grader.

text_similarity
name string required

The name of the grader.

input string required

The text being graded.

reference string required

The text being graded against.

evaluation_metric enum required

The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.

cosinefuzzy_matchbleugleumeteorrouge_1rouge_2rouge_3rouge_4rouge_5rouge_l
type enum required

The object type, which is always python.

python
name string required

The name of the grader.

source string required

The source code of the python script.

image_tag string

The image tag to use for the python script.

type enum required

The object type, which is always score_model.

score_model
name string required

The name of the grader.

model string required

The model to use for the evaluation.

sampling_params OpenAI.EvalGraderScoreModelSamplingParams
seed integer | null
top_p number | null
temperature number | null
max_completions_tokens integer | null
reasoning_effort string | enum | null

Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.

Any of the following:
string
enum
noneminimallowmediumhighxhighmax
input OpenAI.EvalItem[] required

The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.

range OpenAI.numeric[]

The service requires two numbers for the score range. Defaults to [0, 1].

type enum required

The object type, which is always label_model.

label_model
name string required

The name of the grader.

model string required

The model to use for the evaluation. Must support structured outputs.

input OpenAI.EvalItem[] required
labels string[] required

The labels to assign to each item in the evaluation.

passing_labels string[] required

The labels that indicate a passing result. Must be a subset of labels.

calculate_output string required

A formula to calculate the output based on grader results.

Client error

Body

Content-Type: application/json
error OpenAI.Error required
code string | null required
message string required
param string | null
type string
misalignment OpenAI.MisalignmentErrorDetailsResource
review_target string | null
error_type string | enum
Any of the following:
string
enum
potentially_unintended_data_transferpotentially_unintended_data_accesspotentially_unintended_destructive_activityother
detailed_explanation string

The public explanation for this block.

steer OpenAI._MisalignmentSteer
message string required

The public continuation instruction.

details OpenAI.Error[]
additionalInfo object
debugInfo object

Server error

Body

Content-Type: application/json
error OpenAI.Error required
code string | null required
message string required
param string | null
type string
misalignment OpenAI.MisalignmentErrorDetailsResource
review_target string | null
error_type string | enum
Any of the following:
string
enum
potentially_unintended_data_transferpotentially_unintended_data_accesspotentially_unintended_destructive_activityother
detailed_explanation string

The public explanation for this block.

steer OpenAI._MisalignmentSteer
message string required

The public continuation instruction.

details OpenAI.Error[]
additionalInfo object
debugInfo object

Request

curl https://api.openai.com/v1/fine_tuning/alpha/graders/validate \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "grader": {
      "type": "string_check",
      "name": "Example string check grader",
      "input": "{{sample.output_text}}",
      "reference": "{{item.label}}",
      "operation": "eq"
    }
  }'
import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env['OPENAI_API_KEY'], // This is the default and can be omitted
});

const response = await client.fineTuning.alpha.graders.validate({
  grader: {
    input: 'input',
    name: 'name',
    operation: 'eq',
    reference: 'reference',
    type: 'string_check',
  },
});

console.log(response.grader);
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("OPENAI_API_KEY"),  # This is the default and can be omitted
)
response = client.fine_tuning.alpha.graders.validate(
    grader={
        "input": "input",
        "name": "name",
        "operation": "eq",
        "reference": "reference",
        "type": "string_check",
    },
)
print(response.grader)
package main

import (
    "context"
    "fmt"

    "github.com/openai/openai-go"
    "github.com/openai/openai-go/option"
)

func main() {
    client := openai.NewClient(
        option.WithAPIKey("My API Key"),
    )
    response, err := client.FineTuning.Alpha.Graders.Validate(context.TODO(), openai.FineTuningAlphaGraderValidateParams{
        Grader: openai.FineTuningAlphaGraderValidateParamsGraderUnion{
            OfStringCheckGrader: &openai.StringCheckGraderParam{
                Input:     "input",
                Name:      "name",
                Operation: openai.StringCheckGraderOperationEq,
                Reference: "reference",
            },
        },
    })
    if err != nil {
        panic(err.Error())
    }
    fmt.Printf("%+v
", response.Grader)
}
package com.openai.example;

import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.finetuning.alpha.graders.GraderValidateParams;
import com.openai.models.finetuning.alpha.graders.GraderValidateResponse;
import com.openai.models.graders.gradermodels.StringCheckGrader;

public final class Main {
    private Main() {}

    public static void main(String[] args) {
        OpenAIClient client = OpenAIOkHttpClient.fromEnv();

        GraderValidateParams params = GraderValidateParams.builder()
            .grader(StringCheckGrader.builder()
                .input("input")
                .name("name")
                .operation(StringCheckGrader.Operation.EQ)
                .reference("reference")
                .build())
            .build();
        GraderValidateResponse response = client.fineTuning().alpha().graders().validate(params);
    }
}
require "openai"

openai = OpenAI::Client.new(api_key: "My API Key")

response = openai.fine_tuning.alpha.graders.validate(
  grader: {input: "input", name: "name", operation: :eq, reference: "reference", type: :string_check}
)

puts(response)

Response

{
  "grader": {
    "type": "string_check",
    "name": "Example string check grader",
    "input": "{{sample.output_text}}",
    "reference": "{{item.label}}",
    "operation": "eq"
  }
}
en-us
Your Privacy Choices
  • AI Disclaimer
  • Previous Versions
  • Blog
  • Contribute
  • Privacy
  • Consumer Health Privacy
  • Terms of Use
  • Trademarks
  • © Microsoft 2026