Fine-tuning
Validate a grader
Validates a grader.
Anforderungstext
The request body.
grader
OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderMulti
Erforderlich
The grader used for the fine-tuning job.
type
enum
Erforderlich
The object type, which is always string_check.
name
string
Erforderlich
The name of the grader.
input
string
Erforderlich
The input text. This may include template strings.
reference
string
Erforderlich
The reference text. This may include template strings.
operation
enum
Erforderlich
The string check operation to perform. One of eq, ne, like, or ilike.
type
enum
Erforderlich
The type of grader.
name
string
Erforderlich
The name of the grader.
input
string
Erforderlich
The text being graded.
reference
string
Erforderlich
The text being graded against.
evaluation_metric
enum
Erforderlich
The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.
type
enum
Erforderlich
The object type, which is always python.
name
string
Erforderlich
The name of the grader.
source
string
Erforderlich
The source code of the python script.
image_tag
string
The image tag to use for the python script.
type
enum
Erforderlich
The object type, which is always score_model.
name
string
Erforderlich
The name of the grader.
model
string
Erforderlich
The model to use for the evaluation.
sampling_params
OpenAI.EvalGraderScoreModelSamplingParams
reasoning_effort
string | enum | null
Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.
enum
input
OpenAI.EvalItem[]
Erforderlich
The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
range
OpenAI.numeric[]
The service requires two numbers for the score range. Defaults to [0, 1].
type
enum
Erforderlich
The object type, which is always multi.
name
string
Erforderlich
The name of the grader.
graders
object
Erforderlich
[key: string]
OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderLabelModel
type
enum
Erforderlich
The object type, which is always string_check.
name
string
Erforderlich
The name of the grader.
input
string
Erforderlich
The input text. This may include template strings.
reference
string
Erforderlich
The reference text. This may include template strings.
operation
enum
Erforderlich
The string check operation to perform. One of eq, ne, like, or ilike.
type
enum
Erforderlich
The type of grader.
name
string
Erforderlich
The name of the grader.
input
string
Erforderlich
The text being graded.
reference
string
Erforderlich
The text being graded against.
evaluation_metric
enum
Erforderlich
The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.
type
enum
Erforderlich
The object type, which is always python.
name
string
Erforderlich
The name of the grader.
source
string
Erforderlich
The source code of the python script.
image_tag
string
The image tag to use for the python script.
type
enum
Erforderlich
The object type, which is always score_model.
name
string
Erforderlich
The name of the grader.
model
string
Erforderlich
The model to use for the evaluation.
sampling_params
OpenAI.EvalGraderScoreModelSamplingParams
reasoning_effort
string | enum | null
Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.
enum
input
OpenAI.EvalItem[]
Erforderlich
The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
range
OpenAI.numeric[]
The service requires two numbers for the score range. Defaults to [0, 1].
type
enum
Erforderlich
The object type, which is always label_model.
name
string
Erforderlich
The name of the grader.
model
string
Erforderlich
The model to use for the evaluation. Must support structured outputs.
labels
string[]
Erforderlich
The labels to assign to each item in the evaluation.
passing_labels
string[]
Erforderlich
The labels that indicate a passing result. Must be a subset of labels.
calculate_output
string
Erforderlich
A formula to calculate the output based on grader results.
Antworten
The request has succeeded.
Text
grader
OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderMulti
The grader used for the fine-tuning job.
type
enum
Erforderlich
The object type, which is always string_check.
name
string
Erforderlich
The name of the grader.
input
string
Erforderlich
The input text. This may include template strings.
reference
string
Erforderlich
The reference text. This may include template strings.
operation
enum
Erforderlich
The string check operation to perform. One of eq, ne, like, or ilike.
type
enum
Erforderlich
The type of grader.
name
string
Erforderlich
The name of the grader.
input
string
Erforderlich
The text being graded.
reference
string
Erforderlich
The text being graded against.
evaluation_metric
enum
Erforderlich
The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.
type
enum
Erforderlich
The object type, which is always python.
name
string
Erforderlich
The name of the grader.
source
string
Erforderlich
The source code of the python script.
image_tag
string
The image tag to use for the python script.
type
enum
Erforderlich
The object type, which is always score_model.
name
string
Erforderlich
The name of the grader.
model
string
Erforderlich
The model to use for the evaluation.
sampling_params
OpenAI.EvalGraderScoreModelSamplingParams
reasoning_effort
string | enum | null
Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.
enum
input
OpenAI.EvalItem[]
Erforderlich
The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
range
OpenAI.numeric[]
The service requires two numbers for the score range. Defaults to [0, 1].
type
enum
Erforderlich
The object type, which is always multi.
name
string
Erforderlich
The name of the grader.
graders
object
Erforderlich
[key: string]
OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderLabelModel
type
enum
Erforderlich
The object type, which is always string_check.
name
string
Erforderlich
The name of the grader.
input
string
Erforderlich
The input text. This may include template strings.
reference
string
Erforderlich
The reference text. This may include template strings.
operation
enum
Erforderlich
The string check operation to perform. One of eq, ne, like, or ilike.
type
enum
Erforderlich
The type of grader.
name
string
Erforderlich
The name of the grader.
input
string
Erforderlich
The text being graded.
reference
string
Erforderlich
The text being graded against.
evaluation_metric
enum
Erforderlich
The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.
type
enum
Erforderlich
The object type, which is always python.
name
string
Erforderlich
The name of the grader.
source
string
Erforderlich
The source code of the python script.
image_tag
string
The image tag to use for the python script.
type
enum
Erforderlich
The object type, which is always score_model.
name
string
Erforderlich
The name of the grader.
model
string
Erforderlich
The model to use for the evaluation.
sampling_params
OpenAI.EvalGraderScoreModelSamplingParams
reasoning_effort
string | enum | null
Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.
enum
input
OpenAI.EvalItem[]
Erforderlich
The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
range
OpenAI.numeric[]
The service requires two numbers for the score range. Defaults to [0, 1].
type
enum
Erforderlich
The object type, which is always label_model.
name
string
Erforderlich
The name of the grader.
model
string
Erforderlich
The model to use for the evaluation. Must support structured outputs.
labels
string[]
Erforderlich
The labels to assign to each item in the evaluation.
passing_labels
string[]
Erforderlich
The labels that indicate a passing result. Must be a subset of labels.
calculate_output
string
Erforderlich
A formula to calculate the output based on grader results.
Anfordern
curl https://api.openai.com/v1/fine_tuning/alpha/graders/validate \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"grader": {
"type": "string_check",
"name": "Example string check grader",
"input": "{{sample.output_text}}",
"reference": "{{item.label}}",
"operation": "eq"
}
}'
import OpenAI from 'openai';
const client = new OpenAI({
apiKey: process.env['OPENAI_API_KEY'], // This is the default and can be omitted
});
const response = await client.fineTuning.alpha.graders.validate({
grader: {
input: 'input',
name: 'name',
operation: 'eq',
reference: 'reference',
type: 'string_check',
},
});
console.log(response.grader);
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ.get("OPENAI_API_KEY"), # This is the default and can be omitted
)
response = client.fine_tuning.alpha.graders.validate(
grader={
"input": "input",
"name": "name",
"operation": "eq",
"reference": "reference",
"type": "string_check",
},
)
print(response.grader)
package main
import (
"context"
"fmt"
"github.com/openai/openai-go"
"github.com/openai/openai-go/option"
)
func main() {
client := openai.NewClient(
option.WithAPIKey("My API Key"),
)
response, err := client.FineTuning.Alpha.Graders.Validate(context.TODO(), openai.FineTuningAlphaGraderValidateParams{
Grader: openai.FineTuningAlphaGraderValidateParamsGraderUnion{
OfStringCheckGrader: &openai.StringCheckGraderParam{
Input: "input",
Name: "name",
Operation: openai.StringCheckGraderOperationEq,
Reference: "reference",
},
},
})
if err != nil {
panic(err.Error())
}
fmt.Printf("%+v
", response.Grader)
}
package com.openai.example;
import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.finetuning.alpha.graders.GraderValidateParams;
import com.openai.models.finetuning.alpha.graders.GraderValidateResponse;
import com.openai.models.graders.gradermodels.StringCheckGrader;
public final class Main {
private Main() {}
public static void main(String[] args) {
OpenAIClient client = OpenAIOkHttpClient.fromEnv();
GraderValidateParams params = GraderValidateParams.builder()
.grader(StringCheckGrader.builder()
.input("input")
.name("name")
.operation(StringCheckGrader.Operation.EQ)
.reference("reference")
.build())
.build();
GraderValidateResponse response = client.fineTuning().alpha().graders().validate(params);
}
}
require "openai"
openai = OpenAI::Client.new(api_key: "My API Key")
response = openai.fine_tuning.alpha.graders.validate(
grader: {input: "input", name: "name", operation: :eq, reference: "reference", type: :string_check}
)
puts(response)
Antwort
{
"grader": {
"type": "string_check",
"name": "Example string check grader",
"input": "{{sample.output_text}}",
"reference": "{{item.label}}",
"operation": "eq"
}
}