Fine-tuning
Validate a grader
Validates a grader.
Corpo della richiesta
The request body.
grader
OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderMulti
necessario
The grader used for the fine-tuning job.
type
enum
necessario
The object type, which is always string_check.
name
string
necessario
The name of the grader.
input
string
necessario
The input text. This may include template strings.
reference
string
necessario
The reference text. This may include template strings.
operation
enum
necessario
The string check operation to perform. One of eq, ne, like, or ilike.
type
enum
necessario
The type of grader.
name
string
necessario
The name of the grader.
input
string
necessario
The text being graded.
reference
string
necessario
The text being graded against.
evaluation_metric
enum
necessario
The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.
type
enum
necessario
The object type, which is always python.
name
string
necessario
The name of the grader.
source
string
necessario
The source code of the python script.
image_tag
string
The image tag to use for the python script.
type
enum
necessario
The object type, which is always score_model.
name
string
necessario
The name of the grader.
model
string
necessario
The model to use for the evaluation.
sampling_params
OpenAI.EvalGraderScoreModelSamplingParams
reasoning_effort
string | enum | null
Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.
enum
input
OpenAI.EvalItem[]
necessario
The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
range
OpenAI.numeric[]
The service requires two numbers for the score range. Defaults to [0, 1].
type
enum
necessario
The object type, which is always multi.
name
string
necessario
The name of the grader.
graders
object
necessario
[key: string]
OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderLabelModel
type
enum
necessario
The object type, which is always string_check.
name
string
necessario
The name of the grader.
input
string
necessario
The input text. This may include template strings.
reference
string
necessario
The reference text. This may include template strings.
operation
enum
necessario
The string check operation to perform. One of eq, ne, like, or ilike.
type
enum
necessario
The type of grader.
name
string
necessario
The name of the grader.
input
string
necessario
The text being graded.
reference
string
necessario
The text being graded against.
evaluation_metric
enum
necessario
The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.
type
enum
necessario
The object type, which is always python.
name
string
necessario
The name of the grader.
source
string
necessario
The source code of the python script.
image_tag
string
The image tag to use for the python script.
type
enum
necessario
The object type, which is always score_model.
name
string
necessario
The name of the grader.
model
string
necessario
The model to use for the evaluation.
sampling_params
OpenAI.EvalGraderScoreModelSamplingParams
reasoning_effort
string | enum | null
Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.
enum
input
OpenAI.EvalItem[]
necessario
The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
range
OpenAI.numeric[]
The service requires two numbers for the score range. Defaults to [0, 1].
type
enum
necessario
The object type, which is always label_model.
name
string
necessario
The name of the grader.
model
string
necessario
The model to use for the evaluation. Must support structured outputs.
labels
string[]
necessario
The labels to assign to each item in the evaluation.
passing_labels
string[]
necessario
The labels that indicate a passing result. Must be a subset of labels.
calculate_output
string
necessario
A formula to calculate the output based on grader results.
Risposte
The request has succeeded.
Corpo
grader
OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderMulti
The grader used for the fine-tuning job.
type
enum
necessario
The object type, which is always string_check.
name
string
necessario
The name of the grader.
input
string
necessario
The input text. This may include template strings.
reference
string
necessario
The reference text. This may include template strings.
operation
enum
necessario
The string check operation to perform. One of eq, ne, like, or ilike.
type
enum
necessario
The type of grader.
name
string
necessario
The name of the grader.
input
string
necessario
The text being graded.
reference
string
necessario
The text being graded against.
evaluation_metric
enum
necessario
The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.
type
enum
necessario
The object type, which is always python.
name
string
necessario
The name of the grader.
source
string
necessario
The source code of the python script.
image_tag
string
The image tag to use for the python script.
type
enum
necessario
The object type, which is always score_model.
name
string
necessario
The name of the grader.
model
string
necessario
The model to use for the evaluation.
sampling_params
OpenAI.EvalGraderScoreModelSamplingParams
reasoning_effort
string | enum | null
Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.
enum
input
OpenAI.EvalItem[]
necessario
The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
range
OpenAI.numeric[]
The service requires two numbers for the score range. Defaults to [0, 1].
type
enum
necessario
The object type, which is always multi.
name
string
necessario
The name of the grader.
graders
object
necessario
[key: string]
OpenAI.GraderStringCheck | OpenAI.GraderTextSimilarity | OpenAI.GraderPython | OpenAI.GraderScoreModel | OpenAI.GraderLabelModel
type
enum
necessario
The object type, which is always string_check.
name
string
necessario
The name of the grader.
input
string
necessario
The input text. This may include template strings.
reference
string
necessario
The reference text. This may include template strings.
operation
enum
necessario
The string check operation to perform. One of eq, ne, like, or ilike.
type
enum
necessario
The type of grader.
name
string
necessario
The name of the grader.
input
string
necessario
The text being graded.
reference
string
necessario
The text being graded against.
evaluation_metric
enum
necessario
The evaluation metric to use. One of cosine, fuzzy_match, bleu, gleu, meteor, rouge_1, rouge_2, rouge_3, rouge_4, rouge_5, or rouge_l.
type
enum
necessario
The object type, which is always python.
name
string
necessario
The name of the grader.
source
string
necessario
The source code of the python script.
image_tag
string
The image tag to use for the python script.
type
enum
necessario
The object type, which is always score_model.
name
string
necessario
The name of the grader.
model
string
necessario
The model to use for the evaluation.
sampling_params
OpenAI.EvalGraderScoreModelSamplingParams
reasoning_effort
string | enum | null
Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the reasoning guide for model-specific support.
enum
input
OpenAI.EvalItem[]
necessario
The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
range
OpenAI.numeric[]
The service requires two numbers for the score range. Defaults to [0, 1].
type
enum
necessario
The object type, which is always label_model.
name
string
necessario
The name of the grader.
model
string
necessario
The model to use for the evaluation. Must support structured outputs.
labels
string[]
necessario
The labels to assign to each item in the evaluation.
passing_labels
string[]
necessario
The labels that indicate a passing result. Must be a subset of labels.
calculate_output
string
necessario
A formula to calculate the output based on grader results.
Richiedi
curl https://api.openai.com/v1/fine_tuning/alpha/graders/validate \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"grader": {
"type": "string_check",
"name": "Example string check grader",
"input": "{{sample.output_text}}",
"reference": "{{item.label}}",
"operation": "eq"
}
}'
import OpenAI from 'openai';
const client = new OpenAI({
apiKey: process.env['OPENAI_API_KEY'], // This is the default and can be omitted
});
const response = await client.fineTuning.alpha.graders.validate({
grader: {
input: 'input',
name: 'name',
operation: 'eq',
reference: 'reference',
type: 'string_check',
},
});
console.log(response.grader);
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ.get("OPENAI_API_KEY"), # This is the default and can be omitted
)
response = client.fine_tuning.alpha.graders.validate(
grader={
"input": "input",
"name": "name",
"operation": "eq",
"reference": "reference",
"type": "string_check",
},
)
print(response.grader)
package main
import (
"context"
"fmt"
"github.com/openai/openai-go"
"github.com/openai/openai-go/option"
)
func main() {
client := openai.NewClient(
option.WithAPIKey("My API Key"),
)
response, err := client.FineTuning.Alpha.Graders.Validate(context.TODO(), openai.FineTuningAlphaGraderValidateParams{
Grader: openai.FineTuningAlphaGraderValidateParamsGraderUnion{
OfStringCheckGrader: &openai.StringCheckGraderParam{
Input: "input",
Name: "name",
Operation: openai.StringCheckGraderOperationEq,
Reference: "reference",
},
},
})
if err != nil {
panic(err.Error())
}
fmt.Printf("%+v
", response.Grader)
}
package com.openai.example;
import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.finetuning.alpha.graders.GraderValidateParams;
import com.openai.models.finetuning.alpha.graders.GraderValidateResponse;
import com.openai.models.graders.gradermodels.StringCheckGrader;
public final class Main {
private Main() {}
public static void main(String[] args) {
OpenAIClient client = OpenAIOkHttpClient.fromEnv();
GraderValidateParams params = GraderValidateParams.builder()
.grader(StringCheckGrader.builder()
.input("input")
.name("name")
.operation(StringCheckGrader.Operation.EQ)
.reference("reference")
.build())
.build();
GraderValidateResponse response = client.fineTuning().alpha().graders().validate(params);
}
}
require "openai"
openai = OpenAI::Client.new(api_key: "My API Key")
response = openai.fine_tuning.alpha.graders.validate(
grader: {input: "input", name: "name", operation: :eq, reference: "reference", type: :string_check}
)
puts(response)
Response
{
"grader": {
"type": "string_check",
"name": "Example string check grader",
"input": "{{sample.output_text}}",
"reference": "{{item.label}}",
"operation": "eq"
}
}