Evals

Get an output item of an evaluation run

GET /openai/v1/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}

Retrieves one output item produced by an evaluation run, including its input, generated output when applicable, and testing-criterion results.

Path parameters

eval_id string required

The identifier of the evaluation that owns the run.

run_id string required

The identifier of the run that produced the output item.

output_item_id string required

The identifier of the output item to retrieve.

Responses

The request has succeeded.

Body

Content-Type: application/json

object enum required

The type of the object. Always "eval.run.output_item".

eval.run.output_item

id string required

Unique identifier for the evaluation run output item.

run_id string required

The identifier of the evaluation run associated with this output item.

eval_id string required

The identifier of the evaluation group.

created_at integer required

Unix timestamp (in seconds) when the evaluation run was created.

status string required

The status of the evaluation run.

datasource_item_id integer required

datasource_item object required

Details of the input data source item.

results EvalRunOutputItemResult[] required

A list of grader results for this output item.

sample OpenAI.EvalRunOutputItemSample required

input EvalRunOutputItemSampleInput[] required

output EvalRunOutputItemSampleOutput[] required

finish_reason string required

model string required

usage OpenAI.EvalRunOutputItemSampleUsage required

total_tokens integer required

completion_tokens integer required

prompt_tokens integer required

cached_tokens integer required

error OpenAI.EvalApiError | null required

An object representing an error response from the Eval API.

code string required

The error code.

message string required

The error message.

temperature number required

max_completion_tokens integer required

top_p number required

seed integer required

Request

curl https://api.openai.com/v1/evals/eval_67abd54d9b0081909a86353f6fb9317a/runs/evalrun_67abd54d60ec8190832b46859da808f7/output_items/outputitem_67abd55eb6548190bb580745d5644a33 \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json"

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("OPENAI_API_KEY"),  # This is the default and can be omitted
)
output_item = client.evals.runs.output_items.retrieve(
    output_item_id="output_item_id",
    eval_id="eval_id",
    run_id="run_id",
)
print(output_item.id)

import OpenAI from "openai";

const openai = new OpenAI();

const outputItem = await openai.evals.runs.outputItems.retrieve(
  "outputitem_67abd55eb6548190bb580745d5644a33",
  {
    eval_id: "eval_67abd54d9b0081909a86353f6fb9317a",
    run_id: "evalrun_67abd54d60ec8190832b46859da808f7",
  }
);
console.log(outputItem);

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env['OPENAI_API_KEY'], // This is the default and can be omitted
});

const outputItem = await client.evals.runs.outputItems.retrieve('output_item_id', {
  eval_id: 'eval_id',
  run_id: 'run_id',
});

console.log(outputItem.id);

package com.openai.example;

import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.evals.runs.outputitems.OutputItemRetrieveParams;
import com.openai.models.evals.runs.outputitems.OutputItemRetrieveResponse;

public final class Main {
    private Main() {}

    public static void main(String[] args) {
        OpenAIClient client = OpenAIOkHttpClient.fromEnv();

        OutputItemRetrieveParams params = OutputItemRetrieveParams.builder()
            .evalId("eval_id")
            .runId("run_id")
            .outputItemId("output_item_id")
            .build();
        OutputItemRetrieveResponse outputItem = client.evals().runs().outputItems().retrieve(params);
    }
}

require "openai"

openai = OpenAI::Client.new(api_key: "My API Key")

output_item = openai.evals.runs.output_items.retrieve("output_item_id", eval_id: "eval_id", run_id: "run_id")

puts(output_item)

Response

{
  "object": "eval.run.output_item",
  "id": "outputitem_67e5796c28e081909917bf79f6e6214d",
  "created_at": 1743092076,
  "run_id": "evalrun_67abd54d60ec8190832b46859da808f7",
  "eval_id": "eval_67abd54d9b0081909a86353f6fb9317a",
  "status": "pass",
  "datasource_item_id": 5,
  "datasource_item": {
    "input": "Stock Markets Rally After Positive Economic Data Released",
    "ground_truth": "Markets"
  },
  "results": [
    {
      "name": "String check-a2486074-d803-4445-b431-ad2262e85d47",
      "sample": null,
      "passed": true,
      "score": 1.0
    }
  ],
  "sample": {
    "input": [
      {
        "role": "developer",
        "content": "Categorize a given news headline into one of the following topics: Technology, Markets, World, Business, or Sports.

# Steps

1. Analyze the content of the news headline to understand its primary focus.
2. Extract the subject matter, identifying any key indicators or keywords.
3. Use the identified indicators to determine the most suitable category out of the five options: Technology, Markets, World, Business, or Sports.
4. Ensure only one category is selected per headline.

# Output Format

Respond with the chosen category as a single word. For instance: "Technology", "Markets", "World", "Business", or "Sports".

# Examples

**Input**: "Apple Unveils New iPhone Model, Featuring Advanced AI Features"
**Output**: "Technology"

**Input**: "Global Stocks Mixed as Investors Await Central Bank Decisions"
**Output**: "Markets"

**Input**: "War in Ukraine: Latest Updates on Negotiation Status"
**Output**: "World"

**Input**: "Microsoft in Talks to Acquire Gaming Company for $2 Billion"
**Output**: "Business"

**Input**: "Manchester United Secures Win in Premier League Football Match"
**Output**: "Sports"

# Notes

- If the headline appears to fit into more than one category, choose the most dominant theme.
- Keywords or phrases such as "stocks", "company acquisition", "match", or technological brands can be good indicators for classification.
",
        "tool_call_id": null,
        "tool_calls": null,
        "function_call": null
      },
      {
        "role": "user",
        "content": "Stock Markets Rally After Positive Economic Data Released",
        "tool_call_id": null,
        "tool_calls": null,
        "function_call": null
      }
    ],
    "output": [
      {
        "role": "assistant",
        "content": "Markets",
        "tool_call_id": null,
        "tool_calls": null,
        "function_call": null
      }
    ],
    "finish_reason": "stop",
    "model": "gpt-4o-mini-2024-07-18",
    "usage": {
      "total_tokens": 325,
      "completion_tokens": 2,
      "prompt_tokens": 323,
      "cached_tokens": 0
    },
    "error": null,
    "temperature": 1.0,
    "max_completion_tokens": 2048,
    "top_p": 1.0,
    "seed": 42
  }
}