Structured outputs
Structured outputs make a model follow a JSON Schema definition that you provide as part of your inference API call. This is in contrast to the older JSON mode feature, which guaranteed valid JSON would be generated, but was unable to ensure strict adherence to the supplied schema. Structured outputs is recommended for function calling, extracting structured data, and building complex multi-step workflows.
Note
- Currently structured outputs is not supported on bring your own data scenario.
gpt-4o-mini
version:2024-07-18
gpt-4o
version:2024-08-06
Support for structured outputs was first added in API version 2024-08-01-preview
. It is available in the latest preview APIs as well as the latest GA API: 2024-10-21
.
You can use Pydantic
to define object schemas in Python. Depending on what version of the OpenAI and Pydantic
libraries you're running you may need to upgrade to a newer version. These examples were tested against openai 1.42.0
and pydantic 2.8.2
.
pip install openai pydantic --upgrade
If you are new to using Microsoft Entra ID for authentication see How to configure Azure OpenAI Service with Microsoft Entra ID authentication.
from pydantic import BaseModel
from openai import AzureOpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
token_provider = get_bearer_token_provider(
DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
)
client = AzureOpenAI(
azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"),
azure_ad_token_provider=token_provider,
api_version="2024-10-21"
)
class CalendarEvent(BaseModel):
name: str
date: str
participants: list[str]
completion = client.beta.chat.completions.parse(
model="MODEL_DEPLOYMENT_NAME", # replace with the model deployment name of your gpt-4o 2024-08-06 deployment
messages=[
{"role": "system", "content": "Extract the event information."},
{"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
],
response_format=CalendarEvent,
)
event = completion.choices[0].message.parsed
print(event)
print(completion.model_dump_json(indent=2))
name='Science Fair' date='Friday' participants=['Alice', 'Bob']
{
"id": "chatcmpl-A1EUP2fAmL4SeB1lVMinwM7I2vcqG",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "{\n \"name\": \"Science Fair\",\n \"date\": \"Friday\",\n \"participants\": [\"Alice\", \"Bob\"]\n}",
"refusal": null,
"role": "assistant",
"function_call": null,
"tool_calls": [],
"parsed": {
"name": "Science Fair",
"date": "Friday",
"participants": [
"Alice",
"Bob"
]
}
}
}
],
"created": 1724857389,
"model": "gpt-4o-2024-08-06",
"object": "chat.completion",
"service_tier": null,
"system_fingerprint": "fp_1c2eaec9fe",
"usage": {
"completion_tokens": 27,
"prompt_tokens": 32,
"total_tokens": 59
}
}
Structured Outputs for function calling can be enabled with a single parameter, by supplying strict: true
.
Note
Structured outputs is not supported with parallel function calls. When using structured outputs set parallel_tool_calls
to false
.
from enum import Enum
from typing import Union
from pydantic import BaseModel
import openai
from openai import AzureOpenAI
client = AzureOpenAI(
azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"),
api_key=os.getenv("AZURE_OPENAI_API_KEY"),
api_version="2024-10-21"
)
class GetDeliveryDate(BaseModel):
order_id: str
tools = [openai.pydantic_function_tool(GetDeliveryDate)]
messages = []
messages.append({"role": "system", "content": "You are a helpful customer support assistant. Use the supplied tools to assist the user."})
messages.append({"role": "user", "content": "Hi, can you tell me the delivery date for my order #12345?"})
response = client.chat.completions.create(
model="MODEL_DEPLOYMENT_NAME", # replace with the model deployment name of your gpt-4o 2024-08-06 deployment
messages=messages,
tools=tools
)
print(response.choices[0].message.tool_calls[0].function)
print(response.model_dump_json(indent=2))
Azure OpenAI structured outputs support the same subset of the JSON Schema as OpenAI.
- String
- Number
- Boolean
- Integer
- Object
- Array
- Enum
- anyOf
Note
Root objects cannot be the anyOf
type.
All fields or function parameters must be included as required. In the example below location
, and unit
are both specified under "required": ["location", "unit"]
.
{
"name": "get_weather",
"description": "Fetches the weather in the given location",
"strict": true,
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The location to get the weather for"
},
"unit": {
"type": "string",
"description": "The unit to return the temperature in",
"enum": ["F", "C"]
}
},
"additionalProperties": false,
"required": ["location", "unit"]
}
If needed, it's possible to emulate an optional parameter by using a union type with null
. In this example, this is achieved with the line "type": ["string", "null"],
.
{
"name": "get_weather",
"description": "Fetches the weather in the given location",
"strict": true,
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The location to get the weather for"
},
"unit": {
"type": ["string", "null"],
"description": "The unit to return the temperature in",
"enum": ["F", "C"]
}
},
"additionalProperties": false,
"required": [
"location", "unit"
]
}
}
A schema may have up to 100 object properties total, with up to five levels of nesting
This property controls if an object can have additional key value pairs that weren't defined in the JSON Schema. In order to use structured outputs, you must set this value to false.
Structured outputs are ordered the same as the provided schema. To change the output order, modify the order of the schema that you send as part of your inference request.
Type | Unsupported Keyword |
---|---|
String | minlength maxLength pattern format |
Number | minimum maximum multipleOf |
Objects | patternProperties unevaluatedProperties propertyNames minProperties maxProperties |
Arrays | unevaluatedItems contains minContains maxContains minItems maxItems uniqueItems |
Example supported anyOf
schema:
{
"type": "object",
"properties": {
"item": {
"anyOf": [
{
"type": "object",
"description": "The user object to insert into the database",
"properties": {
"name": {
"type": "string",
"description": "The name of the user"
},
"age": {
"type": "number",
"description": "The age of the user"
}
},
"additionalProperties": false,
"required": [
"name",
"age"
]
},
{
"type": "object",
"description": "The address object to insert into the database",
"properties": {
"number": {
"type": "string",
"description": "The number of the address. Eg. for 123 main st, this would be 123"
},
"street": {
"type": "string",
"description": "The street name. Eg. for 123 main st, this would be main st"
},
"city": {
"type": "string",
"description": "The city of the address"
}
},
"additionalProperties": false,
"required": [
"number",
"street",
"city"
]
}
]
}
},
"additionalProperties": false,
"required": [
"item"
]
}
Supported example:
{
"type": "object",
"properties": {
"steps": {
"type": "array",
"items": {
"$ref": "#/$defs/step"
}
},
"final_answer": {
"type": "string"
}
},
"$defs": {
"step": {
"type": "object",
"properties": {
"explanation": {
"type": "string"
},
"output": {
"type": "string"
}
},
"required": [
"explanation",
"output"
],
"additionalProperties": false
}
},
"required": [
"steps",
"final_answer"
],
"additionalProperties": false
}
Example using # for root recursion:
{
"name": "ui",
"description": "Dynamically generated UI",
"strict": true,
"schema": {
"type": "object",
"properties": {
"type": {
"type": "string",
"description": "The type of the UI component",
"enum": ["div", "button", "header", "section", "field", "form"]
},
"label": {
"type": "string",
"description": "The label of the UI component, used for buttons or form fields"
},
"children": {
"type": "array",
"description": "Nested UI components",
"items": {
"$ref": "#"
}
},
"attributes": {
"type": "array",
"description": "Arbitrary attributes for the UI component, suitable for any element",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "The name of the attribute, for example onClick or className"
},
"value": {
"type": "string",
"description": "The value of the attribute"
}
},
"additionalProperties": false,
"required": ["name", "value"]
}
}
},
"required": ["type", "label", "children", "attributes"],
"additionalProperties": false
}
}
Example of explicit recursion:
{
"type": "object",
"properties": {
"linked_list": {
"$ref": "#/$defs/linked_list_node"
}
},
"$defs": {
"linked_list_node": {
"type": "object",
"properties": {
"value": {
"type": "number"
},
"next": {
"anyOf": [
{
"$ref": "#/$defs/linked_list_node"
},
{
"type": "null"
}
]
}
},
"additionalProperties": false,
"required": [
"next",
"value"
]
}
},
"additionalProperties": false,
"required": [
"linked_list"
]
}