Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Per-request controls let you tune how model router in Microsoft Foundry routes a single request, without changing your deployment or the settings of Auto (model router without a deployment). From the request itself, you choose a routing mode, limit the request to specific models, require support for every parameter you send, and set session affinity. In this article, you turn on the controls, use each one with Python and the REST API, and check what model router applied.
- Send the
Foundry-Features: ModelRouterControls=V1Previewheader and arouting_configobject in the request body. Without the header, model router ignoresrouting_config. routing_confighas four fields:mode,models,require_parameters, andsession_affinity.- The controls work on all five endpoints (Responses and Chat Completions on the resource and project endpoints, and Messages on the resource endpoint), for Auto and for model router deployments.
- The header also turns on session affinity with the
adaptivestrategy, even when you don't sendrouting_config. To turn affinity off for a request, setrouting_config.session_affinity.strategytonone. - The response reports the routing mode used and the models attempted in
model_selection_details.
Important
Per-request routing controls for model router are in public preview and are subject to the Supplemental Terms of Use for Microsoft Azure Previews. This preview is provided without a service-level agreement, and Microsoft doesn't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.
Prerequisites
- Auto or a model router deployment. To start with Auto, see Quickstart: Get started with model router. To deploy model router, see Deploy model router.
- Python 3.10 or later with the OpenAI SDK (
pip install openai). For the Messages example, also install the Anthropic SDK (pip install -U anthropic) and deploy a Claude model from the pool in the same Foundry resource. - The
FOUNDRY_RESOURCE_ENDPOINT(https://<resource>.services.ai.azure.com) andFOUNDRY_API_KEYenvironment variables. To find the values, see Find your endpoint and key. - Familiarity with calling model router. See Call model router with Responses, Chat Completions, or Messages.
Turn on per-request controls
A per-request control takes effect only when the same request carries both of these items:
- The
Foundry-Features: ModelRouterControls=V1Previewheader. - A
routing_configobject in the request body.
The following Responses request asks for the cost routing mode:
curl -X POST "$FOUNDRY_RESOURCE_ENDPOINT/openai/v1/responses" \
-H "Content-Type: application/json" \
-H "api-key: $FOUNDRY_API_KEY" \
-H "Foundry-Features: ModelRouterControls=V1Preview" \
-d '{
"model": "auto",
"input": "Summarize the benefits of unit tests in two sentences.",
"routing_config": { "mode": "cost" }
}'
The response confirms the mode that model router used. This excerpt shows the relevant fields:
{
"model": "gpt-5.6-luna-2026-07-09",
"model_selection_details": {
"model_router_details": {
"mode": "cost"
}
}
}
In the OpenAI Python SDK, send the header with extra_headers and the object with extra_body. model_selection_details is an extension field, so read it from model_extra:
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
)
response = client.responses.create(
model="auto",
input="Summarize the benefits of unit tests in two sentences.",
extra_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
extra_body={"routing_config": {"mode": "cost"}},
)
details = (response.model_extra or {}).get("model_selection_details") or {}
print(f"Model: {response.model}")
print(f"Mode: {(details.get('model_router_details') or {}).get('mode')}")
Each client sends the header and routing_config in its own way:
| Client | Header | routing_config |
|---|---|---|
| OpenAI Python SDK | extra_headers={...} on the request, or default_headers={...} on the client |
extra_body={"routing_config": {...}} |
| OpenAI JavaScript SDK | { headers: {...} } as the second argument of create() |
A routing_config property in the request body. TypeScript users might need a type cast. |
| Anthropic Python SDK | extra_headers={...} |
extra_body={"routing_config": {...}} |
| REST API | Foundry-Features: ModelRouterControls=V1Preview |
routing_config in the JSON body |
routing_config accepts these fields:
| Field | Type | Values | Behavior |
|---|---|---|---|
routing_config.mode |
string | balanced, quality, cost (case-insensitive) |
Overrides the routing mode for this request. |
routing_config.models |
array of strings | Model names, for example ["gpt-5.6-sol", "claude-sonnet-5"]. Names only (no versions); case-insensitive; duplicates ignored. |
Routes only across these models for this request. Must be non-empty. One model pins the request to that model. |
routing_config.require_parameters |
boolean | true or false (default) |
true: only models that support every parameter you sent are eligible. If none qualify, the request fails with HTTP 400, and no model is called. |
routing_config.session_affinity |
object | strategy (adaptive, sticky, or none) and session_id |
Session affinity for this request. With the header, the default strategy is adaptive. See Keep a conversation on one model with model router. |
Note
If the header is missing, model router ignores routing_config and routes the request as if you didn't send it. When a control seems to have no effect, check the header first.
Choose a routing mode for one request
The routing_config.mode setting overrides the routing mode for one request. Values are case-insensitive.
| Mode | What model router does |
|---|---|
balanced |
Considers models within a small quality range of the best model for the prompt (for example, 1–2%), and picks the most cost-effective one. Balanced is the default routing mode. |
quality |
Picks the highest-quality model for the prompt, and ignores cost. |
cost |
Considers a wider quality band (for example, 5–6%), and picks the most cost-effective one. |
Use it when one kind of request needs a different tradeoff than the rest of your traffic. For example, use quality for a contract review in an app that otherwise runs on cost.
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
response = client.responses.create(
model="auto",
input="Review this contract clause for risks: ...",
extra_body={"routing_config": {"mode": "quality"}},
)
details = (response.model_extra or {}).get("model_selection_details") or {}
print(f"Model: {response.model}")
print(f"Mode: {(details.get('model_router_details') or {}).get('mode')}")
In the response, model_selection_details.model_router_details.mode is quality. For more about each mode, see Routing modes in depth.
Limit a request to specific models
Use routing_config.models to limit routing to the models you list for a specific request.
- List model names without versions, for example
["gpt-5.6-sol", "gpt-6-sol"]. Names are case-insensitive. Model router ignores duplicates. The list can't be empty. routing_config.modelsnarrows routing; it can't add models. Model router uses only the listed models that your deployment or Auto can already route to: its model subset, if one is set, or otherwise the pool models available in your region. If none of the listed models is available, the request fails with HTTP 400 and the messageNone of the requested routing models are available in this deployment.- List one model to pin the request to that model. When you list one model, model router can't fail over to a different model. To enable failover, list at least two models.
- Claude models, such as
claude-sonnet-5, are eligible only when your Foundry resource has a Global Standard deployment of the version in the supported models table. See Add Claude models. Only Claude models answer Messages requests, so list only Claude models in a Messages request.
Use this feature when a request should go only to models that you validated for a task, or to pin one request to one model for testing.
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
completion = client.chat.completions.create(
model="auto",
messages=[{"role": "user", "content": "Write a SQL query that returns the ten newest orders."}],
extra_body={"routing_config": {"models": ["gpt-5.6-sol", "gpt-6-sol"]}},
)
print(f"Model: {completion.model}")
print(completion.choices[0].message.content)
To pin a request to one model, send one name, for example "routing_config": { "models": ["gpt-6-sol"] }. In the response, the top-level model is one of the models you listed, and routing_trace[].attempts[].model shows the models that model router tried.
Each caller sets per-request models. To restrict routing for everyone who uses a deployment or Auto, set a model subset instead. See Route to a subset of models and Configure Auto for your resource. Azure Policy doesn't filter routed models at request time. For details, see How model router honors Azure Policy.
Require support for every parameter
The routing_config.require_parameters setting controls what happens when a model doesn't support a parameter you send.
| Value | Behavior |
|---|---|
false (default) |
Model router excludes models that can't serve a capability the request needs, such as function tools or JSON schema output. After it selects a model, it drops optional parameters that the model doesn't support, such as temperature on reasoning models. |
true |
Model router checks parameters before ranking: only models that support every parameter you sent, as sent, are eligible. If no model qualifies, the request fails with HTTP 400, and model router doesn't call any model. |
Use this setting when a parameter matters enough that you'd rather get an error than have model router drop it, such as temperature or a reasoning effort. In the following example, models that don't support temperature, such as reasoning models, aren't eligible.
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
completion = client.chat.completions.create(
model="auto",
messages=[{"role": "user", "content": "Suggest a name for a coffee shop. Reply with the name only."}],
temperature=0.2,
extra_body={"routing_config": {"require_parameters": True}},
)
print(f"Model: {completion.model}")
print(completion.choices[0].message.content)
When the request succeeds, the model that answers supports every parameter you sent. When no model qualifies, the request fails with HTTP 400 and a message that no model in the routing pool can preserve all requested parameters. To fix it, remove the parameter, add models that support it to routing_config.models, or set require_parameters to false. For how parameters are handled by default, see How model router handles parameters.
Combine controls
Send any combination of fields in one routing_config. The following Chat Completions request uses the quality mode, routes only between two models, requires support for every parameter, and keeps the conversation on one model with a session ID:
import os
import uuid
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
routing_config = {
"mode": "quality",
"models": ["gpt-5.6-sol", "gpt-6-sol"],
"require_parameters": True,
# Use one opaque ID per conversation. Don't put secrets or personal data in it.
"session_affinity": {"strategy": "sticky", "session_id": f"contract-review-{uuid.uuid4()}"},
}
completion = client.chat.completions.create(
model="auto",
messages=[{"role": "user", "content": "Review this contract clause for risks: ..."}],
extra_body={"routing_config": routing_config},
)
details = (completion.model_extra or {}).get("model_selection_details") or {}
router = details.get("model_router_details") or {}
print(f"Model: {completion.model}")
print(f"Mode: {router.get('mode')}, affinity: {router.get('session_affinity')}")
Use per-request controls with the Messages API
The Anthropic Python SDK uses the same extra_headers and extra_body arguments. Only Claude models deployed in your resource answer Messages requests.
import os
from anthropic import AnthropicFoundry
client = AnthropicFoundry(
api_key=os.environ["FOUNDRY_API_KEY"],
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/anthropic",
)
message = client.messages.create(
model="auto",
max_tokens=1024,
messages=[{"role": "user", "content": "Review this contract clause for risks: ..."}],
extra_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
extra_body={"routing_config": {"mode": "quality", "require_parameters": True}},
)
details = (message.model_extra or {}).get("model_selection_details") or {}
print(f"Model: {message.model}")
print(f"Mode: {(details.get('model_router_details') or {}).get('mode')}")
for block in message.content:
if block.type == "text":
print(block.text)
Check what model router applied
With the header, responses include model_selection_details.model_router_details. The following excerpt is from a Chat Completions response:
{
"model": "gpt-6-sol-2026-09-22",
"model_selection_details": {
"model_router_details": {
"mode": "quality",
"session_affinity": {
"strategy": "sticky",
"source": "session_id_payload",
"decision": "initialize"
},
"routing_trace": [
{
"latency_ms": 840,
"attempts": [
{ "model": "gpt-6-sol", "result": { "status": 200 } }
]
}
]
}
}
}
| Field | What it tells you |
|---|---|
mode |
The routing mode that model router used for the request. |
routing_trace |
The routing entries for the request: one entry for Chat Completions. Responses can include one entry per output item, each with an output_id. |
routing_trace[].latency_ms |
The latency reported for the entry. |
routing_trace[].attempts[] |
The ordered model attempts, each with model and result.status, plus result.error.code and result.error.message when the attempt failed. More than one attempt means that failover happened. |
session_affinity |
The affinity strategy, the source of the conversation identity, and the decision. See Check the affinity decision. |
- In streaming responses, the Responses API attaches the metadata to the
responseobject in theresponse.completedevent. The Messages API attaches it to themessage_deltaevent. model_selection_detailsand its child fields are optional during the preview. Parse them defensively, and tolerate fields that you don't recognize.- The metadata doesn't show whether your own deployment or router-managed capacity served the request, and it doesn't report reasoning effort.
The following code prints the mode and every attempt:
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
completion = client.chat.completions.create(
model="auto",
messages=[{"role": "user", "content": "Summarize the benefits of unit tests in two sentences."}],
extra_body={"routing_config": {"mode": "cost"}},
)
details = (completion.model_extra or {}).get("model_selection_details") or {}
router = details.get("model_router_details") or {}
print(f"Model: {completion.model}")
print(f"Mode: {router.get('mode')}")
for entry in router.get("routing_trace", []):
for attempt in entry.get("attempts", []):
result = attempt.get("result", {})
error = result.get("error", {}).get("code", "")
print(f"Attempt: {attempt.get('model')} -> HTTP {result.get('status')} {error}".rstrip())
The output looks similar to this example:
Model: gpt-5.6-luna-2026-07-09
Mode: cost
Attempt: gpt-5.6-luna -> HTTP 200
For failover analysis and more parsing examples, see Inspect routing attempts and failover. For every field, see Response fields.
Errors
Model router checks routing_config before it calls any model. When a value is invalid, or when the controls leave no model to route to, the request fails with HTTP 400 without calling a model. The error message tells you what to fix.
- Chat Completions and Responses APIs. Errors about
require_parametersand model parameters return anerrorobject with acode, eitherinvalid_require_parametersorinvalid_routing_parameters, plusparamandmessage. The other errors return the message without a code. - Messages API. Every error uses the Anthropic error format, with the message in
error.message.
| Error message | Cause | Fix |
|---|---|---|
The routing_config field must be an object. |
routing_config isn't a JSON object. |
Send an object, for example { "mode": "cost" }. |
The routing mode must be one of: quality, balanced, cost. |
mode isn't a supported value. |
Use balanced, quality, or cost. The value isn't case-sensitive. |
The routing models field must be a non-empty array of model names. |
models is empty or isn't an array. |
Send an array with at least one model name. |
Every routing model must be a non-empty string. |
An entry in models is empty or isn't a string. |
Send model names as strings, without versions, for example ["gpt-5.6-sol", "claude-sonnet-5"]. |
The routing_config.require_parameters field must be a boolean. |
require_parameters isn't a boolean. Code: invalid_require_parameters. |
Send true or false, not a string. |
The session_affinity field must be an object. |
session_affinity isn't an object. |
Send an object, for example { "strategy": "sticky", "session_id": "<id>" }. |
A message that starts with The session affinity strategy must be one of: |
strategy isn't a supported value. The message lists the supported values. |
Use adaptive, sticky, or none. |
routing_config.parameters and routing_config.constraints are not supported. Send model parameters in their native API fields. |
routing_config includes parameters or constraints. Code: invalid_routing_parameters. |
Send model parameters, such as temperature or reasoning effort, in the API's own request fields. |
routing_config must not contain duplicate JSON property names. |
A field appears twice in routing_config. Code: invalid_routing_parameters. |
Send each field once. |
A message that says no model can satisfy the parameter requirements, for example No model can satisfy the parameter requirements. Reasons: required_adaptation, temperature. |
require_parameters is true, and no eligible model supports every parameter you sent. The message names the parameters. Code: invalid_routing_parameters. |
Remove the unsupported parameter, add models that support it to models, or set require_parameters to false. |
None of the requested routing models are available in this deployment. |
None of the models in models is in the deployment's model subset, or in the pool models available to Auto in your region. |
List models that your deployment or Auto can route to. See Supported models. |
Every requested routing model must belong to the deployment pool. |
require_parameters is true, and models lists a model that the deployment or Auto can't route to. |
Remove that model from models, or add it to the deployment's model subset. |
Three situations look like problems but aren't errors:
- A missing
Foundry-Featuresheader: model router ignoresrouting_configand routes normally. - A model in
modelsthat the deployment or Auto can't route to, when another listed model is available: model router ignores that model and routes among the others. Withrequire_parametersset totrue, the request fails instead. - An invalid session ID: model router ignores it and identifies the conversation another way. See How session affinity works.
For more symptoms and fixes, see Troubleshoot per-request controls.