Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Model router in Microsoft Foundry gives you one endpoint and one model name for many models, and it supports the API format your app already uses. In this article, you choose an endpoint, authenticate, call model router with the Responses, Chat Completions, and Messages APIs, and confirm which model answered each request.
- One model router, three API formats: Responses, Chat Completions, and Messages (Anthropic format, preview).
- Set
modeltoautoto use Auto (model router without a deployment, preview), or set it to the name of your model router deployment. - The
modelfield in every response names the model that answered, for examplegpt-5.6-luna-2026-07-09. - Only Claude models answer Messages requests, and you must deploy each one in the same Foundry resource.
- Use the
/openai/v1/or/anthropic/v1/path. You don't need anapi-versionquery parameter.
Important
Features or capabilities marked (preview) in this article are in public preview and are subject to the Supplemental Terms of Use for Microsoft Azure Previews. Previews are provided without a service-level agreement, and Microsoft doesn't recommend them for production workloads. Certain features might not be supported or might have constrained capabilities.
Prerequisites
- An Azure subscription. If you don't have one, create one for free.
- A Foundry project in a model router region. To create one, see Create a project.
- Auto needs no setup. To call your own model router deployment instead, deploy model router first.
- For the Messages API, at least one Claude model from the model pool deployed in the same Foundry resource. See Add Claude models.
- Python 3.10 or later with the OpenAI SDK (
pip install openai). For the Messages API, also install the Anthropic SDK (pip install -U anthropic). For Microsoft Entra ID, also installazure-identityand the Azure CLI. - For JavaScript, Node.js 20 or later with the OpenAI SDK (
npm install openai).
The samples in this article read these environment variables:
| Variable | Value |
|---|---|
FOUNDRY_RESOURCE_ENDPOINT |
Your resource endpoint, https://<resource>.services.ai.azure.com |
FOUNDRY_PROJECT_ENDPOINT |
Your project endpoint, https://<resource>.services.ai.azure.com/api/projects/<project> |
FOUNDRY_API_KEY |
Your resource (project) API key |
MODEL_ROUTER_DEPLOYMENT |
Your model router deployment name, when you don't use Auto |
Choose an endpoint
Model router works on five endpoints, for Auto and for model router deployments. Pick the API format, and then the resource or project endpoint.
| API | Resource endpoint | Project endpoint |
|---|---|---|
| Responses | https://<resource>.services.ai.azure.com/openai/v1/responses |
https://<resource>.services.ai.azure.com/api/projects/<project>/openai/v1/responses |
| Chat Completions | https://<resource>.services.ai.azure.com/openai/v1/chat/completions |
https://<resource>.services.ai.azure.com/api/projects/<project>/openai/v1/chat/completions |
| Messages (preview) | https://<resource>.services.ai.azure.com/anthropic/v1/messages |
Not available |
- Resource endpoint: The simplest choice, and the only endpoint for the Messages API.
- Project endpoint: Use it with Foundry Agent Service and other project-scoped features, and to authenticate with Microsoft Entra ID through the Foundry SDK.
- The
https://<resource>.openai.azure.com/openai/v1/host also works for Responses and Chat Completions.
Find your endpoint and key
- In the Foundry portal, open your project.
- On the Home page, copy the Project endpoint and the Project API key.
- To get your resource endpoint, remove
/api/projects/<project>from the project endpoint. The result has the formhttps://<resource>.services.ai.azure.com.
To get the key with the Azure CLI, run:
az cognitiveservices account keys list --name <resource> --resource-group <resource-group> --query key1 --output tsv
Use the project endpoint
The Foundry SDK (pip install "azure-ai-projects>=2.0.0" azure-identity) returns an OpenAI client for your project endpoint that authenticates with Microsoft Entra ID:
import os
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential
project = AIProjectClient(
endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
credential=DefaultAzureCredential(),
)
client = project.get_openai_client()
response = client.responses.create(model="auto", input="Summarize the benefits of unit tests.")
print(f"Model: {response.model}")
print(response.output_text)
With the plain OpenAI client, set base_url to f"{os.environ['FOUNDRY_PROJECT_ENDPOINT']}/openai/v1/". The Responses and Chat Completions samples in this article then work unchanged on the project endpoint.
Authenticate
Every API accepts an API key or a Microsoft Entra ID token. The API key header differs by API.
| API | API key | Microsoft Entra ID | Token scope |
|---|---|---|---|
| Responses and Chat Completions | api-key: <key> or Authorization: Bearer <key> |
Authorization: Bearer <token> |
https://ai.azure.com/.default |
| Messages (preview) | x-api-key: <key> |
Authorization: Bearer <token> |
https://ai.azure.com/.default |
With Microsoft Entra ID, the caller needs the Foundry User role (previously named Azure AI User) on the resource or project. For role assignment steps, see Role-based access control for Microsoft Foundry.
Note
The Messages API authenticates API keys with the x-api-key header, not api-key. A request that sends api-key instead of x-api-key returns HTTP 401.
To use Microsoft Entra ID, sign in with the Azure CLI (az login), and then pass a token provider to the client.
OpenAI SDK (Responses and Chat Completions)
import os
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import OpenAI
token_provider = get_bearer_token_provider(
DefaultAzureCredential(), "https://ai.azure.com/.default"
)
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=token_provider,
)
response = client.responses.create(model="auto", input="Summarize the benefits of unit tests.")
print(f"Model: {response.model}")
Anthropic SDK (Messages)
import os
from anthropic import AnthropicFoundry
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
token_provider = get_bearer_token_provider(
DefaultAzureCredential(), "https://ai.azure.com/.default"
)
client = AnthropicFoundry(
azure_ad_token_provider=token_provider,
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/anthropic",
)
message = client.messages.create(
model="auto",
max_tokens=1024,
messages=[{"role": "user", "content": "Summarize the benefits of unit tests."}],
)
print(f"Model: {message.model}")
Use the Responses API
The Responses API works on the resource and project endpoints. With model router, it supports streaming, function tools, structured outputs (JSON schema), reasoning effort (reasoning.effort), image input, multi-turn state with previous_response_id and conversation, stored responses (store), background mode (background: true), and built-in tools (web search, file search, code interpreter, and MCP).
Set model to auto, or replace auto with your deployment name, for example os.environ["MODEL_ROUTER_DEPLOYMENT"].
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
)
response = client.responses.create(
model="auto",
input="Explain retrieval-augmented generation in two sentences.",
)
print(f"Model: {response.model}")
print(response.output_text)
The Python and JavaScript samples print output similar to this example:
Model: gpt-5.6-luna-2026-07-09
Retrieval-augmented generation (RAG) retrieves passages that are relevant to a question from your own data and adds them to the prompt. The model then grounds its answer in those passages instead of relying only on its training data.
Stream a Responses API call
Set stream=True to receive events as the model generates them. The response.completed event carries the full response, including the model that answered.
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
)
stream = client.responses.create(model="auto", input="Write a haiku about routing.", stream=True)
for event in stream:
if event.type == "response.output_text.delta":
print(event.delta, end="", flush=True)
elif event.type == "response.completed":
print(f"\nModel: {event.response.model}")
Continue a conversation
Pass the earlier response's id as previous_response_id. When a request continues a conversation, model router considers the earlier turns when it ranks models for the new turn.
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
)
first = client.responses.create(model="auto", input="Suggest a name for a hiking app.")
second = client.responses.create(
model="auto",
previous_response_id=first.id,
input="Now write a one-line tagline for it.",
)
print(f"Turn 1 model: {first.model}")
print(f"Turn 2 model: {second.model}")
print(second.output_text)
- A
conversationID works the same way: model router considers the earlier turns in the conversation when it ranks models. - Without session affinity, model router doesn't try to keep the previous turn's model, so a later turn can go to a different model. After a response starts, model router doesn't switch models within that response. To keep turns on one model, see Keep a conversation on one model with model router.
- Stored responses, background mode, and built-in tools work with model router as they do with other Responses models. For details, see Chaining responses together, Background tasks, Code Interpreter, and Using remote MCP servers.
Use the Chat Completions API
The Chat Completions API works on the resource and project endpoints. With model router, it supports streaming, function tools, structured outputs (JSON schema), reasoning_effort, image input, and sampling parameters such as temperature. Model router drops sampling parameters for models that don't support them.
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
)
completion = client.chat.completions.create(
model="auto",
messages=[{"role": "user", "content": "Explain retrieval-augmented generation in two sentences."}],
)
print(f"Model: {completion.model}")
print(completion.choices[0].message.content)
The Chat Completions API is stateless. To continue a conversation, append the assistant's reply and the next user message to messages, and send the full history again. Send function tools, response_format, and image content the same way you do for any Chat Completions model. When a request uses function tools or JSON schema output, model router ranks only models that can serve them.
Stream a Chat Completions call
Set stream=True, and read the text from each chunk's delta. The chunks also carry the model field.
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
)
stream = client.chat.completions.create(
model="auto",
messages=[{"role": "user", "content": "Name three Seattle landmarks, comma-separated."}],
stream=True,
)
model = None
for chunk in stream:
model = chunk.model or model
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
print(f"\nModel: {model}")
Use the Messages API
The Messages API (preview) accepts Anthropic-format requests, so the Anthropic SDK, Claude Code, and the Claude Agent SDK work with model router. Before you call it, check these requirements:
- Claude deployments: Only the Claude models in the pool answer Messages requests. Each one must be deployed in the same Foundry resource as a Global Standard deployment of the version in the supported models table. If no Claude deployment is available to your model router deployment or to Auto, the request returns HTTP 400. See Add Claude models.
- Endpoint: Use the resource endpoint,
https://<resource>.services.ai.azure.com/anthropic/v1/messages. The Messages API has no project endpoint. - Authentication: Send the
x-api-keyheader or a Microsoft Entra token. A request that sendsapi-keyinstead ofx-api-keyreturns HTTP 401. max_tokens: Required in every request.anthropic-version: Defaults to2023-06-01when you omit it.
The Messages API supports streaming, client tools, image input, system prompts, and prompt caching with cache_control. Model router passes anthropic-beta headers through to the selected model, and support depends on that model.
import os
from anthropic import AnthropicFoundry
client = AnthropicFoundry(
api_key=os.environ["FOUNDRY_API_KEY"],
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/anthropic",
)
message = client.messages.create(
model="auto",
max_tokens=1024,
messages=[{"role": "user", "content": "Explain retrieval-augmented generation in two sentences."}],
)
print(f"Model: {message.model}")
for block in message.content:
if block.type == "text":
print(block.text)
Stream a Messages call
Use the Anthropic SDK's messages.stream() helper to print text as it arrives, and then read the final message.
import os
from anthropic import AnthropicFoundry
client = AnthropicFoundry(
api_key=os.environ["FOUNDRY_API_KEY"],
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/anthropic",
)
with client.messages.stream(
model="auto",
max_tokens=1024,
messages=[{"role": "user", "content": "Write a haiku about routing."}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
final = stream.get_final_message()
print(f"\nModel: {final.model}")
To run Claude Code or the Claude Agent SDK on model router, see Claude Agent SDK and Claude Code.
Feature support by API
The following table shows what model router supports in each API format. Not documented means that this article doesn't cover the feature for that API.
| Feature | Responses | Chat Completions | Messages (preview) |
|---|---|---|---|
| Resource endpoint | Yes | Yes | Yes |
| Project endpoint | Yes | Yes | No |
| Models that can answer | Models in your pool | Models in your pool | Claude models deployed in your resource |
| Streaming | Yes | Yes | Yes |
| Function or client tools | Yes (function tools) | Yes (function tools) | Yes (client tools) |
| Structured outputs (JSON schema) | Yes | Yes | Not documented |
| Image input | Yes | Yes | Yes |
| Reasoning effort parameter | reasoning.effort |
reasoning_effort |
output_config.effort |
| Multi-turn state | previous_response_id and conversation |
No (send the full history in messages) |
No (send the full history in messages) |
| Stored responses | Yes (store) |
Not documented | No |
| Background mode | Yes (background: true) |
No | No |
| Built-in tools (web search, file search, code interpreter, MCP) | Yes | No | Not documented |
| Prompt caching | Yes | Yes | Yes (cache_control) |
| Per-request controls (preview) | Yes | Yes | Yes |
| Session affinity (preview) | Yes | Yes | Yes |
| Routing metadata in the response (preview) | Yes | Yes | Yes |
Prompt caching works through the underlying models: cache hits happen when the same model serves consecutive requests with overlapping prefixes. Session affinity raises the odds. For per-request controls and session affinity, see Control model router routing per request and Keep a conversation on one model with model router.
How model router handles parameters
Send the parameters you'd send to any model. By default, model router handles them in two steps:
- Before it ranks models, model router excludes models that can't serve what your request needs, such as function tools, JSON schema output, or image input.
- After it selects a model, model router drops optional parameters that the model doesn't support, and sends the rest of the request. For example, it drops
temperature,top_p,presence_penalty,frequency_penalty,logit_bias,logprobs, andstopfor reasoning models, andreasoning_effortfor non-reasoning models.
To keep every parameter, set routing_config.require_parameters to true (preview). Model router then checks parameters before ranking, as a filter: only models that support every parameter you sent are eligible, so nothing is dropped. If no model qualifies, the request fails with HTTP 400, and no model is called.
routing_config.require_parameters |
Unsupported parameter | Models that can answer |
|---|---|---|
false (default) |
Dropped after model router selects a model. | Every eligible model in your pool or model subset. |
true (preview) |
Never dropped. Models that don't support it are excluded before ranking. | Only models that support every parameter you sent. |
For example, a request that sets temperature can go to a reasoning model by default, with temperature dropped. With require_parameters set to true, the same request goes only to models that accept temperature. See Require support for every parameter.
Reasoning effort
Send reasoning effort in each API's native field. routing_config has no reasoning-effort field.
| API | Request field | Effort reported in the response |
|---|---|---|
| Chat Completions | reasoning_effort |
No |
| Responses | reasoning.effort |
reasoning.effort, for OpenAI reasoning models |
| Messages | output_config.effort |
No. Claude models don't report it. |
Model router passes the value to the selected model when that model supports it, and drops it for a model that doesn't, such as a non-reasoning model. With routing_config.require_parameters set to true, only models that support reasoning effort are eligible. Some values, such as minimal or xhigh, are supported by only some reasoning models, and model router doesn't route a request to a reasoning model that rejects the value. model_selection_details doesn't report the effort.
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
)
response = client.responses.create(
model="auto",
input="Is 221 a prime number? Answer yes or no, then give one factor pair if not.",
reasoning={"effort": "low"},
)
print(f"Model: {response.model}")
effort = response.reasoning.effort if response.reasoning else None # Reported by OpenAI reasoning models.
print(f"Reported effort: {effort}")
print(response.output_text)
Input and context limits
- Model router ranks models by the text of the request. Image input is supported, but the routing decision uses only the text. Audio input isn't supported.
- The effective context window is the smallest context window among the models in your pool. For long prompts, route to a subset of large-context models. See Route to a subset of models.
For more about how routing uses your parameters, see How parameters and reasoning effort are handled.
See which model answered
The model field in every response names the model that served the request, for example gpt-5.6-luna-2026-07-09. Read it the same way in each API:
| API | Python | REST API | Streaming |
|---|---|---|---|
| Responses | response.model |
Top-level model |
response.model in the response.completed event |
| Chat Completions | completion.model |
Top-level model |
model on the streamed chunks |
| Messages (preview) | message.model |
Top-level model |
model on the final message from stream.get_final_message() |
In JavaScript, read response.model or completion.model. In the Foundry portal, go to Build > Models, and select your model router deployment to open it in the playground. Each response in the playground shows the model that model router selected.
Tip
Log the model field with every request. Use it to attribute cost, debug routing decisions, and compare routed traffic with single-model baselines.
The model field doesn't show whether your own deployment or router-managed capacity served the request. To see the routing mode, the models attempted, failover, and the session affinity decision, send the Foundry-Features: ModelRouterControls=V1Preview header and read model_selection_details (preview). The header also turns on adaptive session affinity by default. See Get per-request routing details and Response fields.
Compare model router with a single model
The request is the same when you call a single model: only model changes. Use model router (auto or your deployment name) to get a well-suited model for each request. Call a model deployment by its name when you need one specific model, for example for reproducibility or benchmarking. To compare quality, cost, and latency fairly, see Evaluate model router for your workload.