Call model router with Responses, Chat Completions, or Messages

Model router in Microsoft Foundry gives you one endpoint and one model name for many models, and it supports the API format your app already uses. In this article, you choose an endpoint, authenticate, call model router with the Responses, Chat Completions, and Messages APIs, and confirm which model answered each request.

  • One model router, three API formats: Responses, Chat Completions, and Messages (Anthropic format, preview).
  • Set model to auto to use Auto (model router without a deployment, preview), or set it to the name of your model router deployment.
  • The model field in every response names the model that answered, for example gpt-5.6-luna-2026-07-09.
  • Only Claude models answer Messages requests, and you must deploy each one in the same Foundry resource.
  • Use the /openai/v1/ or /anthropic/v1/ path. You don't need an api-version query parameter.

Important

Features or capabilities marked (preview) in this article are in public preview and are subject to the Supplemental Terms of Use for Microsoft Azure Previews. Previews are provided without a service-level agreement, and Microsoft doesn't recommend them for production workloads. Certain features might not be supported or might have constrained capabilities.

Prerequisites

  • An Azure subscription. If you don't have one, create one for free.
  • A Foundry project in a model router region. To create one, see Create a project.
  • Auto needs no setup. To call your own model router deployment instead, deploy model router first.
  • For the Messages API, at least one Claude model from the model pool deployed in the same Foundry resource. See Add Claude models.
  • Python 3.10 or later with the OpenAI SDK (pip install openai). For the Messages API, also install the Anthropic SDK (pip install -U anthropic). For Microsoft Entra ID, also install azure-identity and the Azure CLI.
  • For JavaScript, Node.js 20 or later with the OpenAI SDK (npm install openai).

The samples in this article read these environment variables:

Variable Value
FOUNDRY_RESOURCE_ENDPOINT Your resource endpoint, https://<resource>.services.ai.azure.com
FOUNDRY_PROJECT_ENDPOINT Your project endpoint, https://<resource>.services.ai.azure.com/api/projects/<project>
FOUNDRY_API_KEY Your resource (project) API key
MODEL_ROUTER_DEPLOYMENT Your model router deployment name, when you don't use Auto

Choose an endpoint

Model router works on five endpoints, for Auto and for model router deployments. Pick the API format, and then the resource or project endpoint.

API Resource endpoint Project endpoint
Responses https://<resource>.services.ai.azure.com/openai/v1/responses https://<resource>.services.ai.azure.com/api/projects/<project>/openai/v1/responses
Chat Completions https://<resource>.services.ai.azure.com/openai/v1/chat/completions https://<resource>.services.ai.azure.com/api/projects/<project>/openai/v1/chat/completions
Messages (preview) https://<resource>.services.ai.azure.com/anthropic/v1/messages Not available
  • Resource endpoint: The simplest choice, and the only endpoint for the Messages API.
  • Project endpoint: Use it with Foundry Agent Service and other project-scoped features, and to authenticate with Microsoft Entra ID through the Foundry SDK.
  • The https://<resource>.openai.azure.com/openai/v1/ host also works for Responses and Chat Completions.

Find your endpoint and key

  1. In the Foundry portal, open your project.
  2. On the Home page, copy the Project endpoint and the Project API key.
  3. To get your resource endpoint, remove /api/projects/<project> from the project endpoint. The result has the form https://<resource>.services.ai.azure.com.

To get the key with the Azure CLI, run:

az cognitiveservices account keys list --name <resource> --resource-group <resource-group> --query key1 --output tsv

Use the project endpoint

The Foundry SDK (pip install "azure-ai-projects>=2.0.0" azure-identity) returns an OpenAI client for your project endpoint that authenticates with Microsoft Entra ID:

import os
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

project = AIProjectClient(
    endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
    credential=DefaultAzureCredential(),
)
client = project.get_openai_client()

response = client.responses.create(model="auto", input="Summarize the benefits of unit tests.")
print(f"Model: {response.model}")
print(response.output_text)

With the plain OpenAI client, set base_url to f"{os.environ['FOUNDRY_PROJECT_ENDPOINT']}/openai/v1/". The Responses and Chat Completions samples in this article then work unchanged on the project endpoint.

Authenticate

Every API accepts an API key or a Microsoft Entra ID token. The API key header differs by API.

API API key Microsoft Entra ID Token scope
Responses and Chat Completions api-key: <key> or Authorization: Bearer <key> Authorization: Bearer <token> https://ai.azure.com/.default
Messages (preview) x-api-key: <key> Authorization: Bearer <token> https://ai.azure.com/.default

With Microsoft Entra ID, the caller needs the Foundry User role (previously named Azure AI User) on the resource or project. For role assignment steps, see Role-based access control for Microsoft Foundry.

Note

The Messages API authenticates API keys with the x-api-key header, not api-key. A request that sends api-key instead of x-api-key returns HTTP 401.

To use Microsoft Entra ID, sign in with the Azure CLI (az login), and then pass a token provider to the client.

OpenAI SDK (Responses and Chat Completions)

import os
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import OpenAI

token_provider = get_bearer_token_provider(
    DefaultAzureCredential(), "https://ai.azure.com/.default"
)
client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=token_provider,
)

response = client.responses.create(model="auto", input="Summarize the benefits of unit tests.")
print(f"Model: {response.model}")

Anthropic SDK (Messages)

import os
from anthropic import AnthropicFoundry
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

token_provider = get_bearer_token_provider(
    DefaultAzureCredential(), "https://ai.azure.com/.default"
)
client = AnthropicFoundry(
    azure_ad_token_provider=token_provider,
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/anthropic",
)

message = client.messages.create(
    model="auto",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarize the benefits of unit tests."}],
)
print(f"Model: {message.model}")

Use the Responses API

The Responses API works on the resource and project endpoints. With model router, it supports streaming, function tools, structured outputs (JSON schema), reasoning effort (reasoning.effort), image input, multi-turn state with previous_response_id and conversation, stored responses (store), background mode (background: true), and built-in tools (web search, file search, code interpreter, and MCP).

Set model to auto, or replace auto with your deployment name, for example os.environ["MODEL_ROUTER_DEPLOYMENT"].

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
)

response = client.responses.create(
    model="auto",
    input="Explain retrieval-augmented generation in two sentences.",
)

print(f"Model: {response.model}")
print(response.output_text)

The Python and JavaScript samples print output similar to this example:

Model: gpt-5.6-luna-2026-07-09
Retrieval-augmented generation (RAG) retrieves passages that are relevant to a question from your own data and adds them to the prompt. The model then grounds its answer in those passages instead of relying only on its training data.

Stream a Responses API call

Set stream=True to receive events as the model generates them. The response.completed event carries the full response, including the model that answered.

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
)

stream = client.responses.create(model="auto", input="Write a haiku about routing.", stream=True)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
    elif event.type == "response.completed":
        print(f"\nModel: {event.response.model}")

Continue a conversation

Pass the earlier response's id as previous_response_id. When a request continues a conversation, model router considers the earlier turns when it ranks models for the new turn.

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
)

first = client.responses.create(model="auto", input="Suggest a name for a hiking app.")
second = client.responses.create(
    model="auto",
    previous_response_id=first.id,
    input="Now write a one-line tagline for it.",
)

print(f"Turn 1 model: {first.model}")
print(f"Turn 2 model: {second.model}")
print(second.output_text)

Use the Chat Completions API

The Chat Completions API works on the resource and project endpoints. With model router, it supports streaming, function tools, structured outputs (JSON schema), reasoning_effort, image input, and sampling parameters such as temperature. Model router drops sampling parameters for models that don't support them.

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
)

completion = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Explain retrieval-augmented generation in two sentences."}],
)

print(f"Model: {completion.model}")
print(completion.choices[0].message.content)

The Chat Completions API is stateless. To continue a conversation, append the assistant's reply and the next user message to messages, and send the full history again. Send function tools, response_format, and image content the same way you do for any Chat Completions model. When a request uses function tools or JSON schema output, model router ranks only models that can serve them.

Stream a Chat Completions call

Set stream=True, and read the text from each chunk's delta. The chunks also carry the model field.

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
)

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Name three Seattle landmarks, comma-separated."}],
    stream=True,
)

model = None
for chunk in stream:
    model = chunk.model or model
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
print(f"\nModel: {model}")

Use the Messages API

The Messages API (preview) accepts Anthropic-format requests, so the Anthropic SDK, Claude Code, and the Claude Agent SDK work with model router. Before you call it, check these requirements:

  • Claude deployments: Only the Claude models in the pool answer Messages requests. Each one must be deployed in the same Foundry resource as a Global Standard deployment of the version in the supported models table. If no Claude deployment is available to your model router deployment or to Auto, the request returns HTTP 400. See Add Claude models.
  • Endpoint: Use the resource endpoint, https://<resource>.services.ai.azure.com/anthropic/v1/messages. The Messages API has no project endpoint.
  • Authentication: Send the x-api-key header or a Microsoft Entra token. A request that sends api-key instead of x-api-key returns HTTP 401.
  • max_tokens: Required in every request.
  • anthropic-version: Defaults to 2023-06-01 when you omit it.

The Messages API supports streaming, client tools, image input, system prompts, and prompt caching with cache_control. Model router passes anthropic-beta headers through to the selected model, and support depends on that model.

import os
from anthropic import AnthropicFoundry

client = AnthropicFoundry(
    api_key=os.environ["FOUNDRY_API_KEY"],
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/anthropic",
)

message = client.messages.create(
    model="auto",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Explain retrieval-augmented generation in two sentences."}],
)

print(f"Model: {message.model}")
for block in message.content:
    if block.type == "text":
        print(block.text)

Stream a Messages call

Use the Anthropic SDK's messages.stream() helper to print text as it arrives, and then read the final message.

import os
from anthropic import AnthropicFoundry

client = AnthropicFoundry(
    api_key=os.environ["FOUNDRY_API_KEY"],
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/anthropic",
)

with client.messages.stream(
    model="auto",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Write a haiku about routing."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final = stream.get_final_message()

print(f"\nModel: {final.model}")

To run Claude Code or the Claude Agent SDK on model router, see Claude Agent SDK and Claude Code.

Feature support by API

The following table shows what model router supports in each API format. Not documented means that this article doesn't cover the feature for that API.

Feature Responses Chat Completions Messages (preview)
Resource endpoint Yes Yes Yes
Project endpoint Yes Yes No
Models that can answer Models in your pool Models in your pool Claude models deployed in your resource
Streaming Yes Yes Yes
Function or client tools Yes (function tools) Yes (function tools) Yes (client tools)
Structured outputs (JSON schema) Yes Yes Not documented
Image input Yes Yes Yes
Reasoning effort parameter reasoning.effort reasoning_effort output_config.effort
Multi-turn state previous_response_id and conversation No (send the full history in messages) No (send the full history in messages)
Stored responses Yes (store) Not documented No
Background mode Yes (background: true) No No
Built-in tools (web search, file search, code interpreter, MCP) Yes No Not documented
Prompt caching Yes Yes Yes (cache_control)
Per-request controls (preview) Yes Yes Yes
Session affinity (preview) Yes Yes Yes
Routing metadata in the response (preview) Yes Yes Yes

Prompt caching works through the underlying models: cache hits happen when the same model serves consecutive requests with overlapping prefixes. Session affinity raises the odds. For per-request controls and session affinity, see Control model router routing per request and Keep a conversation on one model with model router.

How model router handles parameters

Send the parameters you'd send to any model. By default, model router handles them in two steps:

  1. Before it ranks models, model router excludes models that can't serve what your request needs, such as function tools, JSON schema output, or image input.
  2. After it selects a model, model router drops optional parameters that the model doesn't support, and sends the rest of the request. For example, it drops temperature, top_p, presence_penalty, frequency_penalty, logit_bias, logprobs, and stop for reasoning models, and reasoning_effort for non-reasoning models.

To keep every parameter, set routing_config.require_parameters to true (preview). Model router then checks parameters before ranking, as a filter: only models that support every parameter you sent are eligible, so nothing is dropped. If no model qualifies, the request fails with HTTP 400, and no model is called.

routing_config.require_parameters Unsupported parameter Models that can answer
false (default) Dropped after model router selects a model. Every eligible model in your pool or model subset.
true (preview) Never dropped. Models that don't support it are excluded before ranking. Only models that support every parameter you sent.

For example, a request that sets temperature can go to a reasoning model by default, with temperature dropped. With require_parameters set to true, the same request goes only to models that accept temperature. See Require support for every parameter.

Reasoning effort

Send reasoning effort in each API's native field. routing_config has no reasoning-effort field.

API Request field Effort reported in the response
Chat Completions reasoning_effort No
Responses reasoning.effort reasoning.effort, for OpenAI reasoning models
Messages output_config.effort No. Claude models don't report it.

Model router passes the value to the selected model when that model supports it, and drops it for a model that doesn't, such as a non-reasoning model. With routing_config.require_parameters set to true, only models that support reasoning effort are eligible. Some values, such as minimal or xhigh, are supported by only some reasoning models, and model router doesn't route a request to a reasoning model that rejects the value. model_selection_details doesn't report the effort.

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
)

response = client.responses.create(
    model="auto",
    input="Is 221 a prime number? Answer yes or no, then give one factor pair if not.",
    reasoning={"effort": "low"},
)

print(f"Model: {response.model}")
effort = response.reasoning.effort if response.reasoning else None  # Reported by OpenAI reasoning models.
print(f"Reported effort: {effort}")
print(response.output_text)

Input and context limits

  • Model router ranks models by the text of the request. Image input is supported, but the routing decision uses only the text. Audio input isn't supported.
  • The effective context window is the smallest context window among the models in your pool. For long prompts, route to a subset of large-context models. See Route to a subset of models.

For more about how routing uses your parameters, see How parameters and reasoning effort are handled.

See which model answered

The model field in every response names the model that served the request, for example gpt-5.6-luna-2026-07-09. Read it the same way in each API:

API Python REST API Streaming
Responses response.model Top-level model response.model in the response.completed event
Chat Completions completion.model Top-level model model on the streamed chunks
Messages (preview) message.model Top-level model model on the final message from stream.get_final_message()

In JavaScript, read response.model or completion.model. In the Foundry portal, go to Build > Models, and select your model router deployment to open it in the playground. Each response in the playground shows the model that model router selected.

Tip

Log the model field with every request. Use it to attribute cost, debug routing decisions, and compare routed traffic with single-model baselines.

The model field doesn't show whether your own deployment or router-managed capacity served the request. To see the routing mode, the models attempted, failover, and the session affinity decision, send the Foundry-Features: ModelRouterControls=V1Preview header and read model_selection_details (preview). The header also turns on adaptive session affinity by default. See Get per-request routing details and Response fields.

Compare model router with a single model

The request is the same when you call a single model: only model changes. Use model router (auto or your deployment name) to get a well-suited model for each request. Call a model deployment by its name when you need one specific model, for example for reproducibility or benchmarking. To compare quality, cost, and latency fairly, see Evaluate model router for your workload.