Use model router with agents, agent frameworks, and coding tools

An agent makes many model calls to finish one task, and each call needs something different. In Microsoft Foundry, model router picks a model for each call behind one model name. A simple turn can go to a fast, low-cost model, while planning or tool orchestration goes to a stronger reasoning model. Coding tools work the same way: a quick edit goes to a fast model, and a hard refactoring goes to a frontier model. In this article, you connect model router to Foundry Agent Service, your agent framework, or your coding tool, check which model served each step, and tune routing for each agent role.

  • Set your agent's model to your model router deployment name. In LangGraph and the OpenAI Agents SDK, you can also set it to auto to use Auto (model router without a deployment, preview).
  • Model router works with Foundry Agent Service, Microsoft Agent Framework, LangGraph, the OpenAI Agents SDK, Google ADK, CrewAI, and the Claude Agent SDK.
  • It also works with coding tools: GitHub Copilot (CLI, app, Visual Studio Code, and Visual Studio), Codex, OpenCode, and Claude Code.
  • Before it ranks models, model router excludes models that can't serve what the request needs, such as function tools or JSON schema output.
  • Every response names the model that served it in the model field. Log it to see which model handled each agent step.
  • Give each agent role its own model router deployment when roles need a different routing mode or model subset.

Important

Features or capabilities marked (preview) in this article are in public preview and are subject to the Supplemental Terms of Use for Microsoft Azure Previews. Previews are provided without a service-level agreement, and Microsoft doesn't recommend them for production workloads. Certain features might not be supported or might have constrained capabilities.

Prerequisites

  • An Azure subscription. If you don't have one, create one for free.
  • A Foundry project in a region that supports model router. To create one, see Create a project.
  • A model router deployment for the Foundry Agent Service, Microsoft Agent Framework, Google ADK, CrewAI, Claude, and coding tool examples. To create one, see Deploy model router. The LangGraph and OpenAI Agents SDK examples also work with Auto.
  • Python 3.10 or later. Google ADK needs Python 3.11 or later.
  • For Microsoft Entra ID authentication, the Azure CLI (sign in with az login) and the Foundry User role on the resource or project. For details, see Role-based access control in Foundry. The Foundry Agent Service and Agent Framework Responses examples use Microsoft Entra ID.

Get your endpoint, key, and deployment name

In the Foundry portal, open your project and get these values:

  • Endpoints and API key: On the Home page, copy the Project endpoint and the Project API key. Your resource endpoint is the project endpoint without /api/projects/<project>.
  • Model router deployment name: Go to Build > Models, and copy the name of your model router deployment, such as model-router. You pass this name as model. Auto doesn't need it.

The code examples read these values from environment variables. The coding tool configurations use them in place of <resource>, <your-api-key>, and <model-router-deployment-name>.

export FOUNDRY_RESOURCE_ENDPOINT="https://<resource>.services.ai.azure.com"
export FOUNDRY_PROJECT_ENDPOINT="https://<resource>.services.ai.azure.com/api/projects/<project>"
export FOUNDRY_API_KEY="<your-api-key>"
export MODEL_ROUTER_DEPLOYMENT="<model-router-deployment-name>"  # Not needed for Auto

Choose an integration

Each framework calls model router through one of three API formats, on your resource endpoint or your project endpoint. Find your framework, and then go to its section.

Framework API format used Endpoint model value Section
Foundry Agent Service Responses Project Your model router deployment name Foundry Agent Service
Microsoft Agent Framework Chat Completions or Responses Resource (Chat Completions) or project (Responses) Your model router deployment name Microsoft Agent Framework
LangGraph Chat Completions Resource, /openai/v1/ auto (preview) or your model router deployment name LangGraph
OpenAI Agents SDK Chat Completions or Responses Resource, /openai/v1/ (Chat Completions) or project, /openai/v1/ (Responses) auto (preview) or your model router deployment name OpenAI Agents SDK
Google ADK, through LiteLLM Chat Completions Resource azure/<model-router-deployment-name> Google ADK
CrewAI, through LiteLLM Chat Completions Resource azure/<model-router-deployment-name> CrewAI
Claude Agent SDK and Claude Code Messages (preview; Claude models only) Resource, /anthropic/v1/ Your model router deployment name Claude Agent SDK and Claude Code
GitHub Copilot CLI Chat Completions Resource Your model router deployment name GitHub Copilot CLI
GitHub Copilot app Chat Completions Resource Your model router deployment name GitHub Copilot app
GitHub Copilot in Visual Studio Code Chat Completions Resource Your model router deployment name GitHub Copilot in Visual Studio Code
GitHub Copilot in Visual Studio Chat Completions Resource Your model router deployment name GitHub Copilot in Visual Studio
Codex Responses Project, /openai/v1 Your model router deployment name Codex
OpenCode Chat Completions Resource, /openai/v1 Your model router deployment name OpenCode

In the LangGraph and OpenAI Agents SDK examples, set model to auto to use Auto, or to your deployment name. The other examples call your model router deployment by name. For production agents, use a deployment: it gives you dedicated rate limits, your own routing mode and model subset, and a custom content filter.

Foundry Agent Service

Foundry Agent Service calls your agent's model through the Responses API on your project endpoint. Set your model router deployment as the agent's model in the Foundry portal or in code.

  1. In the Foundry portal, open your project, and then create or open an agent in the agent playground.
  2. Select your model router deployment from the model dropdown.
  3. Add instructions and tools as usual, and then send a test message in the playground.

As of November 2026, these Foundry Agent Service built-in tools work with model router as the agent's model:

Tool type Tools
Actions and integrations Function calling, OpenAPI, MCP, A2A (preview), browser automation, and code interpreter
Knowledge and data File search, Foundry IQ (Azure AI Search), SharePoint grounding, Fabric data agent, Fabric IQ (preview), Work IQ (preview), and memory search (preview)
Web Web search, Grounding with Bing Search, and Grounding with Bing Custom Search

Tool availability also varies by region and model. For the current matrix, see Tool support by region and model. To let model router choose Claude models for agent requests, deploy those Claude models in the same Foundry resource. To learn more about building agents, see Quickstart: Create a prompt agent.

Tip

For hands-on samples, try the Foundry Agent Lab. Its progressive agent demos use model router as the default model, from function tools and web search to code interpreter, file search, and MCP.

Microsoft Agent Framework

Microsoft Agent Framework (Python package agent-framework) works with model router through two clients: OpenAIChatCompletionClient calls the Chat Completions API on your resource endpoint, and FoundryChatClient calls the Responses API on your project endpoint.

pip install agent-framework azure-identity
import asyncio
import os
from typing import Annotated

from agent_framework.openai import OpenAIChatCompletionClient


def get_weather(city: Annotated[str, "The city to get the weather for."]) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is light rain, 12 degrees Celsius."


async def main() -> None:
    client = OpenAIChatCompletionClient(
        model=os.environ["MODEL_ROUTER_DEPLOYMENT"],
        azure_endpoint=os.environ["FOUNDRY_RESOURCE_ENDPOINT"],
        api_key=os.environ["FOUNDRY_API_KEY"],
    )
    agent = client.as_agent(
        name="weather-agent",
        instructions="You are a helpful assistant. Use the get_weather tool for weather questions.",
        tools=[get_weather],
    )
    result = await agent.run("What's the weather like in Seattle?")
    print(f"Model: {result.raw_representation.model}")  # ChatResponse for the final model call
    print(result.text)


asyncio.run(main())

The result.raw_representation is the ChatResponse for the final model call, and its model field names the served model. To use Microsoft Entra ID instead of a key, replace api_key=... with credential=DefaultAzureCredential() from azure.identity.

LangGraph

LangGraph agents call model router through the LangChain ChatOpenAI client and the Chat Completions API. Point base_url at the /openai/v1/ path on your resource endpoint.

pip install langchain-openai langgraph
import os

from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent


@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is light rain, 12 degrees Celsius."


llm = ChatOpenAI(
    model="auto",  # or your model router deployment name
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
)
agent = create_react_agent(llm, tools=[get_weather])

result = agent.invoke({"messages": [("user", "What's the weather like in Seattle?")]})
for message in result["messages"]:
    if message.type == "ai":
        print(f"Model: {message.response_metadata.get('model_name')}")
print(result["messages"][-1].content)

Each AI message's response_metadata["model_name"] names the model that served that step. To use Microsoft Entra ID, install azure-identity, import DefaultAzureCredential and get_bearer_token_provider from azure.identity, and pass api_key=get_bearer_token_provider(DefaultAzureCredential(), "https://ai.azure.com/.default").

OpenAI Agents SDK

The OpenAI Agents SDK calls model router through an AsyncOpenAI client. Use OpenAIChatCompletionsModel with your resource endpoint for the Chat Completions API, or use OpenAIResponsesModel with your project endpoint for the Responses API.

pip install openai-agents

The Agents SDK doesn't expose the response's model field, so the following examples record the served model by using an HTTP response hook on the client.

Define the response hook and a function tool:

import asyncio
import os

from agents import Agent, OpenAIChatCompletionsModel, Runner, function_tool, set_tracing_disabled
from openai import AsyncOpenAI, DefaultAsyncHttpxClient

served_models = []


async def record_served_model(response):
    """Record the served model from each Chat Completions response."""
    if response.request.url.path.endswith("/chat/completions") and response.status_code == 200:
        await response.aread()
        served_models.append(response.json().get("model"))


@function_tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is light rain, 12 degrees Celsius."

Create the client and the agent, and then run it:

async def main() -> None:
    set_tracing_disabled(True)  # Don't export traces to the OpenAI platform.
    client = AsyncOpenAI(
        base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
        api_key=os.environ["FOUNDRY_API_KEY"],
        http_client=DefaultAsyncHttpxClient(event_hooks={"response": [record_served_model]}),
    )
    agent = Agent(
        name="weather-agent",
        instructions="You are a helpful assistant. Use the get_weather tool for weather questions.",
        tools=[get_weather],
        model=OpenAIChatCompletionsModel(model="auto", openai_client=client),  # or your deployment name
    )
    result = await Runner.run(agent, "What's the weather like in Seattle?")
    print(f"Models: {served_models}")
    print(result.final_output)


asyncio.run(main())

served_models lists the model that served each call in the run. To use Microsoft Entra ID, install azure-identity, import DefaultAzureCredential and get_bearer_token_provider from azure.identity.aio (AsyncOpenAI needs an async token provider), and pass api_key=get_bearer_token_provider(DefaultAzureCredential(), "https://ai.azure.com/.default").

Google ADK

Google ADK calls model router through the LiteLLM azure/ provider, which uses the Chat Completions API and calls your deployment by name. Set the model to azure/ followed by your model router deployment name, for example azure/model-router.

pip install google-adk litellm

Define the agent:

import asyncio
import os

from google.adk.agents import Agent
from google.adk.models.lite_llm import LiteLlm
from google.adk.runners import InMemoryRunner
from google.genai import types


def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is light rain, 12 degrees Celsius."


agent = Agent(
    name="weather_agent",
    model=LiteLlm(
        model=f"azure/{os.environ['MODEL_ROUTER_DEPLOYMENT']}",
        api_base=os.environ["FOUNDRY_RESOURCE_ENDPOINT"],
        api_version="2024-10-21",
        api_key=os.environ["FOUNDRY_API_KEY"],
    ),
    instruction="You are a helpful assistant. Use the get_weather tool for weather questions.",
    tools=[get_weather],
)

Run the agent, and print the model that served each model call:

async def main() -> None:
    runner = InMemoryRunner(agent=agent, app_name="model-router-demo")
    session = await runner.session_service.create_session(app_name="model-router-demo", user_id="user")
    message = types.Content(role="user", parts=[types.Part(text="What's the weather like in Seattle?")])
    async for event in runner.run_async(user_id="user", session_id=session.id, new_message=message):
        if event.model_version:  # ADK copies the response's model field into model_version.
            print(f"Model: {event.model_version}")
        if event.is_final_response() and event.content and event.content.parts:
            print(event.content.parts[0].text)


asyncio.run(main())

event.model_version names the model that served each call. To use Microsoft Entra ID, install azure-identity, import DefaultAzureCredential and get_bearer_token_provider from azure.identity, and replace api_key with azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), "https://ai.azure.com/.default").

CrewAI

CrewAI calls model router through the LiteLLM azure/ provider, which uses the Chat Completions API and calls your deployment by name. Set the model to azure/ followed by your model router deployment name, for example azure/model-router.

pip install crewai litellm

Register your deployment with LiteLLM, and add a callback that records the served model:

  • LiteLLM's model map lists azure/model-router without function calling, which makes CrewAI fall back to text-based tool use. The register_model call marks your deployment as supporting native tool calls.
  • CrewAI replaces litellm.callbacks with its own handlers, so the example adds its callback to litellm.success_callback instead.
import os
import time

from crewai import LLM, Agent, Crew, Task
from crewai.tools import tool
import litellm

deployment = os.environ["MODEL_ROUTER_DEPLOYMENT"]
served_models = []


def record_served_model(kwargs, completion_response, start_time, end_time):
    served_models.append(getattr(completion_response, "model", None))


litellm.success_callback.append(record_served_model)
litellm.register_model(
    {f"azure/{deployment}": {**litellm.model_cost.get("azure/model-router", {}), "litellm_provider": "azure",
                             "mode": "chat", "supports_function_calling": True}}
)

Create the agent and the task, and then run the crew:

@tool("get_weather")
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is light rain, 12 degrees Celsius."


llm = LLM(
    model=f"azure/{deployment}",
    is_litellm=True,
    base_url=os.environ["FOUNDRY_RESOURCE_ENDPOINT"],
    api_key=os.environ["FOUNDRY_API_KEY"],
    api_version="2024-10-21",
)
agent = Agent(
    role="Weather assistant",
    goal="Answer weather questions by calling the get_weather tool.",
    backstory="You give short, factual weather answers.",
    tools=[get_weather],
    llm=llm,
)
task = Task(description="What's the weather like in Seattle?", expected_output="One sentence.", agent=agent)
result = Crew(agents=[agent], tasks=[task]).kickoff()
time.sleep(1)  # LiteLLM runs success callbacks on a background thread.

print(f"Models: {served_models}")
print(result.raw)

CrewAI doesn't return the served model, so the LiteLLM success callback collects it from each response in served_models.

Claude Agent SDK and Claude Code

Claude Code and the Claude Agent SDK call model router through the Messages API on your resource endpoint, https://<resource>.services.ai.azure.com/anthropic/v1/messages. Only the Claude models in your model router deployment answer these requests, and each one must be deployed in the same Foundry resource. If none of them is deployed, requests return HTTP 400.

Use a model router deployment, not Auto, for coding agents. Coding agents send many large requests and need higher rate limits.

  1. Deploy each Claude model that you want model router to use in the same Foundry resource, as a Global Standard deployment of the version in the supported models table. The Foundry portal selects the Azure-hosted version by default, and model router doesn't use it. See Deploy Claude models.

  2. Deploy model router as a Global Standard deployment, with the full model pool or with a model subset that includes those Claude models. See Add Claude models.

  3. Set the Foundry environment variables. ANTHROPIC_FOUNDRY_RESOURCE is your Foundry resource name, the <resource> part of your endpoint. Claude Code picks a model tier for each kind of work: Opus for complex reasoning, Sonnet for everyday coding, and Haiku for simple and background tasks. Set all three tiers to your model router deployment name, so that every request goes to model router, which then picks the Claude model for that request:

    export CLAUDE_CODE_USE_FOUNDRY=1
    export ANTHROPIC_FOUNDRY_RESOURCE="<resource>"
    export ANTHROPIC_DEFAULT_OPUS_MODEL="<model-router-deployment-name>"
    export ANTHROPIC_DEFAULT_SONNET_MODEL="<model-router-deployment-name>"
    export ANTHROPIC_DEFAULT_HAIKU_MODEL="<model-router-deployment-name>"
    
  4. Authenticate. Sign in with az login to use Microsoft Entra ID, or set your API key:

    export ANTHROPIC_FOUNDRY_API_KEY="<your-api-key>"
    
  5. Start Claude Code in your project directory with claude.

For the Claude Agent SDK, configure it the same way as Claude Code, with the Foundry environment variables. For installation, the Visual Studio Code extension, and validation steps, see Configure Claude Code for Microsoft Foundry.

Each Messages API response names the Claude model that served it in the model field. To see the mix of Claude models across a coding session, use your deployment's Monitor tab, described in Observe routing in agent runs.

Claude Code might warn that it doesn't recognize your deployment name and assume a 200,000-token context window. Requests still go through model router.

GitHub Copilot

GitHub Copilot can use your model router deployment as a custom model, with bring your own key (BYOK), in the Copilot CLI, the GitHub Copilot app, Visual Studio Code, and Visual Studio. Copilot calls it through the Chat Completions API on your resource endpoint. As with every coding tool, use a model router deployment rather than Auto, because coding agents send many large requests and need higher rate limits.

Copilot shows your deployment name as the model for every response. To see which models served your requests, use your deployment's Monitor tab. If you use Copilot Business or Copilot Enterprise, your organization's policy can turn off bring your own key. For all Copilot clients that support it, see Bring your own key.

GitHub Copilot CLI

Add your model router deployment to the providers.json file in your Copilot CLI configuration directory, usually ~/.copilot. Create the file if it doesn't exist. If it already has providers and models, add these entries to them. Set wireModel to your model router deployment name:

{
  "providers": [
    {
      "name": "foundry",
      "type": "azure",
      "baseUrl": "https://<resource>.services.ai.azure.com",
      "apiKey": "<your-api-key>"
    }
  ],
  "models": [
    {
      "id": "model-router",
      "provider": "foundry",
      "wireModel": "<model-router-deployment-name>"
    }
  ]
}

The file stores your API key as plain text, so keep it private and out of source control. Then start Copilot CLI in your project directory with model router selected:

copilot --model foundry/model-router

To switch models during a session, use the /model command. With your own provider, Copilot CLI doesn't need a GitHub sign-in. For all provider settings, see Adding LLM models to GitHub Copilot CLI.

GitHub Copilot app

In the GitHub Copilot app, open Settings, select Model providers > Add provider, and then select Microsoft Foundry. Enter your resource endpoint and API key, and save the provider. Then choose your model router deployment in the model picker. You don't need a Copilot plan to use your own provider. For details, see Adding LLM models to the GitHub Copilot app.

GitHub Copilot in Visual Studio Code

In the Chat view, open the model picker, select Manage Language Models, and add your model router deployment with the Azure provider. Use your resource endpoint, your API key, and your model router deployment name. To make model router available to agents, set toolCalling to true for it in the chatLanguageModels.json file where VS Code stores the model's settings. For details, see Bring your own language model key.

GitHub Copilot in Visual Studio

In Visual Studio 2026 version 18.10 or later, bring your own key is in preview and works with the Agent (Preview) experience. In Chat, open the Model picker, select Add a model, and connect to Microsoft Foundry with your resource endpoint, API key, and model router deployment name. Then start working in Agent (Preview). For details, see Bring your own key in Visual Studio.

Codex

Codex calls model router through the Responses API. Add a model provider to your Codex configuration file, ~/.codex/config.toml, and set model to your model router deployment name:

model = "<model-router-deployment-name>"
model_provider = "foundry"

[model_providers.foundry]
name = "Microsoft Foundry"
base_url = "https://<resource>.services.ai.azure.com/api/projects/<project>/openai/v1"
env_key = "FOUNDRY_API_KEY"
wire_api = "responses"

Set your API key, and then start Codex in your project directory:

export FOUNDRY_API_KEY="<your-api-key>"
codex

Codex might warn that it doesn't have metadata for your deployment name and use fallback settings. Requests still go through model router. To see which models served your requests, use your deployment's Monitor tab.

OpenCode

OpenCode calls model router through the Chat Completions API. Add a provider to opencode.json in your project directory, or to ~/.config/opencode/opencode.json for every project. Use your model router deployment name as the model key and in model:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "foundry": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Microsoft Foundry",
      "options": {
        "baseURL": "https://<resource>.services.ai.azure.com/openai/v1",
        "apiKey": "{env:FOUNDRY_API_KEY}"
      },
      "models": {
        "<model-router-deployment-name>": { "name": "Model router" }
      }
    }
  },
  "model": "foundry/<model-router-deployment-name>"
}

Set your API key, and then start OpenCode in your project directory:

export FOUNDRY_API_KEY="<your-api-key>"
opencode

Routing patterns for agents

Model router chooses a model for each call, so one agent run can use several models. The following diagram shows how different steps of one run can go to different models.

Diagram of one agent run with three model calls. Model router sends a greeting to a fast, low-cost model, a tool call to a model that supports the requested tools, and a planning step to a stronger reasoning model.

The following table describes typical routing for common agent scenarios, with a routing mode and model subset to start from. Specific model selections change over time as models join and leave the pool.

Scenario Typical routing Routing mode Model subset idea
Greetings, factual lookups, and simple follow-ups Fast, low-cost models, even when the agent has tools attached Balanced (default) Full model pool
Classification, triage, and simple Q&A at high volume Fast, low-cost models Cost Smaller, lower-cost models
Tool orchestration, such as function calls, web search, and code execution Models that produce valid tool calls; straightforward calls go to capable mid-tier models Balanced Full model pool; model router already skips models that can't serve the request's tools
Retrieval-augmented generation and document synthesis Higher-capability models, because synthesis across documents needs more reasoning Balanced or Quality Large-context models, because the effective context window is the smallest one in the pool
Summarization Models suited to summarization Balanced Full model pool
Multi-step orchestration, such as chained tool calls and MCP servers The most capable models Quality Models you validated for multi-step reasoning
High-stakes answers, such as legal or medical The highest-quality model for each prompt Quality Models your organization approved

Use one deployment per agent role

When agent roles need different routing modes or model subsets, create one model router deployment per role, and set each agent's model to its role's deployment. Each agent then makes its own routing decisions within its own model subset.

Deployment Routing mode Model subset Agent
router-research Quality gpt-6.1-sol, gpt-6-sol, claude-opus-5 Research agent that synthesizes complex information
router-assistant Balanced Full model pool General assistant for mixed conversations
router-triage Cost gpt-5.6-luna, gpt-5.4-mini, gpt-5.4-nano Triage agent that classifies incoming requests

In the deployment request, the routing mode and subset go in properties.routing. For example, the router-triage deployment uses this object:

{
  "mode": "cost",
  "models": [
    { "format": "OpenAI", "name": "gpt-5.6-luna", "version": "2026-07-09" },
    { "format": "OpenAI", "name": "gpt-5.4-mini", "version": "2026-03-17" },
    { "format": "OpenAI", "name": "gpt-5.4-nano", "version": "2026-03-17" }
  ]
}

To create these deployments, see Choose a routing mode and Route to a subset of models. Keep these rules in mind:

  • Include at least two models in each subset so that failover still works.
  • Subsets don't get new models automatically. Add new models to a subset yourself.
  • Claude models in a subset need their own deployment in the same Foundry resource.
  • To change the routing mode or the models for one request instead of a whole deployment, use per-request controls (preview).

Before you change routing for an agent, evaluate the change with representative multi-turn traces from your workload, including tool calls, retrieval, and handoffs. Keep the baseline, instructions, tools, and trace set fixed, and change one routing setting at a time. For guidance, see Evaluate model router.

Observe routing in agent runs

See which models your agents use, how often failover happens, and what routing costs.

Log the model for each call

Every response names the model that served it in the model field, for example gpt-5.6-luna-2026-07-09. Log it with each agent step. The following table shows where each framework exposes it.

Framework Where to read the served model
Foundry Agent Service response.model
Microsoft Agent Framework, Chat Completions result.raw_representation.model
Microsoft Agent Framework, Responses result.raw_representation.raw_representation.model
LangGraph message.response_metadata["model_name"] on each AI message
OpenAI Agents SDK The model field of each HTTP response, read with a response hook on the client
Google ADK event.model_version
CrewAI completion_response.model in a LiteLLM success callback
Claude Agent SDK and Claude Code Your deployment's Monitor tab
GitHub Copilot, Codex, and OpenCode Your deployment's Monitor tab

Use the Monitor tab

In the Foundry portal, go to Build > Models, select your model router deployment, and then select the Monitor tab. When each agent role has its own deployment, these views show routing for that role:

  • Requests by model: the share of requests routed to each model.
  • Request latency: Model processing, Router overhead, and End-to-end latency, with P50 and P95.
  • Fallback rate over time and Fallback by source model: how often model router retried a request on another model after a service error.
  • Estimated cost: model router vs. single-model baselines: what the same traffic would have cost if every request had gone to one model.

For details, see Use the Monitor tab.

Inspect routing decisions per request (preview)

Add the Foundry-Features: ModelRouterControls=V1Preview header to a request to get model_selection_details.model_router_details in the response. It shows the routing mode that model router applied and a routing_trace of the models it attempted. More than one attempt means that failover happened.

When you create the OpenAI client yourself, as in the OpenAI Agents SDK examples, set the header once with default_headers on the client. With the header, adaptive session affinity is on by default, so a conversation stays on the previous model while that model's prompt cache is likely still warm and its expected quality is close to the best-ranked model. For details, see Get per-request routing details and Keep a conversation on one model.