Quickstart: Get started with model router

Model router in Microsoft Foundry picks the best model for each request, so you write your code once and stop choosing models by hand. In this quickstart, you call model router with Auto (model router without a deployment), so your first request needs only an endpoint and a key. Then you deploy your own model router for production traffic.

  • Auto (preview) is ready in every model router region, with nothing to enable or deploy.
  • Set model to auto in a Responses, Chat Completions, or Messages (preview) request.
  • By default, Auto routes with the Balanced routing mode, and every response's model field names the model that answered.
  • To move to your own deployment, change model from auto to your deployment name. The rest of your code stays the same.

Important

Features or capabilities marked (preview) in this article are in public preview and are subject to the Supplemental Terms of Use for Microsoft Azure Previews. Previews are provided without a service-level agreement, and Microsoft doesn't recommend them for production workloads. Certain features might not be supported or might have constrained capabilities.

Prerequisites

Important

The Foundry RBAC roles were recently renamed. Foundry User, Foundry Owner, Foundry Account Owner, and Foundry Project Manager were previously named Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. You might still see the previous names in some places while the rename rolls out. The role IDs and core permissions are unchanged by the rename.

Get your endpoint and key

The examples in this quickstart call your Foundry resource endpoint with an API key.

  1. Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).
  2. Open your project.
  3. On the Home page, copy the Project endpoint and the Project API key.
  4. Remove /api/projects/<project> from the end of the project endpoint. What's left is your resource endpoint, in the form https://<resource>.services.ai.azure.com, where <resource> is your Foundry resource name.

The project endpoint also works for the Responses and Chat Completions APIs. For the full list of endpoints, see Choose an endpoint.

To get the key with the Azure CLI instead, run this command:

az cognitiveservices account keys list --name <resource> --resource-group <resource-group> --query key1 --output tsv

To read the key, you need a role on the Foundry resource that includes the Microsoft.CognitiveServices/accounts/listkeys/action permission, such as Foundry User, Cognitive Services Contributor, or Contributor. Of these roles, only Foundry User also lets you call model router with Microsoft Entra ID.

Set two environment variables that the code in this quickstart reads. Use the endpoint without a trailing slash, and keep the key out of source code.

export FOUNDRY_RESOURCE_ENDPOINT="https://<resource>.services.ai.azure.com"
export FOUNDRY_API_KEY="<your-api-key>"

The REST API examples in this quickstart use bash.

Send your first request with Auto

Send a Responses API request with model set to auto. Model router reads the prompt, picks a model, and returns that model's answer.

Install the OpenAI library:

pip install openai

Save the following code as quickstart.py:

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
)

response = client.responses.create(
    model="auto",
    input="Explain retrieval-augmented generation in two sentences.",
)

print(f"Model: {response.model}")
print(response.output_text)

Run the code:

python quickstart.py

The output looks similar to this example:

Model: gpt-5.6-luna-2026-07-09
Retrieval-augmented generation (RAG) retrieves relevant content from a knowledge source and adds it to the prompt. The model then grounds its answer in that content, which makes responses more accurate and current.

The model in your output is the one that model router picked for your prompt, so it can differ from the example. Try a harder prompt, such as a multistep math or coding problem, and see which model answers.

Use Microsoft Entra ID instead of an API key

For keyless authentication in Python, install the Azure Identity library, sign in with az login, and create the client with a token provider. The caller needs the Foundry User role. The rest of the code doesn't change.

pip install azure-identity
az login
import os
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import OpenAI

token_provider = get_bearer_token_provider(
    DefaultAzureCredential(), "https://ai.azure.com/.default"
)
client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=token_provider,
)

For Microsoft Entra ID with the REST API or the Anthropic SDK, see Authenticate.

Use Chat Completions or Messages

Auto also works with the Chat Completions and Messages APIs. Use the API format that your app already uses.

Python

Install the OpenAI library (pip install openai), and then run this code:

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
)

completion = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Explain retrieval-augmented generation in two sentences."}],
)

print(f"Model: {completion.model}")
print(completion.choices[0].message.content)

REST API

curl -X POST "$FOUNDRY_RESOURCE_ENDPOINT/openai/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "api-key: $FOUNDRY_API_KEY" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Explain retrieval-augmented generation in two sentences."}]
  }'

The response's model field names the model that answered, and choices[0].message.content holds the generated text.

For streaming, tools, multi-turn conversations, and other features, see Call model router with Responses, Chat Completions, or Messages.

See which model answered

Model router tells you which model served every request:

  • In code: The model field in every Responses, Chat Completions, and Messages response names the model that served the request, for example gpt-5.6-luna-2026-07-09. Log it to track where your traffic goes.
  • In the Foundry portal: In the playground, each response shows the selected model. To open the playground for a model router deployment, go to Build > Models, and select the deployment.
  • On the Monitor tab: For a model router deployment, Requests by model shows the share of requests that went to each model.

For more detail about each routing decision, add the Foundry-Features: ModelRouterControls=V1Preview header (preview). The response then includes model_selection_details, with the routing mode that model router applied and the models it attempted. The header also turns on session affinity with the adaptive strategy, which can keep later turns of a conversation on the same model. See Keep a conversation on one model with model router. The following example reuses the client from the Python example in Send your first request with Auto:

response = client.responses.create(
    model="auto",
    input="Explain retrieval-augmented generation in two sentences.",
    extra_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)

details = (response.model_extra or {}).get("model_selection_details", {})
router = details.get("model_router_details", {})
print(f"Model: {response.model}")
print(f"Mode: {router.get('mode')}")
for entry in router.get("routing_trace", []):
    print(f"Attempts: {[attempt['model'] for attempt in entry['attempts']]}")

The output looks similar to this example. More than one model in an attempts list means that failover happened.

Model: gpt-5.6-luna-2026-07-09
Mode: balanced
Attempts: ['gpt-5.6-luna']

For every field, see Get per-request routing details.

Deploy your own model router

Auto is the fastest way to start. Deploy your own model router when you need:

  • Production traffic with dedicated rate limits (TPM and RPM).
  • A custom routing mode or model subset.
  • A custom content filter.
  • Data Zone Standard, so data is processed within the data zone.
  • Higher limits than Auto's. Outside West US 3, Auto's limit is 1% of your subscription's model router quota (10,000 TPM and 10 RPM at Tier 1). See Quota for Auto.

A new deployment uses 10 units (10,000 TPM) of your subscription's model router quota by default.

  1. Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).
  2. Open your project.
  3. Select Discover in the upper-right navigation, and then select Models in the left pane to open the model catalog.
  4. Find model-router, select it, and then select Deploy.
  5. Choose how to deploy:
    • Default settings: the Balanced routing mode and the full model pool.
    • Custom settings: your choice of routing mode, Route to a subset of models, capacity, and content filter. The routing mode and subset options are in the new Foundry portal.
  6. Select Deploy. Your deployment appears under Build > Models. Its name is the value you pass as model.

Screenshot of the model router deployment pane in the Foundry portal.

To choose a routing mode or a model subset, or to deploy with the REST API or Bicep, see Choose a routing mode, Route to a subset of models, and Deploy model router.

Call your deployment

To call your deployment, replace auto with your deployment name. Nothing else changes. For example, save the deployment name in an environment variable. If you named your deployment something other than model-router, use that name.

export MODEL_ROUTER_DEPLOYMENT="model-router"

In PowerShell, run $env:MODEL_ROUTER_DEPLOYMENT = "model-router" instead. Then change the model value in the Python example:

response = client.responses.create(
    model=os.environ["MODEL_ROUTER_DEPLOYMENT"],
    input="Explain retrieval-augmented generation in two sentences.",
)

Make the same change to model in the JavaScript, REST API, Chat Completions, and Messages examples. If you name a deployment auto, requests with model set to auto go to that deployment instead of to Auto.

Explore samples

To see how model router performs on your own prompts, use the Model Router Auto Evaluation sample (Python). It sends your prompts to model router and to a baseline model, and then reports quality, scored by an LLM judge, along with cost, latency, and which models model router selected, in one HTML dashboard. Start with its demo, which needs no API keys, and then point it at your model router deployment.

For tested code that connects model router to agent frameworks and coding tools, see Use model router with agents, agent frameworks, and coding tools.

Important

Samples are for learning and experimentation, and they aren't production-ready. Before you deploy code derived from a sample, review it against your organization's security, compliance, and responsible AI policies.

Clean up resources

Auto doesn't create any resources, so there's nothing to clean up. If you deployed model router and no longer need it, delete the deployment to free its model router quota. If you deployed a Claude model for the Messages API, delete that deployment the same way.

In the Foundry portal, go to Build > Models, select your model router deployment, and then select Delete.

Next steps