Deploy and configure model router

Model router sends each request to the best model for the prompt, behind one endpoint and one model name. Auto (model router without a deployment) needs no setup. Your own model router deployment adds dedicated rate limits and your choice of routing mode, models, and content filter. In this article, you choose between the two, deploy model router, configure how it routes, and set Auto's defaults for your Foundry resource.

  • A new deployment uses model model-router, version 2025-11-18, with the Balanced routing mode, the full model pool, and 10 units of capacity (10,000 TPM), unless you change them.
  • You choose the routing mode (Balanced, Quality, or Cost) and the model subset for each deployment. Changes take up to five minutes to take effect.
  • Claude models need their own deployment in the same Foundry resource before model router can route to them.
  • Your own deployments of pool models in the same resource serve routed requests first (preview): provisioned throughput (PTU) deployments for OpenAI models, and then pay-as-you-go deployments. There's nothing to configure.
  • Admins can set Auto's routing mode, models, and content filter, or turn Auto off, for each Foundry resource (preview).

Important

Features or capabilities marked (preview) in this article are in public preview and are subject to the Supplemental Terms of Use for Microsoft Azure Previews. Previews are provided without a service-level agreement, and Microsoft doesn't recommend them for production workloads. Certain features might not be supported or might have constrained capabilities.

Choose Auto or your own deployment

Start with Auto to try model router in minutes. Deploy your own model router when you move to production or need a setting that Auto doesn't offer.

Consideration Auto (preview) Your model router deployment
Setup None. Set model to auto in the request. Deploy model-router, and then set model to your deployment name.
Rate limits and quota In West US 3, your subscription's Global Standard model router quota that isn't assigned to deployments. In every other model router region, 1% of your subscription's model router quota. See Quota for Auto. Dedicated TPM and RPM limits that you assign from your subscription's model router quota.
Routing mode and model subset Balanced across all pool models available in your region, unless an admin configures Auto for the resource. Your choice, for each deployment.
Content filter The resource's instant-access content filter. The content filter (guardrail) that you assign to the deployment.
Data processing Global Standard only. Global Standard or Data Zone Standard.
Per-request controls (preview) Supported. Supported.
Best for The fastest start, prototypes, and light traffic. Production traffic, higher limits, a custom routing mode or model subset, a custom content filter, and Data Zone Standard.

Auto and your deployments work side by side in the same resource: call Auto with model set to auto, and call a deployment by its name. For request examples, see Call model router with Responses, Chat Completions, or Messages.

Prerequisites

Deploy model router

Deploy the model-router model, version 2025-11-18. With default settings, the deployment uses the Balanced routing mode and routes across every pool model available in your region.

Don't name a deployment auto unless you want it to take over Auto traffic: requests with model set to auto go to a deployment with that name instead of to Auto.

  1. Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).
  2. Select Discover in the upper-right navigation, and then select Models in the left pane to open the model catalog.
  3. Find model-router, select it, and then select Deploy.
  4. Choose how to deploy:
    • Default settings: the Balanced routing mode and the full model pool.
    • Custom settings: deployment details such as the name, deployment type, capacity, and content filter, plus the Routing mode and Route to a subset of models options.
  5. Select Deploy. Your deployment appears under Build > Models.

Screenshot of the model router deployment pane in the Foundry portal.

To call your deployment, set model to the deployment name. See Call model router with Responses, Chat Completions, or Messages.

Choose a routing mode

The routing mode sets the tradeoff between quality and cost for every request that the deployment routes. Balanced is the default.

Mode API value How model router chooses Best for
Balanced (default) balanced Considers models within a small quality range of the best model for the prompt (for example, 1–2%) and picks the most cost-effective one. Most workloads.
Quality quality Picks the highest-quality model for the prompt and ignores cost. Critical tasks such as legal review, medical summaries, or complex reasoning.
Cost cost Considers a wider quality band (for example, 5–6%) and picks the most cost-effective one. High-volume, budget-sensitive workloads such as classification or simple Q&A.

For how each mode ranks models, see Routing modes in depth.

The routing mode and model subset options are in the new Foundry portal.

  1. In the model catalog, select model-router, and then select Deploy > Custom settings.
  2. Select a mode from the Routing mode dropdown.
  3. Select Deploy.

To change the routing mode of an existing deployment, use the REST API tab.

Screenshot of the Routing mode dropdown in the model router deployment pane.

A new routing mode takes up to five minutes to take effect. To use a different mode for one request without changing the deployment, see Choose a routing mode for one request (preview).

Route to a subset of models

A model subset limits a deployment to the models you choose. Use one to control cost, meet compliance requirements, or keep long prompts on models with large context windows.

Follow these rules when you choose a subset:

  • Choose at least two models, so model router can fail over to another model when one returns a retryable error.
  • Use the exact format, name, and version values from the supported models table.
  • Deploy each Claude model in the subset first. See Add Claude models.
  • New models aren't added to a subset automatically, so add them yourself. Deployments that use the full pool get new models automatically.
  • When a model in your subset retires, it leaves the pool. Replace it with a supported model.
  • The effective context window is the smallest context window among the models in the subset.
  1. In the model catalog, select model-router, and then select Deploy > Custom settings.
  2. In the Models subset section, select Route to a subset of models.
  3. Select the models to include. If your organization restricts models with Azure Policy, unapproved models stay visible, but you can't select them.
  4. Select Deploy.

To change the model subset of an existing deployment, use the REST API tab.

Screenshot of the Route to a subset of models option and the list of selectable models in the model router deployment pane.

A subset change takes up to five minutes to take effect. To limit one request to specific models without changing the deployment, see Limit a request to specific models (preview).

Add Claude models

A model router deployment routes to an Anthropic Claude model in its pool only after you deploy that model in the same Foundry resource, as a Global Standard deployment of the version in the supported models table. If the model router deployment uses a model subset, the subset must include the Claude model too. Claude models are provided by Anthropic, and Claude deployments have their own quotas and rate limits.

  1. Deploy the Claude model in the same Foundry resource as your model router deployment. In the Foundry portal, select Deploy > Custom settings, set Model version to the version in the supported models table, such as 1: Hosted on Anthropic infrastructure, and keep the Global region scope. The portal selects 2: Hosted on Azure by default, and model router doesn't use that version. See Deploy Claude models.
  2. If your model router deployment uses a model subset, add the Claude model to it:
    • Foundry portal: select the model under Route to a subset of models.
    • REST API: add its entry to properties.routing.models, for example {"format": "Anthropic", "name": "claude-sonnet-5", "version": "1"}.
  3. Send requests to your model router deployment, not to the Claude model's name. The model field in each response shows when a Claude model answered.

Important

If properties.routing.models lists a Claude model that isn't deployed in the same resource with a matching deployment type, the deployment request fails with an InvalidResourceProperties error. Deploy the Claude model first, and then create or update the model router deployment.

Only Claude models answer Messages API requests. If no Claude deployment is available to your model router deployment, or to Auto, a Messages request returns HTTP 400. See Use the Messages API.

As of November 2026, the Claude versions in the pool are available only as Global Standard deployments, so a Data Zone Standard model router deployment doesn't route to Claude models. To use Claude models, use a Global Standard model router deployment or Auto.

Auto can route to a Claude model when your resource has a deployment of that model. If your organization assigns the built-in approved-models policy, allow the Anthropic publisher in addition to Microsoft. See Govern model router with Azure Policy.

Use your own deployments for routed requests

Your own deployments serve routed requests first (preview). After model router picks a model, it serves the request from your own deployment of that exact model in the same Foundry resource, if you have one, in this order: your provisioned throughput (PTU) deployment for OpenAI models, and then your pay-as-you-go deployment for any model in the pool. Otherwise, it uses router-managed capacity. There's nothing to turn on.

To make one of your deployments eligible, deploy a pool model that meets every requirement in this table:

Requirement Details
Same resource The deployment is in the same Foundry resource as your model router deployment.
Exact model The model name and version match an entry in the supported models table exactly.
Deployment type For OpenAI models, with format OpenAI in the supported models table, provisioned throughput: Global Provisioned, Data Zone Provisioned, or Regional Provisioned. These deployments serve first. For every model in the pool, pay-as-you-go: Standard, Global Standard, or Data Zone Standard.
Data zone For a Data Zone Standard model router deployment, only your Data Zone Provisioned and Data Zone Standard deployments are eligible, so data stays in the data zone. Auto processes data as Global Standard.

When your deployment serves a request:

  • On a provisioned deployment, the request uses that deployment's PTUs. You pay only the model router routing fee, because you already pay for the PTUs.
  • On a pay-as-you-go deployment, the request counts against that deployment's quota and rate limits. You pay that deployment's usage, plus the routing fee. See Model router pricing.
  • The request also counts against your model router deployment's limits.
  • If your deployment returns a retryable error, such as HTTP 429, model router moves down the capacity order for the same model: from your provisioned deployment to your pay-as-you-go deployment, and then to router-managed capacity. If those also fail, it fails over to the next-ranked model.

Responses don't show whether your deployment or router-managed capacity served a request. For quota details, see Quota when your deployments serve requests. For the full capacity flow, see How model router chooses capacity.

Set capacity, content filters, and version

Each model router deployment has its own capacity, content filter, and version. These settings apply to every request the deployment routes, whichever model answers.

Set capacity

Capacity sets the deployment's rate limits in units. One unit is 1,000 TPM and 1 RPM. A new deployment gets 10 units (10,000 TPM and 10 RPM) unless you set sku.capacity or choose a different value in Custom settings.

Capacity comes from your subscription's model router quota. To change it later, edit the deployment's allocation on the Quota page in the Foundry portal, or send the deployment's PUT request with a new sku.capacity. See Change a deployment's limit.

Set a content filter

The content filter (guardrail) on a model router deployment applies to all traffic to and from it. You don't set filters for each underlying model.

Auto uses the resource's instant-access content filter instead. See Set Auto's content filter.

Update the version

Version 2025-11-18 is generally available, is the default for new deployments, and gets new models over time. Preview versions 2025-08-07 and 2025-05-19 retire on January 27, 2027. A retirement date applies to a model router version, not to model router.

Move deployments on 2025-05-19 or 2025-08-07 to 2025-11-18 before January 27, 2027. To find your model router deployments and their versions, run:

az cognitiveservices account deployment list \
  --name <resource> \
  --resource-group <resource-group> \
  --query "[?properties.model.name=='model-router'].{deployment:name, version:properties.model.version}" \
  --output table

To move a deployment, send its PUT request with "version": "2025-11-18" in properties.model. For the request format, see Update and deploy models via the API. To check a deployment's version update setting, see Model updates. For every version's retirement date, see Versions and lifecycle.

Configure Auto for your resource

Admins can set Auto's routing mode and models, turn Auto off, and choose its content filter for a whole Foundry resource. Auto and these settings are in preview; see the Supplemental Terms of Use for Microsoft Azure Previews.

The settings live in the resource's properties.instant object, with ARM API version 2026-01-15-preview:

Property What it sets
properties.instant.routing Auto's routing mode (mode) and model subset (models) for the whole resource.
properties.instant.modelAllowList The models that you can call without a deployment (instant access), including Auto.
properties.instant.raiPolicyName The content filter policy for Auto and the other instant-access models.

Apply the settings with a PATCH request (shown with az rest) or with Bicep. An update to instant resets routing and raiPolicyName if you leave them out, and keeps the current modelAllowList. So when you change one of these two settings, send the current value of the other one too. To see the current values, run the GET command that follows. Changes can take up to 15 minutes to reach every request.

Important

A Bicep deployment replaces the Foundry resource with its declaration, so it can reset properties that the declaration leaves out, such as network settings. Add the instant block to the existing declaration of your Foundry resource, with all of its current properties. Don't deploy the Bicep examples in this section on their own.

To see the current settings, run:

az rest --method get \
  --url "https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.CognitiveServices/accounts/<resource>?api-version=2026-01-15-preview" \
  --query "properties.instant"

If no one changed Auto's settings, the command returns nothing, and Auto uses its defaults: the Balanced routing mode across all pool models available in your region.

A deployment named auto takes precedence: if your resource has one, requests with model set to auto go to that deployment, and these settings don't apply to them. For how these settings fit with Azure Policy, see Govern Auto. For more about instant access, see Enterprise controls.

Set Auto's routing mode and models

Set routing.mode to balanced, quality, or cost (case-insensitive), and list the models Auto can route to in routing.models. Use the exact values from the supported models table.

  • These settings are the defaults for every Auto request on the resource. A request's routing_config takes precedence: it can set a different mode, or narrow the models further. It can't add models outside routing.models.
  • models can't be empty when it's present. Leave it out to route across all pool models available in your region.
  • Models that aren't available to Auto are ignored. Claude models also need a deployment in the resource.
  • To keep failover, list at least two models.
  • If you set a content filter for Auto, include raiPolicyName in the same request. Otherwise, the update resets it.
az rest --method patch \
  --url "https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.CognitiveServices/accounts/<resource>?api-version=2026-01-15-preview" \
  --body '{
    "properties": {
      "instant": {
        "routing": {
          "mode": "balanced",
          "models": [
            {"format": "OpenAI", "name": "gpt-6-sol", "version": "2026-09-22"},
            {"format": "OpenAI", "name": "gpt-5.6-sol", "version": "2026-07-09"},
            {"format": "OpenAI", "name": "gpt-5.6-luna", "version": "2026-07-09"}
          ]
        }
      }
    }
  }'

Turn off Auto

The modelAllowList property lists the models that the resource can call without a deployment. Auto is the {"format": "OpenAI", "name": "model-router", "version": "2025-11-18"} entry.

  • Turn off Auto: set modelAllowList to an empty array ([]). Outside West US 3, Auto is the only model that you can call without a deployment, so this setting turns off only Auto.
  • Turn off Auto in West US 3 and keep other instant-access models: set modelAllowList to the models that you want to keep, without the model-router entry. To see the current list, run the GET command in Configure Auto for your resource.
  • Turn Auto back on: add the model-router entry to the list again.

Each model in the list must support instant access in your resource's region. Otherwise, the request fails with HTTP 400 and the message Model '<format>/<name>/<version>' does not support instant inference. While Auto is off, requests with model set to auto return HTTP 400.

Caution

A new modelAllowList replaces the old one; it doesn't merge. Include every instant-access model that you want to keep in each request. After you set modelAllowList, you can replace it, but you can't remove it to return to the default list.

This request turns off Auto:

az rest --method patch \
  --url "https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.CognitiveServices/accounts/<resource>?api-version=2026-01-15-preview" \
  --body '{"properties": {"instant": {"modelAllowList": []}}}'

In West US 3, to turn off Auto but keep instant access to other models, list them instead. For example, "modelAllowList": [{"format": "OpenAI", "name": "gpt-5.6-sol", "version": "2026-07-09"}] keeps instant access to gpt-5.6-sol.

To apply the same Auto settings to many resources, add the instant block to the Bicep or ARM template that you use to create them.

Set Auto's content filter

Set raiPolicyName to the content filter policy to use, for example Microsoft.DefaultV2. The policy applies to Auto and to every other instant-access model in the resource; you can't set a separate filter for Auto alone. If you set Auto's routing mode or models, include routing in the same request. Otherwise, the update resets it. For a different filter on model router traffic, deploy model router and set the deployment's content filter.

az rest --method patch \
  --url "https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.CognitiveServices/accounts/<resource>?api-version=2026-01-15-preview" \
  --body '{"properties": {"instant": {"raiPolicyName": "Microsoft.DefaultV2"}}}'