Use model router with AI Gateway

If your organization routes model traffic through AI Gateway, automatic model selection brings model router from Microsoft Foundry to your gateway's model aliases. Your apps keep calling one alias. For each request, model router ranks the models behind the alias for that prompt, and the gateway combines that ranking with your priorities and weights to choose the backend that serves it. Your gateway's access controls, policies, and telemetry keep working as they do today. This article explains how automatic model selection works, what you configure on an alias, and when to use it instead of calling model router directly.

  • AI Gateway is generally available. Its automatic model selection feature is in preview.
  • Model router ranks the alias's models for each prompt, with the alias's selection mode: Balanced, Quality, or Cost.
  • The gateway applies its own controls and routing rules: authorization, policies such as token and cost limits, priorities, and weights.
  • Your backends serve the requests, wherever they run: deployments in Foundry or Azure OpenAI, model providers such as OpenAI and Anthropic, other clouds such as Amazon Bedrock and Google Vertex AI, or your own OpenAI-compatible endpoints, including self-hosted open models. Each backend must run a model that model router supports.
  • Automatic model selection is available in every AI Gateway region and uses API version 2026-11-01.
  • AI Gateway has its own charges. There's no model router routing fee while automatic model selection is in preview, and your backends' usage charges still apply.

Important

Features or capabilities marked (preview) in this article are in public preview and are subject to the Supplemental Terms of Use for Microsoft Azure Previews. Previews are provided without a service-level agreement, and Microsoft doesn't recommend them for production workloads. Certain features might not be supported or might have constrained capabilities.

How automatic model selection works

In AI Gateway, a model alias is the stable name that your apps call. Behind the alias are registered models from your model providers: deployments in Foundry or Azure OpenAI, and models from model providers such as OpenAI and Anthropic, other clouds such as Amazon Bedrock and Google Vertex AI, or your own OpenAI-compatible endpoints. Each registered model behind the alias has a priority, where a lower number is preferred, and a weight. When you turn on automatic model selection for an alias, model router ranks the alias's models for each request, and the gateway applies its controls and routing rules around that ranking. The following diagram shows how they work together.

Diagram of automatic model selection in AI Gateway. An app calls a model alias. The gateway applies its controls, model router ranks the alias's models, precedence and weights pick a backend, and that backend serves the request.

For each request:

  1. The gateway applies its controls. It authorizes the caller with Microsoft Entra ID or a gateway runtime API key, and applies your gateway policies, such as token limits and cost limits.
  2. Model router ranks the models. The gateway sends the request's prompt and the alias's models to model router. Model router judges the prompt and returns the models ranked for it, according to the alias's selection mode:
    • Balanced: balances quality and cost across simpler and harder requests.
    • Quality: favors the highest-quality eligible model, regardless of cost.
    • Cost: favors lower-cost models, with a quality tradeoff on harder requests.
  3. Precedence combines the ranking with your priorities.
    • Model first: the gateway chooses the best-fit model across all priority groups. If that model appears in more than one group, the gateway uses the highest-priority group.
    • Priorities first: the gateway chooses a model only from the highest-priority group.
  4. Weights pick the backend. When the chosen model has several backends at that priority, such as deployments in two regions, the weights split traffic among them.
  5. Your backend serves the request. The request runs on your backend, wherever it's hosted: a Foundry or Azure OpenAI deployment, a model provider such as OpenAI or Anthropic, another cloud such as Amazon Bedrock or Google Vertex AI, or your own self-hosted endpoint. It counts against that backend's quota and is billed at that backend's price. Model router ranks the models, but it doesn't serve the request.

For example, suppose an alias has a provisioned throughput deployment of gpt-5.6-luna at priority 0, because you already pay for that capacity, and pay-as-you-go deployments of gpt-5.6-luna and gpt-5.6-sol at priority 1. For a simple prompt, model router ranks gpt-5.6-luna highest, and both precedence options send the request to your provisioned deployment. For a hard prompt, model router ranks gpt-5.6-sol highest:

  • With Model first, the gateway sends the request to gpt-5.6-sol at priority 1.
  • With Priorities first, the gateway sends the request to gpt-5.6-luna at priority 0, because it's the only model in the highest-priority group.

Choose Model first when you want the best model for each prompt and use priorities to prefer your own capacity for that model. Choose Priorities first when keeping traffic on your preferred capacity matters more than the per-prompt choice.

What you configure on an alias

Setting in the AI Gateway portal Alias property What it does
Choose a model based on the prompt autoModelSelection Turns on automatic model selection, so model router ranks the alias's models for each request.
Selection mode: Balanced, Quality, or Cost autoModelSelection.objective Sets how model router trades quality against cost. The values Balanced, Quality, and Cost are case-sensitive.
Precedence: Model first or Priorities first selectionOrder ["autoModelSelection", "priority"] for Model first, or ["priority", "autoModelSelection"] for Priorities first.
Selector model for each backend model backendModels[].selectorModelName The model router name of the backend's model, without a version, such as gpt-5.6-luna. Only models that model router supports can be used. To use another model, turn off automatic model selection for the alias.
Priority and weight for each backend model backendModels[].priority, backendModels[].weight Priority is a whole number from 0 to 100, where a lower number is preferred. Weight is a whole number from 1 to 100 that splits traffic among backends of the same model at the same priority.

For example, these alias properties turn on automatic model selection with Model first and the Balanced mode over two Foundry deployments. Each resourceId is the Azure resource ID of a registered model in your gateway, which ends in /modelProviders/<provider-name>/models/<model-name>.

{
  "properties": {
    "aliasName": "chat-auto",
    "backendModels": [
      { "resourceId": "<registered-model-resource-id-for-gpt-5.6-luna>", "selectorModelName": "gpt-5.6-luna", "priority": 0, "weight": 1 },
      { "resourceId": "<registered-model-resource-id-for-gpt-5.6-sol>", "selectorModelName": "gpt-5.6-sol", "priority": 0, "weight": 1 }
    ],
    "selectionOrder": ["autoModelSelection", "priority"],
    "autoModelSelection": { "objective": "Balanced" }
  }
}

The models behind an alias must share at least one API format, such as Chat Completions. The alias offers only the formats that all its models support, so a model that supports only Chat Completions removes the Responses API from the whole alias.

What AI Gateway adds

Automatic model selection runs inside the gateway that your platform team already manages, so routed traffic gets the same controls as the rest of your model traffic:

  • Any provider behind one alias: model router can choose among your own deployments in Foundry or Azure OpenAI, models served directly by model providers such as OpenAI and Anthropic, models on other clouds such as Amazon Bedrock and Google Vertex AI, and your own OpenAI-compatible endpoints, including self-hosted open models. Model router ranks them all the same way, as long as it supports the model.
  • Access: apps authenticate with Microsoft Entra ID, with roles that you can scope to one alias, or with a gateway runtime API key. Apps never hold your providers' credentials.
  • Usage and spend limits: policies such as token limits and cost limits cap usage and estimated spend. Cost limits can differ by API key or by Microsoft Entra identity.
  • Capacity preference: priorities let you prefer capacity that you already pay for, such as provisioned throughput. Weights split each model's traffic across its backends, such as deployments in two regions or from two providers.
  • Observability: with a telemetry destination such as Application Insights, gateway traces show each routed request's selection mode, the ranked models, the chosen model, and how long ranking took, alongside the gateway's logs and estimated cost metrics.

When to use AI Gateway or model router directly

Call model router directly, with Auto (model router without a deployment) or your own model router deployment, when you build apps and agents on Foundry. Use automatic model selection in AI Gateway when you already govern model traffic through a gateway alias, and you want model router to choose among your own backends, including backends outside Foundry. The following comparison is current as of November 17, 2026.

Consideration Call model router directly AI Gateway automatic model selection (preview)
Best for Apps and agents that you build on Foundry Teams that already govern model traffic through an AI Gateway alias
What your app calls auto (Auto, preview) or your model router deployment name, on your Foundry resource or project endpoint Your AI Gateway alias
Models that can be chosen The model router model pool (43 models as of November 11, 2026) The models behind the alias, from any provider: Foundry or Azure OpenAI deployments, model providers such as OpenAI and Anthropic, other clouds such as Amazon Bedrock and Google Vertex AI, or your own OpenAI-compatible endpoints. Model router must support each model.
Who serves each request Your own deployment of the selected model in the same Foundry resource, when one exists (preview): provisioned throughput (PTU) first for OpenAI models, and then pay-as-you-go. Router-managed capacity otherwise. Always one of your backends, wherever it runs: in Foundry, at a model provider, on another cloud, or on your own endpoint. The alias's precedence, priorities, and weights choose which one.
How you steer routing Routing mode (Balanced, Quality, or Cost) and model subset on your deployment, or Auto's settings on your Foundry resource The alias's selection mode, precedence, backend models, priorities, and weights
Per-request controls Available in preview: change the routing mode or models, require support for every parameter, and set session affinity for one request Set on the alias
Multi-turn Responses API Supported, including previous_response_id and conversation. Model router ranks every turn. A follow-up turn that uses previous_response_id goes to the backend that served the earlier turn, without ranking.
Tools and parameters Skips models that can't serve the request's tools or JSON schema output, and drops optional parameters that the selected model doesn't support Skips models that can't serve the request's function tools. A parameter that the chosen model doesn't support returns an error.
Governance Content filters on your deployment, model subsets, Azure Policy, and Auto admin controls on your Foundry resource Gateway access control, policies such as token and cost limits, and gateway telemetry
Regions See Supported regions Every region where AI Gateway is available. See What is AI Gateway?
Billing A routing fee for each 1 million input tokens, plus the selected model's usage AI Gateway's own charges, plus your backends' usage. No model router fee during the preview.
Status Model router version 2025-11-18 is generally available. Auto, per-request controls, and serving routed requests from your own deployments are in preview. AI Gateway is generally available. Automatic model selection is in preview.

To start calling model router directly, see Quickstart: Get started with model router.

Regions

Automatic model selection is available in every region where AI Gateway is available. For AI Gateway's regions, see What is AI Gateway?. For the regions where you can call model router directly, see Supported regions.

Set up automatic model selection

In the AI Gateway portal, create or edit a model alias, select Choose a model based on the prompt, and then choose the selection mode and the precedence. Check the model name for each backend model, and set the priorities and weights. To configure the alias in code, set the alias properties in What you configure on an alias with the AI Gateway REST API. For the full steps, see automatic model selection in AI Gateway, and have these details ready before you start:

Setting Value
Region A region where AI Gateway is available. See Regions.
Model alias An alias with the backend models that you want model router to choose among, each mapped to its model router model name
Selection mode Balanced, Quality, or Cost
Precedence Model first or Priorities first
API version 2026-11-01

After setup, your apps keep sending requests to the alias, and model router ranks the alias's models for each one. To change the selection mode later, edit the alias in the portal, or update autoModelSelection.objective. The change applies to the next request. For example, with the Azure CLI:

az rest --method patch \
  --url "https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.ApiManagement/aigateways/<gateway-name>/workspaces/<workspace-name>/aliases/<alias-name>?api-version=2026-11-01" \
  --body '{"properties": {"autoModelSelection": {"objective": "Cost"}}}'

For an overview of the gateway itself, see What is AI Gateway?

Pricing

With automatic model selection in AI Gateway, you pay for:

  • AI Gateway: the gateway itself has its own charges, separate from model usage. For prices, see the AI Gateway section of Azure API Management pricing.
  • Model router: there's no routing fee while automatic model selection is in preview. When the feature becomes generally available, model router charges apply.
  • Your backends: each request is billed by the backend that serves it, at that provider's price.

When you call model router directly, with Auto or your own deployment, you pay a routing fee for each 1 million input tokens, plus the selected model's usage at that model's rates. For current prices, see Model router pricing.

Limitations

As of November 2026, automatic model selection in AI Gateway has the following preview limitations.

Limitation How to work with it
Parameters aren't adapted. A parameter that the chosen model doesn't support, such as a non-default temperature on some reasoning models, returns an error. Send only parameters that every model behind the alias supports.
A follow-up turn in the Responses API that uses previous_response_id goes to the backend that served the earlier turn. Model router doesn't rank it again. To let model router choose a model for every turn, send the conversation history in each request, or call model router directly.
Claude models can't share an alias with OpenAI models, because the models behind an alias must share an API format. Use one alias for Claude models and another for OpenAI models.