Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
AI Gateway helps platform teams give applications and agents shared, governed access to AI models and tools. You can bring existing AI traffic through AI Gateway to centralize connections, apply AI-focused controls, and gain visibility into usage.
This article explains how you can move AI APIs in Azure API Management service to Azure AI Gateway.
Map the workload to AI Gateway
Inventory the model connections and tools you plan to bring through AI Gateway, their callers, backend services, credentials, controls, networking, and operational dependencies. Use that inventory to design the target configuration.
| Existing configuration | Target design to assess |
|---|---|
| Direct model calls or model APIs behind a gateway | Model providers and registered models, with the required protocol, endpoint, and backend authentication |
| Client-facing model names | Model aliases that preserve the names applications use where possible |
| MCP servers or HTTP API operations exposed as tools | MCP backends and selected tools published through gateway MCP endpoints |
| Application credentials and caller authorization, including API Management subscriptions | Gateway runtime credentials or Microsoft Entra access grants for the required callers |
| Controls in applications or an existing gateway | Supported AI Gateway controls configured for the appropriate models, tools, and callers |
| Consumption limits and caller grouping | Token-limit and cost-limit policies that meet the workload's allowance and estimated-spend requirements |
| Diagnostics, logs, and dashboards | Gateway telemetry, alerts, and dashboards that support equivalent operational checks |
| Deployment automation | Resource definitions and management operations for Microsoft.ApiManagement/aigateways |
Treat this mapping as a design assessment, not a direct configuration conversion. For workloads in API Management, APIs, products, subscriptions, and policies need to be mapped to AI Gateway's model- and tool-centered configuration. A token quota and an estimated monetary budget serve different purposes; neither replaces provider billing.
Use the AI Gateway plugin to assess and migrate
The AI Gateway plugin performs the identification described in the previous section and carries it through to provisioning. It works in four steps: inventory the objects in the source instance, plan the gateway objects that represent them, adapt the plan where capabilities differ, and provision the result. Each step is distinct.
Prerequisites
- An Azure API Management instance that publishes AI workloads: model APIs, Model Context Protocol (MCP) servers, or agent-to-agent (A2A) agents.
- The Reader role on the API Management instance, and the Reader role on any Azure AI Foundry or Azure OpenAI account that you want resolved to its deployments.
- Permission to create resources in the target resource group, and permission to assign roles if your workload uses managed identity to reach its model backends.
- Azure CLI 2.75.0 or later, signed in with
az login, and theaz aigatewayextension. For installation steps, see the AI Gateway CLI guide. - Python 3.10 or later, which the migration tool runs on.
- A coding agent with the AI Gateway plugin installed: GitHub Copilot CLI, Claude Code, Cursor, or Gemini CLI. For installation steps, see Coding-agent plugin.
How the plugin works
The plugin adds a migrate-apim skill and an /aigateway:migrate command to your coding agent, and it ships the migration tool that does the reading and writing. The tool is a dependency-free Python package, so there's nothing else to install, and it reads Azure Resource Manager with your Azure CLI sign-in.
Start your agent in a working folder and run the command with the source instance:
/aigateway:migrate <apim-name> <resource-group>
The agent drives a pipeline of files rather than a single command, because a migration needs your decisions in the middle of it:
| Artifact | Produced by | Edited | Contains |
|---|---|---|---|
inventory.json |
collect |
No | What API Management holds, plus resolved Azure deployments |
plan.json |
plan |
Yes, in place | One item per source aspect: the proposal, the options, the evidence, and pre-shaped target objects |
journal.json |
apply |
No | Attempts and resume state |
Three properties of the workflow are worth knowing before you start:
- The plan is the record. Every decision lives on the item it settles, so a later session continues by pointing at the same file. Re-running
planpreserves your decisions. - Migration is additive. Everything is provisioned into the gateway, and nothing is removed from API Management. A failed run leaves a partially built gateway and a source that still serves traffic exactly as before.
- Secrets stay where they are. No credential value is written to any artifact. Each one becomes a reference that carries the exact call that retrieves it, and the value is resolved just in time when the migration applies it.
Create the inventory
When you invoke the skill, it first creates an inventory. The inventory identifies the objects on the source side, including which AI workloads the instance publishes, and the backends, policies, and credentials that each one depends on.
Sign in and select the subscription that holds the API Management instance.
az login az account set --subscription <subscription-id>Let the agent collect the inventory. The command it runs is:
migrate-apim collect --apim <apim-name> -g <resource-group> -o inventory.jsonYou can ask the agent to narrow the scope, restrict the asset families, or filter the APIs. It updates the command as follows:
migrate-apim collect --apim <apim-name> -g <resource-group> --family mcp --family a2a migrate-apim collect --apim <apim-name> -g <resource-group> --include-api 'aoai-*' --exclude-api 'legacy-*'Review what the agent found. The inventory records each asset family and how confident the detection is.
Family How API Management represents it How it's detected MCP servers APIs of type mcpFrom the API type A2A agents APIs of type a2aFrom the API type Unified model APIs APIs of type unifiedModelApioverlanguageModelbackendsFrom the API type and the backend declaration Proxied model APIs An ordinary httpAPI in front of a model endpointScored from weighted evidence: backend resource ID, provider host, AI policies, naming conventions, and inference routes Model providers Not represented. Derived from the backends and service URLs that assets reference Derived, including backends that no API publishes Every API that the agent excluded is listed with its reason, so you can confirm that the agent didn't drop anything silently. Confirm any API reported with low confidence before you plan against it.
Check
collection.coveragefor failed reads before you trust the result. A failed read is an unknown, not an absence. If an entry failed because of access, collect again with the right permissions.Read the summary with your workload owners.
migrate-apim summary inventory.json --format markdown
Tip
Operation-level policies aren't collected by default. Ask the agent to read them if you have them.
Resolve items that need attention
The plan identifies the objects on the target side. It matches the inventory against a capability map of what the gateway and the installed CLI can actually do.
Let the agent produce the plan. Pass the target gateway as a full resource ID or ask to create a new one.
Work through the items that aren't settled. Each item carries a status:
Status Meaning readyThe gateway represents this natively, and the item already carries its target objects needs-decisionConfirm the proposal or choose a different option adaptedYou settled it excludedAnother object carries it, and it's cross-referenced rather than dropped staleThe source facts that this item rested on changed, and the item says what moved Every
needs-decisionitem arrives with a proposal, a rationale, and the consequence of each option.Record each decision on its item by setting only
statustoadaptedanddecisionto the value you chose, then runplanagain. Save the file as UTF-8 without a byte order mark."decision": "split" "decision": {"value": "list-models", "models": ["gpt-4o", "gpt-5"]}Read the plan's prerequisites.
Important
These changes are outside the gateway's resource provider. Typical entries are Foundry role grants for the gateway's managed identity, the AI Gateway Models User and AI Gateway Tools User roles for callers, Microsoft Entra consent, OAuth authorization for an MCP backend, a delegated subnet for a network-integrated source, private endpoint approvals, and enabling the OpenTelemetry endpoint on an Application Insights resource.
Deploy the resources
After you prepare your plan, start provisioning. The process runs in eight phases:
| Phase | What it provisions |
|---|---|
gateway |
The Microsoft.ApiManagement/aigateways resource |
identity |
The gateway's managed identity, used to reach backends |
providers |
One model provider per upstream model endpoint |
models |
One registration per model that the providers serve |
aliases |
The callable names, with their priority and weight pools |
toolServers |
Federated MCP servers and their backends, allow lists, and block lists |
policies |
Token limits on models, and the rate limit and IP filter baseline |
keys |
Gateway API keys and telemetry exporters, when the plan calls for them |
For each phase, the agent runs a preflight check.
migrate-apim apply --plan plan.json --inventory inventory.json --preflightThe preflight check prints the full phase checklist. Keep that checklist: a migration runs over many turns, and the phase that nobody tracks is the phase that goes missing.
Complete the prerequisites, and acknowledge each one only after you finish the work.
Rehearse the next phase. A run without
--executewrites nothing and prints the exact command that each step would run. Review this output before creating anything.migrate-apim apply --plan plan.json --inventory inventory.json --next-phaseExecute the phase that you just rehearsed.
migrate-apim apply --plan plan.json --inventory inventory.json --next-phase --executeRepeat the rehearse-then-execute cycle for each remaining phase.
Check the status of the run at any point.
migrate-apim apply --plan plan.json --inventory inventory.json --status
Verify
Finish the migration by validating the gateway with real traffic, then cut over:
- Issue new gateway keys or assign the data-plane roles to each caller.
- Update every caller's host, route, and authorization header. If you split an API, some routes now resolve on the gateway and the rest still resolve on API Management.
- Confirm that telemetry is arriving at the destination that you chose.
- Retire the source assets from API Management only after you validate the new gateway. The migration process never removes anything, so you're responsible for that step.
Workload-specific considerations
Each workload type carries its own constraints. Review the ones that apply to your instance before you settle on the plan.
Model APIs
AI Gateway publishes a fixed set of inference routes, rather than publishing whatever operations an API defines:
/openai/v1/chat/completions
/openai/v1/responses
/openai/v1/embeddings
/v1/messages
A typical Azure OpenAI import in API Management publishes far more routes than that. Assistants, threads, vector stores, audio, images, and legacy completions operations have no route on the gateway. So the first question about a model API isn't which object represents it but how much of it can move. The usual answer is to split the API: inference moves to the gateway, and the remaining operations keep serving from API Management until you retire them.
Two more constraints apply to model APIs:
- Every alias needs a name. The name that clients send in the
modelfield is a new decision for each alias, because API Management has no equivalent to take it from. - The gateway doesn't translate between formats. A unified model API can accept one dialect and call another. AI Gateway routes protocol-natively, so an Anthropic model is reached at
/v1/messages. Callers either move to the backend dialect or keep a translating layer in front of the gateway.
MCP servers
How you build an MCP API in API Management determines what moves:
- A proxied server has no stored tool list. API Management forwards
tools/listto the upstream server, so the tool names that an allow list needs exist only at runtime and you must enumerate them first. The plugin does this withdiscover-tools. - A server that projects a REST API as tools becomes a Federated MCP Server backend of kind
openApiplus an allow list. Policies attached to individual tools don't carry over, so re-express them as allow list and block list entries, or keep them in API Management.
A2A agents
AI Gateway has no equivalent for agent-to-agent APIs yet, so they stay on API Management for now. Identify them anyway: the inventory captures the agent's upstream endpoint, its JSON-RPC configuration, and the location of its agent card, so the configuration survives and you can move it when the gateway supports it.
Policies
A token limit lands on each model registration that the source policy covered, so each model keeps its own ceiling. A rate limit or an IP filter has no per-model form: the gateway holds one baseline on the default workspace, so several source policies collapse into one and the strictest wins.