Migrate to Unity Gateway

Unity Gateway is the Azure Databricks control plane for enterprise AI. Built on Unity Catalog, it governs model services, external model providers, MCP servers, agents, skills, and tools from one place, with the same privileges, cost controls, guardrails, and observability you use for data. Legacy AI Gateway endpoints stay bound to a single workspace and don't provide this centralized governance.

Use this guide to move existing model serving and legacy AI Gateway workloads to Unity Gateway, and to turn on the Enforce Unity Gateway workspace setting so all generative AI traffic is governed through Unity Catalog.

Requirements

  • A Azure Databricks workspace enabled for Unity Catalog. See Enable a workspace for Unity Catalog.
  • Workspace admin access to turn on the Enforce Unity Gateway setting.
  • Account admin access to query the system.serving usage tables when you identify legacy traffic.

Choose your migration path

Pick the path that matches your situation.

Your situation Recommended path
You don't use AI Gateway, use it only casually, or you're setting up a new account or workspace Start fresh with Unity Gateway.
You have active legacy AI Gateway workloads Migrate existing workloads.
You enrolled in the Foundation Model Unity Catalog Permissions preview Migrate existing workloads, and review your model service permissions. Permissions set on individual models don't automatically grant access to the corresponding model service.

Start fresh with Unity Gateway

If you're new to Unity Gateway or setting up a new account or workspace, start directly on Unity Gateway.

Step 1: Review access to Azure Databricks-hosted model services

Unity Gateway provides ready-to-use Azure Databricks-hosted model services in the system.ai schema. Review who can access them and restrict access where you need to.

By default, all account users have EXECUTE on system-provided model services. Every model service also requires USE CATALOG on system and USE SCHEMA on system.ai. To apply least privilege, remove broad schema access and grant EXECUTE on individual model services. Underlying model permissions don't grant model service access.

Action Required permissions
Query a model service EXECUTE on the model service, plus USE CATALOG and USE SCHEMA on its catalog and schema.
Create a model service EXECUTE on the underlying model, plus CREATE SERVICE, USE CATALOG, and USE SCHEMA where you create the model service.
Query an external model provider EXECUTE on the external model provider, plus USE CATALOG and USE SCHEMA on its catalog and schema.
Create an external model provider CREATE SERVICE, USE CATALOG, and USE SCHEMA where you create the external model provider.

For governed-tag and attribute-based access, see GRANT policies.

Note

If you were enrolled in the Foundation Model Unity Catalog Permissions preview, permissions set on individual models don't automatically apply to the corresponding model services. Review and reapply your least-privilege grants on the model services.

Step 2: Enable Enforce Unity Gateway

Turn on the Enforce Unity Gateway workspace setting to disable legacy AI Gateway experiences so all generative AI traffic is governed through Unity Catalog. If you don't turn it on, your existing legacy configurations are unchanged.

When you enable enforcement, each product surface behaves as follows:

Surface Behavior when enforcement is on
Pay-per-token Foundation Models All pay-per-token traffic must route through a model service. Azure Databricks-provided pay-per-token serving endpoints (the databricks- models) are disabled.
Provisioned Throughput Foundation Models You can no longer create provisioned throughput serving endpoints without Unity Gateway. Existing endpoints stay intact and remain queryable.
External Models You can no longer create external model serving endpoints without Unity Gateway. Existing endpoints stay intact and remain queryable.

To enable the setting:

  1. Log in to your Azure Databricks workspace as a workspace admin.
  2. Go to Settings > Advanced.
  3. Turn on Enforce Unity Gateway.

After enforcement is on, requests to disabled pay-per-token serving endpoints return a PERMISSION_DENIED error:

{
  "error_code": "PERMISSION_DENIED",
  "message": "Querying pay-per-token foundation model endpoint 'databricks-gpt-5' is disabled for this workspace. Please use Unity Gateway."
}

Warning

Enabling enforcement stops traffic to active legacy endpoints. If your workspace runs legacy workloads, complete Migrate existing workloads and validate that traffic has moved before you turn on the setting.

The Enforce Unity Gateway setting is available in workspaces that have legacy AI Gateway endpoints. It stays available after their traffic drops to zero, so you can turn it on at the end of your migration. New workspaces have enforcement on by default and don't show the setting. If the setting isn't available in your workspace, contact your Azure Databricks account team after you complete and validate your migration.

When enforcement is on, the following limitations apply because not all products that relied on legacy serving endpoints have migrated to Unity Gateway:

  • AI Search: Creating AI Search endpoints through Unity Gateway is not yet supported. Knowledge Assistant and Multi-Agent Supervisor agents that depend on AI Search endpoints don't work.
  • ai_query: ai_query supports system-provided model services in system.ai only, not custom model services.
  • Databricks Apps: Apps configured with the model serving endpoint resource (a service principal with access to system.ai models) stop working. Grant the service principal EXECUTE on the corresponding system.ai model services, and update the app to query model services.

Migrate existing workloads

If you have active legacy AI Gateway workloads, follow these steps before you enable enforcement.

Step 1: Identify active legacy usage

Find which legacy endpoints still receive traffic, in which workspaces, and who calls them.

  1. Enable usage tracking on your legacy endpoints. Enabling it is idempotent, so it's safe to re-run.
  2. Query the system.serving.endpoint_usage and system.serving.served_entities system tables for recent requests, callers, and last-request times. Only account admins can query these tables.

Step 2: Configure model services and providers

Governance configured on a legacy endpoint doesn't carry over. The existing configuration stays on the legacy endpoint. Create or identify the model service or external model provider you need, then re-create the settings you rely on, such as permissions, rate limits, budgets, service policies, usage tracking, inference tables, and traffic routing and fallbacks. Also update any CI/CD or infrastructure-as-code workflows to use Unity Gateway APIs.

Migrate each workload type to the following target:

Existing workload Migration target
Azure Databricks-hosted pay-per-token model Use the corresponding system-provided model service in system.ai, review its permissions, and re-create the settings you need. See Discover and govern access to model services.
Azure Databricks-hosted provisioned throughput Keep the existing provisioned throughput serving endpoint and create a model service that references it. Grant callers EXECUTE on the model service.
External provider Create an external model provider with your existing provider, credentials, exposed models, and callers. Query it directly, or create a model service for model-specific governance, in which case the model service's settings take precedence.

Step 3: Update your clients

After you configure the model service or external model provider, move each workload to Unity Gateway. Migrate the gateway URL and the personal access token (PAT) scope together.

API and SDK clients: For Azure Databricks-hosted models, change the base URL from /serving-endpoints to /ai-gateway/mlflow/v1 and change the model from the endpoint name to the fully qualified model service name.

from openai import OpenAI

client = OpenAI(
    api_key=token,
    base_url="https://<workspace-url>/ai-gateway/mlflow/v1",
)

response = client.chat.completions.create(
    model="<catalog>.<schema>.<model-service>",
    messages=[...],
)

For external models, query the model provider service by passing its name in a request header.

from openai import OpenAI

client = OpenAI(
    api_key=token,
    base_url="https://<workspace-url>/ai-gateway/openai/v1",
    default_headers={
        "Databricks-Model-Provider-Service": "<catalog>.<schema>.<model-provider-service>"
    },
)

response = client.chat.completions.create(
    model="<provider-model-name>",
    messages=[...],
)

For more query options, see Query model services and Query external model providers (model provider services).

Authentication: Unity Gateway supports both OAuth and PAT authentication. The PAT scope you need depends on the URL:

  • Workspace /ai-gateway/ route: Use the recommended, least-privilege ai-gateway scope.
  • Legacy regional *.ai-gateway.* host: Use the broader all-apis scope.

Calling the legacy regional URL with an ai-gateway-scoped PAT returns 403: required scopes: all-apis. Move the client to the workspace /ai-gateway/ URL, use an ai-gateway-scoped PAT, and retry.

ai_query: Replace the legacy endpoint name with the corresponding Azure Databricks-provided model service in system.ai. The caller needs EXECUTE on that model service.

-- Legacy
SELECT ai_query('<legacy-endpoint-name>', 'Summarize: ' || text)
FROM my_table;

-- Unity Gateway
SELECT ai_query('system.ai.<model-name>', 'Summarize: ' || text)
FROM my_table;

ai_query supports system-provided model services, not custom model services. Only usage tracking applies. Service policies, inference tables, rate limits, and fallbacks don't apply to ai_query calls. See ai_query function.

Coding agents: Configure a supported coding agent to route through Unity Gateway instead of connecting directly to the model provider. Azure Databricks provides the Unity Gateway CLI (ug) for this setup. Installing it requires Python 3.12 or later and uv.

uv tool install git+https://github.com/databricks/unity-gateway

Launch a supported coding agent through Unity Gateway:

ug claude
ug codex
ug gemini
ug opencode
ug copilot

If you use your own provider credentials, first create an external model provider, then point the coding agent at it. The --provider option is supported for ug claude and ug codex.

ug claude --provider <catalog>.<schema>.<provider-service>

See Get started with coding agents and Set up model capacity.

Step 4: Validate migrated traffic

Confirm that legacy traffic has stopped before you enable enforcement. Query the system.serving.endpoint_usage table for each legacy endpoint and check that its request count has dropped to zero and its last-request time is before your migration.

Step 5: (Optional) Throttle legacy endpoints during a phased migration

To migrate gradually, set an individual legacy endpoint's rate limit to 0 to stop new traffic to it while leaving other legacy endpoints active. Repeat steps 2 through 5 for each remaining workload.

Step 6: Enable Enforce Unity Gateway

After all required traffic has migrated successfully, turn on the Enforce Unity Gateway setting to disable legacy endpoints for the workspace. See Step 2: Enable Enforce Unity Gateway for what changes, how to enable the setting, and its limitations.

Additional resources

  • Last updated on