Custom model services

A model service is an endpoint that translates and routes inference requests to one or more models, with traffic splitting and fallbacks. It supports real-time requests, plus batch inference.

Create a custom model service for use cases such as:

• A model-agnostic endpoint for an application. Give a customer-support assistant a service named production.ai.support-assistant. Change the underlying model without changing the name the application calls.
• Custom routing and fallbacks. Use traffic splitting to send 10% of traffic to a new model before a broader rollout, or configure a backup destination for failed requests.
• A budget for one workload. Tag a service with project=support-assistant and scope a monthly budget to that tag, with an alert at $1,000 of spend.
• Access controls for a team. Grant the legal team and its application service principals access to a legal-review service, and use separate services for other teams.
• Rate limits for a workload. Configure a test service for 100 requests per minute and a production service for 1,000 requests per minute using service rate limits.
• Separate monitoring and logs. Send a support assistant's requests and responses to a dedicated inference table so its traffic can be inspected separately from coding-agent traffic.

To query a Azure Databricks-served foundation model without creating a service, use a system-provided model service in system.ai.

A service can route to Azure Databricks-served models, using pay-per-token or provisioned throughput, or to external models through a model provider. You can mix these destinations in one service.

Custom model services are Unity Catalog securables. Callers invoke them by their fully qualified name, catalog.schema.name, across workspaces or from outside Azure Databricks. See Governance and privileges.

Requirements

Note

Unity Gateway is not supported on Azure Government.

• A Azure Databricks workspace in a Unity Gateway supported region.
• Unity Catalog enabled for your workspace. See Enable a workspace for Unity Catalog.
• To create a model service, you must have:
  • USE CATALOG, USE SCHEMA, and CREATE SERVICE on the catalog and schema where you create the model service.
  • EXECUTE on each model that the model service references as a destination.
  • EXECUTE, USE CATALOG, and USE SCHEMA on each model provider that the model service references as a destination.
  • USE CATALOG, USE SCHEMA, and CREATE TABLE on the catalog and schema where the inference table is created, if you enable inference logging.

Create a custom model service

Create a model service in the Unity Gateway UI or Catalog Explorer. To create one programmatically, use the REST API, the Azure Databricks SDKs, the Azure Databricks CLI, Terraform, or Declarative Automation Bundles (DABs).

Model services and model providers share a single namespace within a Unity Catalog schema. You can't use a name for a model service if a model provider in the schema already uses it, and vice versa.

UI

1. Do one of the following:
   • In the workspace sidebar, click AI Gateway, then click Create.
   • In Catalog Explorer, go to the schema where you want to create the model service, then click Create > Service > Model service.
2. Enter a name for the model service, and select the catalog and schema to create it in. If you start from Catalog Explorer, Catalog Explorer prefills the catalog and schema.
3. Select the primary destination to serve. This destination can be a Azure Databricks-served model that you have EXECUTE on and that Unity Gateway can serve, or a model provider that you have EXECUTE, USE CATALOG, and USE SCHEMA on.
4. Click Create.

After you create the model service, Azure Databricks opens its overview page, where you can get started or configure additional features such as inference logging.

REST API

Send a POST to /api/2.1/unity-catalog/model-services, passing parent and model_service_id as query parameters. The routing config must have at least one destination:

databricks api post \
  "/api/2.1/unity-catalog/model-services?parent=schemas/main.default&model_service_id=my_model_service" \
  --json '{
  "comment": "Routes chat traffic to a foundation model",
  "config": {
    "routing": {
      "destinations": [
        {
          "name": "primary",
          "destination_type": "DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL",
          "pay_per_token_config": { "model": "models/system.ai.databricks-gpt-5" },
          "traffic_percentage": 100
        }
      ]
    }
  }
}'

CLI

Pass the parent schema and a leaf name, and supply the config with --json. The routing config must have at least one destination. To install the CLI, see Install or update the Databricks CLI.

databricks ai-gateway create-model-service schemas/main.default my_model_service --json '{
  "comment": "Routes chat traffic to a foundation model",
  "config": {
    "routing": {
      "destinations": [
        {
          "name": "primary",
          "destination_type": "DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL",
          "pay_per_token_config": { "model": "models/system.ai.databricks-gpt-5" },
          "traffic_percentage": 100
        }
      ]
    }
  }
}'

Terraform

Create and manage a model service with the Databricks Terraform provider and the databricks_ai_gateway_model_service resource:

resource "databricks_ai_gateway_model_service" "example" {
  parent           = "schemas/main.default"
  model_service_id = "my_model_service"
  comment          = "Routes chat traffic to a foundation model"

  config = {
    routing = {
      destinations = [{
        name                 = "primary"
        destination_type     = "DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL"
        pay_per_token_config = { model = "models/system.ai.databricks-gpt-5" }
        traffic_percentage   = 100
      }]
    }
  }
}

DABs (Beta)

Define the model service in a bundle and deploy it with databricks bundle deploy. The routing config must have at least one destination:

resources:
  model_services:
    my_model_service:
      parent: schemas/main.default
      model_service_id: my_model_service
      comment: Routes chat traffic to a foundation model
      config:
        routing:
          destinations:
            - name: primary
              destination_type: DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL
              pay_per_token_config:
                model: models/system.ai.databricks-gpt-5
              traffic_percentage: 100

Python SDK

Create and manage a model service with the Databricks SDK for Python:

from databricks.sdk.service import catalog as c

model_service = w.ai_gateway.create_model_service(
    parent="schemas/main.default",
    model_service_id="my_model_service",
    model_service=c.ModelService(
        comment="Routes chat traffic to a foundation model",
        config=c.ModelServiceConfig(
            routing=c.ModelServiceConfigRoutingConfig(
                destinations=[
                    c.ModelServiceConfigDestinationConfig(
                        name="primary",
                        destination_type=(
                            c.ModelServiceConfigDestinationConfigDestinationType
                            .DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL
                        ),
                        pay_per_token_config=c.ModelServiceConfigPayPerTokenConfig(
                            model="models/system.ai.databricks-gpt-5"
                        ),
                        traffic_percentage=100,
                    )
                ]
            )
        ),
    ),
)

Go SDK

Create and manage a model service with the Databricks SDK for Go:

modelService, err := w.AiGateway.CreateModelService(ctx, catalog.CreateModelServiceRequest{
	Parent:         "schemas/main.default",
	ModelServiceId: "my_model_service",
	ModelService: catalog.ModelService{
		Comment: "Routes chat traffic to a foundation model",
		Config: &catalog.ModelServiceConfig{
			Routing: &catalog.ModelServiceConfigRoutingConfig{
				Destinations: []catalog.ModelServiceConfigDestinationConfig{{
					Name:            "primary",
					DestinationType: catalog.ModelServiceConfigDestinationConfigDestinationTypeDestinationTypePayPerTokenFoundationModel,
					PayPerTokenConfig: &catalog.ModelServiceConfigPayPerTokenConfig{
						Model: "models/system.ai.databricks-gpt-5",
					},
					TrafficPercentage: 100,
				}},
			},
		},
	},
})

Java SDK

Create and manage a model service with the Databricks SDK for Java:

ModelServiceConfig config =
    new ModelServiceConfig()
        .setRouting(
            new ModelServiceConfigRoutingConfig()
                .setDestinations(
                    Collections.singletonList(
                        new ModelServiceConfigDestinationConfig()
                            .setName("primary")
                            .setDestinationType(
                                ModelServiceConfigDestinationConfigDestinationType
                                    .DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL)
                            .setPayPerTokenConfig(
                                new ModelServiceConfigPayPerTokenConfig()
                                    .setModel("models/system.ai.databricks-gpt-5"))
                            .setTrafficPercentage(100L))));

ModelService modelService =
    w.aiGateway()
        .createModelService(
            new CreateModelServiceRequest()
                .setParent("schemas/main.default")
                .setModelServiceId("my_model_service")
                .setModelService(
                    new ModelService()
                        .setComment("Routes chat traffic to a foundation model")
                        .setConfig(config)));

JS SDK

Create and manage a model service with the Databricks AI Gateway SDK for JavaScript:

import { ModelServiceConfig_DestinationConfig_DestinationType as DestType } from '@databricks/sdk-aigateway/v1';

const created = await client.createModelService({
  parent: 'schemas/main.default',
  modelServiceId: 'my_model_service',
  modelService: {
    comment: 'Routes chat traffic to a foundation model',
    config: {
      routing: {
        destinations: [
          {
            name: 'primary',
            destinationType: DestType.DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL,
            typeConfig: {
              $case: 'payPerTokenConfig',
              payPerTokenConfig: { model: 'models/system.ai.databricks-gpt-5' },
            },
            trafficPercentage: 100,
          },
        ],
      },
    },
  },
});

Grant access to a model service

By default, only the model service owner can query it. To let others query a model service, grant them EXECUTE on it, plus USE CATALOG and USE SCHEMA on its catalog and schema. If the model service logs to an inference table, grant SELECT on the table to let them read the logged requests and responses.

UI

1. Open the model service in Catalog Explorer, or go to AI Gateway and select the service.
2. Go to the Permissions tab.
3. Click Grant.
4. Select the users, groups, or service principals to give access to.
5. Select the EXECUTE privilege.
6. Click Grant.

REST API

databricks api patch \
  "/api/2.1/unity-catalog/permissions/model_service/main.default.my_model_service" \
  --json '{
    "changes": [
      { "principal": "data-team", "add": ["EXECUTE"] }
    ]
  }'

CLI

Grant EXECUTE with the Databricks CLI. To install the CLI, see Install or update the Databricks CLI.

databricks grants update model_service main.default.my_model_service \
  --json '{"changes": [{"principal": "data-team", "add": ["EXECUTE"]}]}'

Terraform

Grant EXECUTE with the Databricks Terraform provider and the databricks_grant resource:

resource "databricks_grant" "example" {
  model_service = "main.default.my_model_service"
  principal     = "data-team"
  privileges    = ["EXECUTE"]
}

DABs (Beta)

Add a grants block to the model service resource in your bundle and redeploy to grant access.

resources:
  model_services:
    my_model_service:
      parent: schemas/main.default
      model_service_id: my_model_service
      comment: Routes chat traffic to a foundation model
      config:
        routing:
          destinations:
            - name: primary
              destination_type: DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL
              pay_per_token_config:
                model: models/system.ai.databricks-gpt-5
              traffic_percentage: 100
      grants:
        - principal: data-team
          privileges: [EXECUTE]

Python SDK

Grant EXECUTE with the Databricks SDK for Python:

from databricks.sdk.service import catalog as c

w.grants.update(
    securable_type="model_service",
    full_name="main.default.my_model_service",
    changes=[c.PermissionsChange(principal="data-team", add=[c.Privilege.EXECUTE])],
)

Go SDK

Grant EXECUTE with the Databricks SDK for Go:

_, err := w.Grants.Update(ctx, catalog.UpdatePermissions{
	SecurableType: "model_service",
	FullName:      "main.default.my_model_service",
	Changes: []catalog.PermissionsChange{{
		Principal: "data-team",
		Add:       []catalog.Privilege{catalog.PrivilegeExecute},
	}},
})

Java SDK

Grant EXECUTE with the Databricks SDK for Java:

w.grants().update(
    new UpdatePermissions()
        .setSecurableType("model_service")
        .setFullName("main.default.my_model_service")
        .setChanges(Arrays.asList(
            new PermissionsChange().setPrincipal("data-team").setAdd(Arrays.asList(Privilege.EXECUTE)))));

See Discover and govern access to model services for more about granting and discovering access.

Configure features on a model service

Configure rate limits, inference logging, and guardrails implemented with service policies on the model service from the Unity Gateway UI. See:

• Apply rate limits to model and MCP services
• Log requests and responses to inference tables

Inference logging

When you enable inference logging, Azure Databricks creates a new, empty Unity Catalog table with a predefined schema at the location you specify. Note the following:

• You must have USE CATALOG, USE SCHEMA, and CREATE TABLE on the target catalog and schema.
• The creator of the model service is the owner of the inference table. No other users have access unless you grant it.
• If a table already exists at the specified location, creating the model service fails.
• The inference table has an independent lifecycle from the model service. If you drop the table, the model service keeps working but stops logging.

For more about inference tables, see Log requests and responses to inference tables.

Update a model service

You must be an owner or have MANAGE.

UI

Edit the model service's configuration from the Unity Gateway UI or Catalog Explorer. Changes apply in place.

REST API

databricks api patch \
  "/api/2.1/unity-catalog/model-services/main.default.my_model_service?update_mask=comment" \
  --json '{"comment": "Updated: routes chat traffic"}'

CLI

databricks ai-gateway update-model-service model-services/main.default.my_model_service comment \
  --json '{"comment": "Updated: routes chat traffic"}'

Terraform

Edit comment (or any other mutable field) on the databricks_ai_gateway_model_service resource and re-apply. Changes apply in place.

DABs (Beta)

Edit comment (or any other mutable field) in the bundle resource and run databricks bundle deploy. Changes apply in place.

Python SDK

from databricks.sdk.service import catalog as c
from google.protobuf.field_mask_pb2 import FieldMask

updated = w.ai_gateway.update_model_service(
    name="model-services/main.default.my_model_service",
    update_mask=FieldMask(paths=["comment"]),
    model_service=c.ModelService(comment="Updated: routes chat traffic"),
)

Go SDK

updated, err := w.AiGateway.UpdateModelService(ctx, catalog.UpdateModelServiceRequest{
	Name:         "model-services/main.default.my_model_service",
	UpdateMask:   *fieldmask.New([]string{"comment"}),
	ModelService: catalog.ModelService{Comment: "Updated: routes chat traffic"},
})

Java SDK

ModelService updated =
    w.aiGateway()
        .updateModelService(
            new UpdateModelServiceRequest()
                .setName("model-services/main.default.my_model_service")
                .setUpdateMask(FieldMask.newBuilder().addPaths("comment").build())
                .setModelService(
                    new ModelService().setComment("Updated: routes chat traffic")));

JS SDK

import { modelServiceFieldMask } from '@databricks/sdk-aigateway/v1';

const updated = await client.updateModelService({
  modelService: {
    name: 'model-services/main.default.my_model_service',
    comment: 'Updated: routes chat traffic',
  },
  updateMask: modelServiceFieldMask('comment'),
});

Delete a model service

You must be an owner or have MANAGE. System-provided model services in system.ai cannot be deleted.

UI

Open the model service in the Unity Gateway UI or Catalog Explorer and select Delete from the kebab menu.

REST API

databricks api delete "/api/2.1/unity-catalog/model-services/main.default.my_model_service"

CLI

databricks ai-gateway delete-model-service model-services/main.default.my_model_service

Terraform

Run terraform destroy, or remove the resource block and re-apply.

DABs (Beta)

Remove the resource from the bundle and run databricks bundle deploy to delete it. databricks bundle destroy also works, but it removes every resource the bundle manages, not just this one.

Python SDK

w.ai_gateway.delete_model_service(name="model-services/main.default.my_model_service")

Go SDK

err := w.AiGateway.DeleteModelService(ctx, catalog.DeleteModelServiceRequest{
	Name: "model-services/main.default.my_model_service",
})

Java SDK

w.aiGateway()
    .deleteModelService(
        new DeleteModelServiceRequest().setName("model-services/main.default.my_model_service"));

JS SDK

await client.deleteModelService({ name: 'model-services/main.default.my_model_service' });

Governance and privileges

As a Unity Catalog securable object, a model service:

• Lives in a catalog and schema, where it inherits schema settings such as workspace bindings.
• Carries standard Unity Catalog metadata, such as name, owner, comment, and tags.
• Is governed by Unity Catalog privileges, so you grant access using the same GRANT and REVOKE statements you use for tables, functions, and models.
• Is discoverable in Catalog Explorer, alongside the rest of your Unity Catalog assets.

The following privileges apply:

Privilege	Description
USE CATALOG, USE SCHEMA	Access the catalog and schema that contain the model service. Required for all operations.
CREATE SERVICE	Create model services in a schema. Granted on the catalog or schema.
EXECUTE	Query a model service.
MANAGE	Modify or delete a model service and manage its grants. The owner has a superset of MANAGE.

Model services use definer's privileges. Azure Databricks evaluates a query against the owner's privileges rather than the caller's. When a user queries a model service, Azure Databricks checks that the owner has EXECUTE on the referenced destinations, such as the underlying models and any model providers. The caller does not need direct access to those destinations.

Limitations

The following capabilities are not supported:

• Creating and managing model services with SQL.
• Discovering model services with only the BROWSE privilege.
• Global search for model services.

Additional resources

• Query model services
• Discover and govern access to model services
• Integrate with coding agents
• Track model service and provider lineage