Create and manage model APIs (model services)

Create, share, and manage your own model APIs (model services) in Unity Catalog.

Tip

The system-provided model APIs in the system.ai schema are ready to query with no setup. Create your own model API only when you want to govern additional models or expose a custom endpoint.

Requirements

A Azure Databricks workspace in a Unity AI Gateway supported region.
Unity Catalog enabled for your workspace. See Enable a workspace for Unity Catalog.
To create a model service, you must have:
- USE CATALOG, USE SCHEMA, and CREATE SERVICE on the catalog and schema where you create the model service.
- EXECUTE on each model that the model service references as a destination.
- EXECUTE, USE CATALOG, and USE SCHEMA on each model provider service that the model service references as a destination.
- USE CATALOG, USE SCHEMA, and CREATE TABLE on the catalog and schema where the inference table is created, if you enable inference logging.

Create a model service

You can create a model service in the Unity AI Gateway UI or in Catalog Explorer, or programmatically with the REST API, the Azure Databricks SDKs, the Azure Databricks CLI, or Terraform.

Model services and model provider services share a single name namespace within a Unity Catalog schema. You can't use a name for a model service if a model provider service in the schema already uses it, and vice versa.

UI

Do one of the following:
- In the workspace sidebar, click AI Gateway, then click Create.
- In Catalog Explorer, go to the schema where you want to create the model service, then click Create > Service > Model service.
Enter a name for the model service, and select the catalog and schema to create it in. If you start from Catalog Explorer, Catalog Explorer prefills the catalog and schema.
Select the primary destination to serve. This destination can be a Databricks-hosted model that you have EXECUTE on and that Unity AI Gateway can serve, or a model provider service that you have EXECUTE, USE CATALOG, and USE SCHEMA on.
Click Create.

After you create the model service, Azure Databricks opens its overview page, where you can get started or configure additional features such as inference logging.

REST API

Send a POST to /api/2.1/unity-catalog/model-services, passing parent and model_service_id as query parameters. The routing config must have at least one destination:

databricks api post \
  "/api/2.1/unity-catalog/model-services?parent=schemas/main.default&model_service_id=my_model_service" \
  --json '{
  "comment": "Routes chat traffic to a foundation model",
  "config": {
    "routing": {
      "destinations": [
        {
          "name": "primary",
          "destination_type": "DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL",
          "pay_per_token_config": { "model": "models/system.ai.databricks-gpt-5" },
          "traffic_percentage": 100
        }
      ]
    }
  }
}'

CLI

Pass the parent schema and a leaf name, and supply the config with --json. The routing config must have at least one destination. To install the CLI, see Install or update the Databricks CLI.

databricks ai-gateway create-model-service schemas/main.default my_model_service --json '{
  "comment": "Routes chat traffic to a foundation model",
  "config": {
    "routing": {
      "destinations": [
        {
          "name": "primary",
          "destination_type": "DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL",
          "pay_per_token_config": { "model": "models/system.ai.databricks-gpt-5" },
          "traffic_percentage": 100
        }
      ]
    }
  }
}'

Terraform

Create and manage a model service with the Databricks Terraform provider and the databricks_ai_gateway_model_service resource:

resource "databricks_ai_gateway_model_service" "example" {
  parent           = "schemas/main.default"
  model_service_id = "my_model_service"
  comment          = "Routes chat traffic to a foundation model"

  config = {
    routing = {
      destinations = [{
        name                 = "primary"
        destination_type     = "DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL"
        pay_per_token_config = { model = "models/system.ai.databricks-gpt-5" }
        traffic_percentage   = 100
      }]
    }
  }
}

Python SDK

Create and manage a model service with the Databricks SDK for Python:

from databricks.sdk.service import catalog as c

model_service = w.ai_gateway.create_model_service(
    parent="schemas/main.default",
    model_service_id="my_model_service",
    model_service=c.ModelService(
        comment="Routes chat traffic to a foundation model",
        config=c.ModelServiceConfig(
            routing=c.ModelServiceConfigRoutingConfig(
                destinations=[
                    c.ModelServiceConfigDestinationConfig(
                        name="primary",
                        destination_type=(
                            c.ModelServiceConfigDestinationConfigDestinationType
                            .DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL
                        ),
                        pay_per_token_config=c.ModelServiceConfigPayPerTokenConfig(
                            model="models/system.ai.databricks-gpt-5"
                        ),
                        traffic_percentage=100,
                    )
                ]
            )
        ),
    ),
)

Go SDK

Create and manage a model service with the Databricks SDK for Go:

modelService, err := w.AiGateway.CreateModelService(ctx, catalog.CreateModelServiceRequest{
	Parent:         "schemas/main.default",
	ModelServiceId: "my_model_service",
	ModelService: catalog.ModelService{
		Comment: "Routes chat traffic to a foundation model",
		Config: &catalog.ModelServiceConfig{
			Routing: &catalog.ModelServiceConfigRoutingConfig{
				Destinations: []catalog.ModelServiceConfigDestinationConfig{{
					Name:            "primary",
					DestinationType: catalog.ModelServiceConfigDestinationConfigDestinationTypeDestinationTypePayPerTokenFoundationModel,
					PayPerTokenConfig: &catalog.ModelServiceConfigPayPerTokenConfig{
						Model: "models/system.ai.databricks-gpt-5",
					},
					TrafficPercentage: 100,
				}},
			},
		},
	},
})

Java SDK

Create and manage a model service with the Databricks SDK for Java:

ModelServiceConfig config =
    new ModelServiceConfig()
        .setRouting(
            new ModelServiceConfigRoutingConfig()
                .setDestinations(
                    Collections.singletonList(
                        new ModelServiceConfigDestinationConfig()
                            .setName("primary")
                            .setDestinationType(
                                ModelServiceConfigDestinationConfigDestinationType
                                    .DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL)
                            .setPayPerTokenConfig(
                                new ModelServiceConfigPayPerTokenConfig()
                                    .setModel("models/system.ai.databricks-gpt-5"))
                            .setTrafficPercentage(100L))));

ModelService modelService =
    w.aiGateway()
        .createModelService(
            new CreateModelServiceRequest()
                .setParent("schemas/main.default")
                .setModelServiceId("my_model_service")
                .setModelService(
                    new ModelService()
                        .setComment("Routes chat traffic to a foundation model")
                        .setConfig(config)));

JS SDK

Create and manage a model service with the Databricks AI Gateway SDK for JavaScript:

import { ModelServiceConfig_DestinationConfig_DestinationType as DestType } from '@databricks/sdk-aigateway/v1';

const created = await client.createModelService({
  parent: 'schemas/main.default',
  modelServiceId: 'my_model_service',
  modelService: {
    comment: 'Routes chat traffic to a foundation model',
    config: {
      routing: {
        destinations: [
          {
            name: 'primary',
            destinationType: DestType.DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL,
            typeConfig: {
              $case: 'payPerTokenConfig',
              payPerTokenConfig: { model: 'models/system.ai.databricks-gpt-5' },
            },
            trafficPercentage: 100,
          },
        ],
      },
    },
  },
});

Grant access to a model service

By default, only the model service owner can query it. To let others query a model service, grant them EXECUTE on it, plus USE CATALOG and USE SCHEMA on its catalog and schema. If the model service logs to an inference table, grant SELECT on the table to let them read the logged requests and responses.

UI

Open the model service in Catalog Explorer, or go to AI Gateway and select the service.
Go to the Permissions tab.
Click Grant.
Select the users, groups, or service principals to give access to.
Select the EXECUTE privilege.
Click Grant.

REST API

databricks api patch \
  "/api/2.1/unity-catalog/permissions/model_service/main.default.my_model_service" \
  --json '{
    "changes": [
      { "principal": "data-team", "add": ["EXECUTE"] }
    ]
  }'

For more about granting and discovering access, see Discover and govern access to model APIs (model services).

Configure features on a model service

You configure features such as rate limits, inference logging, and guardrails on the model service from the Unity AI Gateway UI, the same way you configure them on an Unity AI Gateway endpoint. See:

Inference logging

When you enable inference logging, Azure Databricks creates a new, empty Unity Catalog table with a predefined schema at the location you specify. Note the following:

You must have USE CATALOG, USE SCHEMA, and CREATE TABLE on the target catalog and schema.
The creator of the model service is the owner of the inference table. No other users have access unless you grant it.
If a table already exists at the specified location, creating the model service fails.
The inference table has an independent lifecycle from the model service. If you drop the table, the model service keeps working but stops logging.

For more about inference tables, see Log requests and responses to inference tables.

Update a model service

You must be an owner or have MANAGE.

UI

Edit the model service's configuration from the Unity AI Gateway UI or Catalog Explorer. Changes apply in place.

REST API

databricks api patch \
  "/api/2.1/unity-catalog/model-services/main.default.my_model_service?update_mask=comment" \
  --json '{"comment": "Updated: routes chat traffic"}'

CLI

databricks ai-gateway update-model-service model-services/main.default.my_model_service comment \
  --json '{"comment": "Updated: routes chat traffic"}'

Terraform

Edit comment (or any other mutable field) on the databricks_ai_gateway_model_service resource and re-apply. Changes apply in place.

Python SDK

from databricks.sdk.service import catalog as c
from google.protobuf.field_mask_pb2 import FieldMask

updated = w.ai_gateway.update_model_service(
    name="model-services/main.default.my_model_service",
    update_mask=FieldMask(paths=["comment"]),
    model_service=c.ModelService(comment="Updated: routes chat traffic"),
)

Go SDK

updated, err := w.AiGateway.UpdateModelService(ctx, catalog.UpdateModelServiceRequest{
	Name:         "model-services/main.default.my_model_service",
	UpdateMask:   *fieldmask.New([]string{"comment"}),
	ModelService: catalog.ModelService{Comment: "Updated: routes chat traffic"},
})

Java SDK

ModelService updated =
    w.aiGateway()
        .updateModelService(
            new UpdateModelServiceRequest()
                .setName("model-services/main.default.my_model_service")
                .setUpdateMask(FieldMask.newBuilder().addPaths("comment").build())
                .setModelService(
                    new ModelService().setComment("Updated: routes chat traffic")));

JS SDK

import { modelServiceFieldMask } from '@databricks/sdk-aigateway/v1';

const updated = await client.updateModelService({
  modelService: {
    name: 'model-services/main.default.my_model_service',
    comment: 'Updated: routes chat traffic',
  },
  updateMask: modelServiceFieldMask('comment'),
});

Delete a model service

You must be an owner or have MANAGE. System-provided model services in system.ai cannot be deleted.

UI

Open the model service in the Unity AI Gateway UI or Catalog Explorer and select Delete from the kebab menu.

REST API

databricks api delete "/api/2.1/unity-catalog/model-services/main.default.my_model_service"

CLI

databricks ai-gateway delete-model-service model-services/main.default.my_model_service

Terraform

Run terraform destroy, or remove the resource block and re-apply.

Python SDK

w.ai_gateway.delete_model_service(name="model-services/main.default.my_model_service")

Go SDK

err := w.AiGateway.DeleteModelService(ctx, catalog.DeleteModelServiceRequest{
	Name: "model-services/main.default.my_model_service",
})

Java SDK

w.aiGateway()
    .deleteModelService(
        new DeleteModelServiceRequest().setName("model-services/main.default.my_model_service"));

JS SDK

await client.deleteModelService({ name: 'model-services/main.default.my_model_service' });

Next steps

Feedback

Was this page helpful?

Last updated on 2026-08-31

Create and manage model APIs (model services)

Requirements

Create a model service

UI

REST API

CLI

Terraform

Python SDK

Go SDK

Java SDK

JS SDK

Grant access to a model service

UI

REST API

Configure features on a model service

Inference logging

Update a model service

UI

REST API

CLI

Terraform

Python SDK

Go SDK

Java SDK

JS SDK

Delete a model service

UI

REST API

CLI

Terraform

Python SDK

Go SDK

Java SDK

JS SDK

Next steps

Feedback

Additional resources