Express deployments for model serving endpoints

This page describes how to use express deployments on your model serving endpoints. Express deployments lower deployment times and keep the model serving environment the same as the model training environment.

Note

Express deployments were previously called serverless optimized deployments.

What are express deployments?

Express deployments package and stage model artifacts in serverless notebook environments during model registration. This speeds up endpoint deployment and keeps the training and serving environments consistent.

In non-express deployments, model artifacts and environments are packaged into containers at deployment time, so the serving environment may not match the one used during model training.

Standard vs express deployments

The following table compares a standard deployment and an express deployment.

Aspect	Standard deployment	Express deployment
When the environment is built	A container image is built at deployment time.	Artifacts and the environment are packaged when you register the model.
Training and serving environment	The serving environment might not match the training environment.	The serving environment is identical to the notebook environment you registered from.
Deployment speed	Slower. The deployment waits for a container image build.	Faster. The deployment skips the container image build.
Registration speed	Standard.	Adds seconds to a minute for packaging, depending on model and environment size.
Deployment event log	Shows container image creation events.	Does not show container image creation events.

Express deployments move the one-time packaging work to model registration, which adds seconds to a minute to a register_model call, depending on model and environment size. In exchange, deployment is significantly faster: it skips the container image build entirely. That build is also a common source of deployment failures (dependency resolution, image build errors), so skipping it removes a whole class of problems. The deployment event log for an express model contains no container build events.

Requirements

Express deployment endpoints have the same requirements as a model serving endpoint. See Requirements.

In addition:

The model must be a custom model
The model must be logged and registered in a Serverless Notebook using version 3 or later
The model must be logged and registered with mlflow>=3.12 and databricks-sdk>=0.102.0
The model must be registered in Unity Catalog. Serving compute must match the compute the model was registered from. You can register from a regular serverless notebook to serve on CPU, or from serverless GPU compute to serve on GPU.
The model's max environment size is 200GB

Note

To serve a custom LLM on GPU compute using express deployments, see Serve custom LLMs with Custom Model Serving.

Deploy a reranker on GPU

This walkthrough deploys BAAI/bge-reranker-base, a cross-encoder reranker that scores how well a document answers a query. It sets up the environment, logs and registers the model with express packaging, deploys the endpoint, and queries it.

GPU deployments often fail because of dependency version conflicts, for example, torch and CUDA. Express deployments solve this in two ways:

The fixed, published set of libraries preinstalled in each serverless GPU environment version is present during serving exactly as it is in the notebook — same environment version, same versions. These libraries are not repackaged.
Any extra dependencies you install in the notebook session (for example, with %pip install) are packaged during registration and restored during serving.

Together, this means a model that runs in your notebook keeps working when served.

Step 1: Set up a serverless GPU notebook

Create a notebook on serverless GPU compute with an A10 GPU, and select environment version 5, AI environment. The AI environment includes PyTorch and common machine learning libraries (torch, transformers, and others). For the exact pinned versions, see Serverless GPU environment version 5 (Preview).

Install the packages that express deployment requires:

# Express deployment requires recent MLflow and Databricks SDK versions.
%pip install "mlflow>=3.12" "databricks-sdk>=0.102.0"
# Install the libraries your model needs. transformers is preinstalled in the
# v5 AI environment; install it explicitly because this model depends on it.
%pip install transformers
%restart_python

You can also declare dependencies through a serverless environment, but installing in the notebook is the simplest path. Develop against the environment's pinned versions so that the notebook environment matches the serving environment.

Because this model is served on GPU, you must log and register it from a serverless GPU runtime. If you log from serverless CPU compute by accident, the model is packaged with CPU dependencies and the GPU serving endpoint fails to start. Add the following check to fail fast if the notebook is not on a GPU runtime:

import os

# This model is intended to be served on GPU, so we must log and register from a Serverless GPU runtime.
if not os.environ.get("DATABRICKS_ACCELERATOR"):
    raise RuntimeError(
        "This model MUST be logged+registered from a serverless GPU runtime, otherwise the correct dependencies will not be packaged for serving."
    )

Note

This check is only needed because the reranker is served on GPU. A CPU model does not need it.

Step 2: Log the model with MLflow

Load the reranker as a text-classification pipeline and log it with the native mlflow.transformers flavor. The native flavor automatically captures the model's pip dependencies and runs on GPU. You do not need to set pip_requirements, a task, or an entrypoint.

import mlflow
from transformers import pipeline

# BAAI/bge-reranker-base is a cross-encoder reranker: it scores how well a document answers a query.
pipe = pipeline("text-classification", model="BAAI/bge-reranker-base")

model_info = mlflow.transformers.log_model(
    transformers_model=pipe,
    name="bge_reranker",
    input_example={
        "text": "What is Databricks?",
        "text_pair": "Databricks is a data and AI company.",
    },
)

Important

Do not register the model with the registered_model_name argument of log_model. That argument does not accept env_pack, so it registers a non-express model. To enable express deployment, register in a separate step with register_model (Step 3), which accepts env_pack.

Step 3: Register the model to Unity Catalog with express packaging

Register the model to Unity Catalog and set the env_pack parameter to enable express deployment. This packages the model artifacts and the dependencies you added to the notebook session during registration, so serving reuses them on top of the environment version's preinstalled libraries.

import mlflow
from mlflow.utils.env_pack import EnvPackConfig

mlflow.set_registry_uri("databricks-uc")

model_version = mlflow.register_model(
    model_uri=model_info.model_uri,
    name="main.default.bge_reranker",
    env_pack=EnvPackConfig(name="databricks_model_serving"),
)

You can use the string shorthand env_pack="databricks_model_serving" in place of EnvPackConfig(name="databricks_model_serving"). For workspaces without internet access or with custom libraries, set install_dependencies=False (see The env_pack parameter).

Registration requires databricks-sdk>=0.102.0. Earlier versions can time out when uploading large model artifacts.

Step 4: Create a serving endpoint

Deploy the registered model with the Azure Databricks SDK. This deploy step is the same as for any custom model — only the registration step (Step 3) differs for express.

from databricks.sdk import WorkspaceClient
from databricks.sdk.service.serving import (
    EndpointCoreConfigInput,
    ServedEntityInput,
    ServingModelWorkloadType,
)

ENDPOINT_NAME = "bge-reranker-endpoint"

w = WorkspaceClient()
w.serving_endpoints.create_and_wait(
    name=ENDPOINT_NAME,
    config=EndpointCoreConfigInput(
        name=ENDPOINT_NAME,
        served_entities=[
            ServedEntityInput(
                name="bge-reranker",
                entity_name=model_version.name,
                entity_version=model_version.version,
                workload_type=ServingModelWorkloadType.GPU_SMALL,
                workload_size="Small",
                scale_to_zero_enabled=False,
            )
        ],
    ),
)

create_and_wait blocks until the endpoint is ready. GPU_SMALL is sufficient for this reranker. Because serving reuses the same environment version and restores the dependencies you added in the notebook, the served model runs against the same library versions you developed against, regardless of GPU type.

While the endpoint deploys, open the endpoint's Events tab in the Serving UI. Because this is an express deployment, the event log does not show container image creation events (a standard deployment shows Container image creation initiated followed by Container image creation finished successfully). When the endpoint finishes, its state shows Ready.

Step 5: Query the endpoint

A cross-encoder scores a query against a document, so send text and text_pair fields. Query programmatically with the Databricks SDK or curl.

Databricks SDK

w.serving_endpoints.query(
    name=ENDPOINT_NAME,
    dataframe_records=[
        {"text": "What is Databricks?", "text_pair": "Databricks is a data and AI company."},
    ],
)

curl

curl -X POST \
  -u "token:$DATABRICKS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"dataframe_records":[{"text":"What is Databricks?","text_pair":"Databricks is a data and AI company."}]}' \
  https://<workspace-url>/serving-endpoints/bge-reranker-endpoint/invocations

The endpoint returns a relevance score for each query-document pair; higher scores indicate a better match. Use these scores to rerank candidate documents.

Example notebook

Import the following notebook to run this walkthrough end to end.

Express reranker starter notebook

Get notebook

The `env_pack` parameter

The quickstart above shows an express deployment for a GPU model. Express deployments also work for CPU models. In every case, you enable express deployment by passing env_pack to register_model:

import mlflow
from mlflow.utils.env_pack import EnvPackConfig

mlflow.register_model(
    model_info.model_uri,
    model_name,
    env_pack=EnvPackConfig(name="databricks_model_serving"),
)

env_pack packs and stages the model artifacts and the dependencies you added to the notebook session at registration time, which is why registration takes longer than a call without env_pack.

EnvPackConfig accepts an install_dependencies parameter (True by default). When True, the model's dependencies are installed in the current environment to confirm the environment is valid.

Note

Registration can fail in workspaces without internet access, or when the model depends on custom libraries, if install_dependencies is True. In these cases, set install_dependencies to False.

You can substitute the string "databricks_model_serving" for EnvPackConfig(...) as a shorthand. It is equivalent to EnvPackConfig(name="databricks_model_serving", install_dependencies=True).

Feedback

Was this page helpful?

Last updated on 2026-07-27

Express deployments for model serving endpoints

What are express deployments?

Standard vs express deployments

Requirements

Deploy a reranker on GPU

Step 1: Set up a serverless GPU notebook

Step 2: Log the model with MLflow

Step 3: Register the model to Unity Catalog with express packaging

Step 4: Create a serving endpoint

Step 5: Query the endpoint

Databricks SDK

curl

Example notebook

Express reranker starter notebook

The env_pack parameter

Feedback

Additional resources

The `env_pack` parameter