Priority pay-per-token for Foundation Model APIs

This page describes priority pay-per-token for Foundation Model APIs pay-per-token, including how it behaves and how to send priority requests.

What is priority pay-per-token?

Priority pay-per-token, also referred to as priority mode, is a pay-per-token capability for latency-sensitive, real-time applications. When you send a request with the service_tier parameter set to "priority", Azure Databricks admits the request ahead of standard best-effort pay-per-token traffic on the same model. This keeps availability more consistent during periods of high traffic.

Priority pay-per-token is opt-in per request, so you can send priority and standard requests to the same model. It requires no capacity commitment and is billed at a higher per-token rate than standard pay-per-token requests.

Databricks recommends priority mode when:

  • You need more consistent performance and availability than standard pay-per-token, but aren't ready to commit to dedicated capacity.
  • Your production applications require higher availability.

Supported models

The following pay-per-token models support priority mode. To send a priority request, use the model's endpoint name with the service_tier parameter set to "priority".

Important

Priority pay-per-token is available for OpenAI and Google Gemini models through ADI Services, provided by Databricks. To access these partner models in your Azure Databricks environment, see ADI Services.

Open source models

Provider Model Endpoint name Notes
Alibaba Cloud Qwen3.5 122B A10B databricks-qwen35-122b-a10b
  • This model is available in preview. Reach out to your account teams for enablement.
  • Available only in the westeurope region.

How priority pay-per-token behaves

Consider the following behavior before you use priority pay-per-token:

  • Consistent performance and availability. Priority pay-per-token is designed to keep availability consistent under load. It does not guarantee a specific time-to-first-token or end-to-end latency target. Priority requests target a higher availability than standard pay-per-token requests. Azure Databricks defines availability at a tier as the number of successful requests divided by the total number of admitted requests at that tier.
  • Best-effort capacity. Priority mode does not reserve capacity and there is no capacity commitment.
  • Fallback to standard pay-per-token. If priority capacity is fully subscribed, requests are served at standard pay-per-token availability and billed at standard pay-per-token rates.
  • Per-token premium. Priority requests are billed at a higher per-token rate than standard pay-per-token requests.

Priority pay-per-token compared to provisioned throughput

Priority pay-per-token and provisioned throughput both target production workloads, but they make different trade-offs:

Consideration Priority pay-per-token Provisioned throughput
Capacity Best-effort, shared. Dedicated, reserved capacity.
Commitment None. Opt in per request. Requires a provisioned endpoint.
Availability More consistent than standard pay-per-token under load. Predictable, based on reserved capacity.
Billing Per-token, at a premium over standard pay-per-token. Based on provisioned model units.

Send a priority request

To use priority mode, set the service_tier parameter to "priority" on a per-request basis. The following example uses the OpenAI client:

from databricks_openai import DatabricksOpenAI

client = DatabricksOpenAI()

response = client.chat.completions.create(
    model="databricks-model-name",
    messages=[
      {
        "role": "user",
        "content": "What is a mixture of experts model?",
      }
    ],
    max_tokens=256,
    service_tier="priority",
)

See Foundation model REST API reference for parameter syntax and Use foundation models for more query options.

Capacity limits

Each cluster supports a maximum total number of tokens per minute across all tenants. Azure Databricks sets a per-tenant limit during onboarding so that a single tenant cannot consume all cluster capacity. If your workload requires more capacity than the per-tenant limit allows, reach out to your Databricks account team.

For more Foundation Model APIs limits, see Foundation Model APIs limits and quotas.

Additional resources

  • Last updated on