Managed warehouse for MLflow traces

Important

This feature is in Beta. A workspace admin must enable Managed Warehouse for MLflow from the Previews page. See Manage Azure Databricks previews.

To show traces stored in Unity Catalog, the MLflow UI runs SQL queries on your trace tables. With managed warehouse, MLflow manages an optimized warehouse that can execute these queries with low latency and provide a more interactive experience in the UI.```

How managed warehouse works

When you view traces in the MLflow UI, MLflow picks the first available option from this list:

  1. Your SQL warehouse: If you chose a SQL warehouse for the experiment, MLflow uses it.
  2. MLflow Managed Warehouse: If your workspace has Lakehouse//RT enabled, MLflow creates a warehouse named MLflow Managed Warehouse in your workspace and runs trace queries on it. Workspace admins can see this warehouse in the list of SQL warehouses. Admins can edit or delete it, but Databricks recommends against it.
  3. Serverless compute: Otherwise, MLflow runs trace queries on serverless compute that Azure Databricks manages.

MLflow creates the MLflow Managed Warehouse in the background. Until it's ready, MLflow uses serverless compute.

Users don't need explicit permissions on the MLflow Managed Warehouse, but they cannot access it for general use. MLflow restricts access to this warehouse only for its own UI interactions.

To use your own SQL warehouse instead, open the experiment's Settings page and change Trace compute to SQL warehouse. This setting applies only to you.

Billing

You're billed for managed warehouse usage the same way as for other compute. The MLflow Managed Warehouse is billed like any other SQL warehouse, and serverless compute is billed like any other serverless compute usage.

Requirements

Limitations

Managed warehouse works only in the MLflow UI. The MLflow Python SDK still requires a SQL warehouse.