February 2023

These features and Azure Databricks platform improvements were released in February 2023.

Note

Releases are staged. Your Azure Databricks account might not be updated until a week or more after the initial release date.

Support for ND A100 v4 series instances (Public Preview)

February 28, 2023

Databricks now supports ND A100 v4-series instances for GPU clusters. ND A100 instances are optimized for large deep learning workloads.

Ray on Azure Databricks (Public Preview)

February 24, 2023

With Databricks Runtime 12.0 and above, you can create a Ray cluster and run Ray applications in Azure Databricks with the Ray on Spark API. See Use Ray on Azure Databricks.

Notebook cell output results limit increased

February 23, 2023

When you execute a cell in a notebook, the lesser of 10,000 rows or 2 MB of output is displayed.

Azure Databricks Jobs now supports running continuous jobs

February 22, 2023

You can ensure there is always an active run of your Azure Databricks job with the new continuous trigger type. See Run a continuous job.

Trigger your Azure Databricks job when new files arrive

February 22, 2023

You can now use a file arrival trigger to run your Azure Databricks job when new files arrive in an external location such as Amazon S3 or Azure storage. See Trigger jobs when new files arrive.

Databricks Terraform provider updated to version 1.10.0

February 21, 2023

Version 1.10.0 makes updates to databricks_share, various documentation updates, and more. For more details, see the changelog for version 1.10.0.

Legacy global init scripts and cluster-named init scripts disabled

February 21, 2023

Starting February 21, 2023, legacy global init scripts and cluster-named init scripts are disabled and cannot be used in new workspaces. See What are init scripts?.

Improvements to the MLflow experiment UI

February 17, 2023

The MLflow experiment UI now includes a configurable chart view providing visual model performance insights, a revamped parallel coordinates experience for tuning, and a streamlined table view with enhancements for search and filtering. See Organize training runs with MLflow experiments.

Databricks Runtime 12.2 (Beta)

February 17, 2023

Databricks Runtime 12.2 and Databricks Runtime 12.2 ML are now available as Beta releases.

See Databricks Runtime 12.2 LTS and Databricks Runtime 12.2 LTS for Machine Learning.

Use user-assigned managed identities to access storage in Unity Catalog (Public Preview)

February 15, 2023

You can now use user-assigned managed identities instead of system-assigned managed identities to access storage containers on behalf of Unity Catalog users. See Use Azure managed identities in Unity Catalog to access storage.

Databricks extension for Visual Studio Code (Public Preview)

February 14, 2023

The Databricks extension for Visual Studio Code lets developers leverage the powerful authoring capabilities of IDEs while connecting to Azure Databricks clusters to run code remotely, employing best practices such as source control, modularized code, refactoring, and unit testing.

Serverless Real-Time Inference Public Preview now available to all customers

February 9, 2023

Serverless Real-Time Inference is now available in public preview to all customers. You no longer need to request access.

Serverless Real-Time Inference provides a highly available and low-latency service for deploying models. The service automatically scales up or down to meet demand changes within the chosen concurrency range. See Model serving with Azure Databricks

Serverless SQL warehouses are available to workspaces in supported regions (Public Preview)

February 9, 2023

Serverless SQL warehouses are now available to all customers with workspaces in the regions West US (westus), Central US (centralus), East US (eastus), East US 2 (eastus2), North Europe (northeurope), and West Europe (westeurope). See Enable serverless SQL warehouses.

Databricks Terraform provider updated to version 1.9.2

February 3, 2023

Version 1.9.2 adds a databricks_node_type selector, a new file library type for databricks_pipeline, various documentation updates, and more. For more details, see the changelog for version 1.9.2.

Customer-managed keys help protect and control access to some types of data (GA)

February 3, 2023

Some Azure Databricks services and data support adding a customer-managed key to help protect and control access to encrypted data. Azure Databricks has three customer-managed key features that involve different types of data and locations, which you can compare in Customer-managed keys for encryption. All three of these features are now generally available.

Variable explorer in Azure Databricks notebooks

February 1, 2023

With Databricks Runtime 12.1 and above, you can now directly observe current Python variables and their values in the notebook UI.