December 2023

These features and Azure Databricks platform improvements were released in December 2023.

Note

Releases are staged. Your Azure Databricks workspace might not be updated until a week or more after the initial release date.

Share dynamic views using Delta Sharing (Public Preview)

December 21, 2023

You can now use Delta Sharing to share dynamic views that restrict access to certain table data based on recipient properties. Dynamic view sharing requires the Databricks-to-Databricks sharing flow. See Add dynamic views to a share to filter rows and columns.

Share volumes using Delta Sharing (Public Preview)

December 20, 2023

You can now use Delta Sharing to share volumes between Databricks workspaces on different Unity Catalog metastores (including workspaces on different Databricks accounts and different clouds). Volumes are Unity Catalog objects that represent a logical volume of storage in a cloud object storage location. They are intended primarily to provide governance over non-tabular data assets. Delta Sharing on Databricks provides a native integration with Unity Catalog that allows you to manage, govern, audit, and track usage of shared volumes data.

See Add volumes to a share.

Entity Relationship Diagram for primary keys and foreign keys

December 18, 2023

In Catalog Explorer, you can now view primary key and foreign key relationships as a graph in the Entity Relationship Diagram (ERD). For more information, see View the Entity Relationship Diagram.

Unity Catalog volume file upload size limit increase

December 13, 2023

The Unity Catalog volume file upload size limit has increased to 5 GB. See Upload files to a Unity Catalog volume.

New notebook cell results rendering available (Public Preview)

December 8, 2023

You can now select a new cell result table rendering that allows searching results, copying subsets of the results to the clipboard, and other features to make it easier to work with tabular outputs. For details, see New cell result table.

Notebook editor themes available

December 8, 2023

You can now select from a variety of themes in the notebook editor. To do so, in the notebook, select View > Editor theme.

External models support in Model Serving is in Public Preview

December 11, 2023

With external models support in Mosaic AI Model Serving, you can now access, manage, and govern third party hosted models, referred to as external models. This support allows you to add endpoints for accessing models hosted outside of Databricks, such as Azure OpenAI GPT models, Anthropic Claude Models, or Amazon Bedrock Models. Once configured, you can grant teams and applications access to these models, enabling them to query via a standard interface without exposing credentials.

See External models in Mosaic AI Model Serving.

Databricks Online Tables is Public Preview

December 8, 2023

Databricks Online Tables are fully serverless tables that auto-scale with the request load and provide low latency and high throughput access to data of any scale. Online Tables are designed to work with Mosaic AI Model Serving, Feature & Function Serving, and retrieval-augmented generation (RAG) applications where they are used for fast data lookups. For details, see Use online tables for real-time feature serving.

Repos & Git Integration Settings UI now correctly notes support for GitHub Enterprise Server

December 7, 2023

The Databricks Repos and Git Integration settings provider (Settings > Linked accounts > Git integration) was updated to specify a dropdown option for Git provider “GitHub Enterprise Server”, which was previously “GitHub Enterprise”.

Databricks JDBC driver 2.6.36

December 6, 2023

We have released version 2.6.36 of the Databricks JDBC driver (download). This release adds native OAuth support for interactive flow mode (user-to-machine or U2M) and client credential flow mode (machine-to-machine or M2M). For more information, see Databricks JDBC Driver.

  • The driver can launch a browser from the desktop to initiate the browser-based authentication flow.
  • The driver can allow the client to set the OAuth client ID and client secrets to enable OAuth for apps and services.

Support for referencing workspace files from init scripts

December 6, 2023

In Databricks Runtime 11.3 LTS and above, you can now reference other workspace files such as libraries, configuration files, or shell scripts from init scripts stored with workspace files. See What are init scripts?.

Feature & Function Serving is Public Preview

December 5, 2023

With Databricks Feature & Function Serving, data in the Databricks platform can be made available to models or applications deployed outside of Databricks. Like Mosaic AI Model Serving endpoints, Feature & Function Serving endpoints automatically scale to adjust to real-time traffic and provide a high-availability, low-latency service at any scale. For details, see What is Databricks Feature Serving?.

You can use Databricks Feature & Function Serving to serve structured data for retrieval augmented generation (RAG) applications. For an example notebook, see online tables with RAG applications.

Foundation Model APIs is Public Preview

December 4, 2023

You can access and query state-of-the-art open source models from a dedicated model serving endpoint. With Foundation Model APIs, you can quickly and easily build applications that leverage a high-quality generative AI model without maintaining your own model deployment. The Foundation Model APIs are provided on a pay-per-token basis and the supported models are accessible in your Databricks Workspace. See Databricks Foundation Model APIs.

New unified admin settings UI

December 4, 2023

The workspace admin settings UI has been updated to incorporate previously separate SQL settings. The new UI also allows for easy navigation between workspace setting and user settings.

Init scripts on DBFS are end-of-life

December 1, 2023

Init scripts on DBFS are now end-of-life. Support has been extended in some workspaces for legacy workloads, but will be removed in the future. All init scripts stored in DBFS should be migrated. See Recommendations for init scripts.

Legacy global and cluster-named init scripts are end-of-life

December 1, 2023

Legacy global and cluster-named init scripts are now end-of-life. Support has been extended in some workspaces for legacy workloads, but will be removed in the future. All legacy global and cluster-named init scripts should be migrated. See Recommendations for init scripts.