December 2020

These features and Azure Databricks platform improvements were released in December 2020.

Note

Releases are staged. Your Azure Databricks account may not be updated until a week or more after the initial release date.

Databricks Runtime 7.5 GA

December 16, 2020

Databricks Runtime 7.5, Databricks Runtime 7.5 ML, and Databricks Runtime 7.5 for Genomics are now generally available.

For information, see the full release notes at Databricks Runtime 7.5 (unsupported) and Databricks Runtime 7.5 for ML (unsupported).

Jobs API now supports updating existing jobs

December 14, 2020

You can now use the Jobs API Update endpoint to update a subset of fields for a job. The Update endpoint supplements the existing Reset endpoint by allowing you to specify only the fields which should be added, changed, or deleted rather than overwriting all job settings. Using the Update endpoint is faster, easier, and safer when only some job settings need to be changed, particularly for bulk updates.

New global init script framework is GA

December 14, 2020

The new global init script framework, which was released as a Public Preview in July, is now generally available. The new framework brings significant improvements over legacy global init scripts:

• Init scripts are more secure, requiring admin permissions to create, view, and delete.
• Script-related launch failures are logged.
• You can set the execution order of multiple init scripts.
• Init scripts can reference cluster-related environment variables.
• Init scripts can be created and managed using the admin settings page or the new Global Init Scripts REST API.

Databricks recommends that you migrate existing legacy global init scripts to the new framework to take advantage of these improvements.

For details, see Use global init scripts.

Databricks Runtime 7.5 (Beta)

December 3, 2020

Databricks Runtime 7.5, Databricks Runtime 7.5 ML, and Databricks Runtime 7.5 for Genomics are now available as Beta releases.

For information, see the full release notes at Databricks Runtime 7.5 (unsupported) and Databricks Runtime 7.5 for ML (unsupported).

Jobs API end_time field now uses epoch time

December 2-8, 2020: Version 3.34

The Jobs API returns new data for the Runs Get and Runs List endpoints. The new end_time field returns the epoch time (milliseconds since 1/1/1970 UTC) when the run ended, for example 1605266150681.

Find DBFS files using new visual browser

December 2-8, 2020: Version 3.34

You can now browse and search for DBFS objects in the Azure Databricks workspace UI using the new DBFS file browser.

You can also upload files to DBFS using the new upload dialog in the browser.

This feature is disabled by default. An admin user must enable it using the Admin Console. See Manage the DBFS file browser.

Visibility controls for jobs, clusters, notebooks, and other workspace objects are now enabled by default on new workspaces

December 1-8, 2020: Version 3.34

Visibility controls, which were introduced in September, ensure that users can view only the notebooks, folders, clusters, and jobs that they have been given access to through workspace, cluster, or jobs access control. With this release, visibility controls are enabled by default for new workspaces. If your workspace was created before this release, an admin must enable them. See Access controls lists can no longer be disabled

Improved display of nested runs in MLflow

December 2-8, 2020: Version 3.34

We improved the display of nested runs in the MLflow runs table. Child runs now appear grouped beneath the root run.

Admins can now lock user accounts (Public Preview)

December 2-8, 2020: Version 3.34

Administrators can now set a user’s status to inactive. An inactive user can’t access the workspace, but their permissions and associations with objects such as running clusters or access to notebooks remain unchanged. You can manually deactivate users via the SCIM API or automatically deactivate users after a set period of inactivity. See Users API.

Updated NVIDIA driver

December 2-8, 2020: Version 3.34

GPU-enabled clusters now use NVIDIA driver version 450.80.02.