April 2020

These features and Azure Databricks platform improvements were released in April 2020.

Note

Releases are staged. Your Azure Databricks account may not be updated until up to a week after the initial release date.

Unique URLs for each Azure Databricks workspace

April 28, 2020

Azure Databricks has added a new unique URL for each workspace. This new per-workspace URL has the format adb-<workspaceId>.<random number>.azuredatabricks.net. This URL is complementary to the existing regional URLs (<region>.azuredatabricks.net) that you have used up to now to access your workspaces. Both URLs continue to be supported. We do, however, strongly recommend that you use the new per-workspace URL, because it is region independent.

If you have existing scripts or automation tools that use the old regional URLs, we recommend that you change your scripts to use the new per-workspace URLs. We also recommend that you use the per-workspace URL in any future scripts or automation tools. However, because Azure Databricks continues to support the existing regional URLs, existing code will continue to work after the new URL rollout.

For details, see Per-workspace URLs.

MLflow tracking UI enhancement

April 23-30, 2020: Version 3.18

The MLflow UI now offers an option to delete child runs when you delete a root run.

Notebook usability improvements

April 23-30, 2020: Version 3.18

This release brings several usability improvements when working with notebooks:

• You can now select adjacent notebook cells using Shift + Up or Down for the previous and next cell respectively. Multi-selected cells can be copied, cut, deleted, and pasted.
• When you delete a cell, by default a delete confirmation dialog appears. You can now disable the confirmation dialog when you delete the cell by selecting the Do not show this again checkbox and clicking Confirm. You can also toggle the confirmation dialog with the Turn on command delete confirmation option in > User Settings > Developer.

Use tokens to authorize to the Databricks REST API (Public Preview)

April 23-30, 2020: Version 3.18

You can now use Microsoft Entra ID tokens to authorize to the Databricks Workspace API. Microsoft Entra ID tokens enable you to automate the creation and setup of new workspaces. Service principals are application objects in Microsoft Entra ID. You can also use service principals within your Azure Databricks workspaces to automate workflows.

For details, see Microsoft Entra ID (formerly Azure Active Directory) tokens.

Single user can view the Apache Spark logs on a passthrough cluster

April 23-30, 2020: Version 3.18

Previously, credential passthrough on a Standard mode cluster did not allow a single user to view the Apache Spark logs if they were not an admin user. Now a single user with attach permission can view Spark logs even if they are not an admin.

Databricks Connect now supports Databricks Runtime 6.5

April 20, 2020

Databricks Connect now supports Databricks Runtime 6.5.

Databricks Runtime 6.1 and 6.1 ML support ends

April 16, 2020

Support for Databricks Runtime 6.1 and Databricks Runtime 6.1 for Machine Learning ended on April 16. See Databricks support lifecycles.

Databricks Runtime 6.5 GA

April 14, 2020

Databricks Runtime 6.5 brings many library upgrades and new features, including:

• Operation metrics for all writes, updates, and deletes on a Delta table now appear in table history
• You can rate-limit the data processed in Delta Lake streaming micro-batches
• Snowflake connector is updated to 2.5.9

For more information, see the complete Databricks Runtime 6.5 (unsupported) release notes.

Databricks Runtime 6.5 for Machine Learning GA

April 14, 2020

Databricks Runtime 6.5 ML brings the following library upgrade:

• MLflow upgraded from 1.5.0 to 1.7.0

For more information, see the complete Databricks Runtime 6.5 for ML (unsupported) release notes.

Databricks Runtime 6.5 for Genomics GA

April 14, 2020

Databricks Runtime 6.5 for Genomics is built on top of Databricks Runtime 6.5.

Databricks Runtime 6.0 and 6.0 ML support ends

April 1, 2020

Support for Databricks Runtime 6.0 and Databricks Runtime 6.0 for Machine Learning ended on April 1. See Databricks support lifecycles.