December 2021

These features and Azure Databricks platform improvements were released in December 2021.

Note

Releases are staged. Your Azure Databricks account may not be updated until a week or more after the initial release date.

Databricks Runtime 6.4 Extended Support series end-of-support date extended

December 31, 2021

Support for the Databricks Runtime 6.4 Extended Support series has been extended to June 30, 2022. A previous announcement stated that support would end on December 31, 2021. See All supported Databricks Runtime releases.

Databricks Runtime 5.5 Extended Support series reaches end of support

December 31, 2021

Support for the Databricks Runtime 5.5 Extended Support series ended on December 31. See Databricks runtime support lifecycles.

Databricks JDBC driver 2.6.21

December 24, 2021

We have released version 2.6.21 of the Databricks JDBC driver (download). This release includes an upgrade of the Log4j library to version 2.17.0.

Databricks Connector for Tableau 2021.4

** December 23, 2021**

We have released a new version of the Databricks connector in Tableau 2021.4. This release adds support for navigating a three-level namespace in the Unity Catalog.

Databricks Runtime 10.2 and 10.2 ML are GA; 10.2 Photon is Public Preview

December 22, 2021

Databricks Runtime 10.2 and Databricks Runtime 10.2 ML are now generally available. Databricks Runtime 10.2 Photon is in Public Preview.

See Databricks Runtime 10.2 (unsupported) and Databricks Runtime 10.2 for ML (unsupported).

Switzerland West region now available

December 20, 2021

You can now create workspaces in the Switzerland West region. This is a special region that requires an Azure region access request for your subscription.

Updated Markdown parser

December 13, 2021 - January 6, 2022: Version 3.61

The Markdown parser used in Azure Databricks notebooks has been updated. The new version is faster when rendering. Some behavior has changed. The new behavior is closer to the commonmark spec.

The primary changes are:

  • You must now put a space between the hashtags and the text of a heading.
  • When you mix HTML and Markdown code in a cell, you must put a newline after each HTML start tag.

For example, the Markdown code to produce this result:

correct rendering

is as shown:

%md

## Heading

<del>

*example text*
</del>

User interface improvements for Delta Live Tables

December 13, 2021 - January 6, 2022: Version 3.61

This release includes the following enhancements to the Delta Live Tables UI:

  • When you create a pipeline, you can directly edit the JSON specification in the Create Pipeline dialog by clicking the JSON button.
  • You can view tooltips for data quality metrics by hovering over data quality values in the processing graph view for your pipeline.

Additional metastores coming soon to multiple regions

December 9, 2021

The following metastores are coming soon to the following regions:

centralusc2:

  • consolidated-centralusc2-prod-metastore-1.mysql.database.azure.com
  • consolidated-centralusc2-prod-metastore-2.mysql.database.azure.com
  • consolidated-centralusc2-prod-metastore-3.mysql.database.azure.com

eastusc3:

  • consolidated-eastusc3-prod-metastore-1.mysql.database.azure.com
  • consolidated-eastusc3-prod-metastore-2.mysql.database.azure.com
  • consolidated-eastusc3-prod-metastore-3.mysql.database.azure.com

eastus2c2:

  • consolidated-eastus2c2-prod-metastore-1.mysql.database.azure.com
  • consolidated-eastus2c2-prod-metastore-2.mysql.database.azure.com
  • consolidated-eastus2c2-prod-metastore-3.mysql.database.azure.com

northeuropec2:

  • consolidated-northeuropec2-prod-metastore-1.mysql.database.azure.com
  • consolidated-northeuropec2-prod-metastore-2.mysql.database.azure.com
  • consolidated-northeuropec2-prod-metastore-3.mysql.database.azure.com

westeuropec2:

  • consolidated-westeuropec2-prod-metastore-1.mysql.database.azure.com
  • consolidated-westeuropec2-prod-metastore-2.mysql.database.azure.com
  • consolidated-westeuropec2-prod-metastore-3.mysql.database.azure.com

You should always include all relevant hostnames for each region you use in your user-defined routes (UDRs). See User-defined route settings for Azure Databricks.

Update SCC relay for Australia Southeast region

If you use the Australia Southeast region with secure cluster connectivity (SCC), the SCC relay address has been updated to tunnel.australiaeast.azuredatabricks.net. Review and update your user-defined routes (UDRs). When possible, Databricks recommends that UDRs use hostnames, rather than IP addresses. See User-defined route settings for Azure Databricks.

Databricks Runtime 8.3 series support extended

December 8, 2021

Support for Databricks Runtime 8.3 and Databricks Runtime 8.3 for Machine Learning has been extended from December 8, 2021 to January 20, 2022. See All supported Databricks Runtime releases.

Databricks Runtime 10.2 (Beta)

December 6, 2021

Databricks Runtime 10.2, 10.2 Photon, and 10.2 ML are now available as Beta releases.

See Databricks Runtime 10.2 (unsupported) and Databricks Runtime 10.2 for ML (unsupported).

Revert of recent breaking change that removed escaping and quotes from $ in environment variable values for cluster creation

December 3, 2021

The breaking change to environment variable behavior that was introduced in some workspaces last month will be reverted on December 3, 2021 from 01:00-03:00 UTC.