Authentication using Azure Databricks personal access tokens

  • Article
  • 2 minutes to read

To authenticate to and access Databricks REST APIs, you can use Azure Databricks personal access tokens or Azure Active Directory (Azure AD) tokens.

This article discusses how to use Azure Databricks personal access tokens. For Azure AD tokens, see Azure AD tokens.

Important

Tokens replace passwords in an authentication flow and should be protected like passwords. To protect tokens, Databricks recommends that you store tokens in:

Note

As a security best practice, when authenticating with automated tools, systems, scripts, and apps, Databricks recommends you use access tokens belonging to service principals instead of workspace users. To create access tokens for service principals, see Manage access tokens for a service principal.

Requirements

Token-based authentication is enabled by default for all Azure Databricks accounts launched after January 2018. If token-based authentication is disabled, your administrator must enable it before you can perform the tasks described in Manage personal access tokens.

Generate a personal access token

See Azure Databricks personal access tokens.

You can also generate and revoke tokens using the Token API 2.0.

The number of personal access tokens per user is limited to 600 per workspace.

Revoke a personal access token

This section describes how to revoke personal access tokens using the Azure Databricks UI. You can also generate and revoke access tokens using the Token API 2.0.

  1. Click your username in the top bar of your Azure Databricks workspace and select User Settings from the drop down.
  2. Go to the Access Tokens tab.
  3. Click x for the token you want to revoke.
  4. On the Revoke Token dialog, click the Revoke Token button.

Use a personal access token to access the Databricks REST API

You can store a personal access token in a .netrc file and use it in curl or pass it to the Authorization: Bearer header.

Store tokens in a .netrc file and use them in curl

Create a .netrc file with machine, login, and password properties:

machine <databricks-instance>
login token
password <token-value>

where:

  • <databricks-instance> is the instance ID portion of the workspace URL for your Azure Databricks deployment. For example, if the workspace URL is https://adb-1234567890123456.7.azuredatabricks.net then <databricks-instance> is adb-1234567890123456.7.azuredatabricks.net.
  • token is the literal string token.
  • <token-value> is the value of your token, for example dapi1234567890ab1cde2f3ab456c7d89efa.

The result looks like this:

machine adb-1234567890123456.7.azuredatabricks.net
login token
password dapi1234567890ab1cde2f3ab456c7d89efa

For multiple machine/token entries, add one line per entry, with the machine, login and password properties for each machine/token matching pair on the same line. The result looks like this:

machine adb-1234567890123456.7.azuredatabricks.net login token password dapi1234567890ab1cde2f3ab456c7d89efa
machine adb-2345678901234567.8.azuredatabricks.net login token password dapi2345678901cd2efa3b4cd567e8f90abc
machine adb-3456789012345678.9.azuredatabricks.net login token password dapi3456789012de3fab4c5de678f9a01bcd

This example invokes the .netrc file by using --netrc (you can also use -n) in the curl command. It uses the specified workspace URL to find the matching machine entry in the .netrc file.

curl --netrc -X GET https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/clusters/list

Pass token to Bearer authentication

You can include the token in the header using Bearer authentication. You can use this approach with curl or any client that you build. For the latter, see Upload a big file into DBFS.

This example uses Bearer authentication to list all available clusters in the specified workspace.

export DATABRICKS_TOKEN=dapi1234567890ab1cde2f3ab456c7d89efa

curl -X GET --header "Authorization: Bearer $DATABRICKS_TOKEN" \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/clusters/list