Share via

Authentication setup for the Databricks extension for VS Code

  • Article

This article describes how to set up authentication between the Databricks extension for Visual Studio Code and your Azure Databricks workspace. See What is the Databricks extension for Visual Studio Code?

The Databricks extension for Visual Studio Code implements portions of the Databricks client unified authentication standard, a consolidated and consistent architectural and programmatic approach to authentication. This approach helps make setting up and automating authentication with Azure Databricks more centralized and predictable. It enables you to configure Azure Databricks authentication once and then use that configuration across multiple Azure Databricks tools and SDKs without further authentication configuration changes.

The following information assumes that you have already installed the Databricks extension for Visual Studio Code. See Install the Databricks extension for Visual Studio Code.

Note

If you have already completed the VSCode extension for Databricks tutorial: Run Python on a cluster and as a job, and want to keep using the Azure Databricks personal access token authentication that you set up in that tutorial, skip ahead to Select a cluster for the Databricks extension for Visual Studio Code.

Depending on the type of authentication that you want to use, complete the instructions for your target Azure Databricks authentication type.

Note

The Databricks extension for Visual Studio Code does not yet support authentication using Azure managed identities (formerly Azure Managed Service Identities (MSI)).

Databricks Connect supports authentication using Azure managed identities. However, you cannot use the Databricks Connect integration within the Databricks extension for Visual Studio Code to do Azure managed identity authentication. To use Databricks Connect with Visual Studio Code by itself, separate from the Databricks extension for Visual Studio Code, see Use Visual Studio Code with Databricks Connect for Python.

Set up authentication with a configuration profile

The following instructions assume that you have already set up an Azure Databricks configuration profile with the required fields for your Azure Databricks authentication type. See Azure Databricks configuration profiles and the Profile tab or section for your authentication type in Authentication for Azure Databricks automation - overview.

For example, if you want to authenticate by using Azure Databricks personal access token authentication, create your personal access token by doing the following:

  1. In your Azure Databricks workspace, click your Azure Databricks username in the top bar, and then select Settings from the drop down.
  2. Click Developer.
  3. Next to Access tokens, click Manage.
  4. Click Generate new token.
  5. (Optional) Enter a comment that helps you to identify this token in the future, and change the token’s default lifetime of 90 days. To create a token with no lifetime (not recommended), leave the Lifetime (days) box empty (blank).
  6. Click Generate.
  7. Copy the displayed token to a secure location, and then click Done.

Note

Be sure to save the copied token in a secure location. Do not share your copied token with others. If you lose the copied token, you cannot regenerate that exact same token. Instead, you must repeat this procedure to create a new token. If you lose the copied token, or you believe that the token has been compromised, Databricks strongly recommends that you immediately delete that token from your workspace by clicking the trash can (Revoke) icon next to the token on the Access tokens page.

If you are not able to create or use tokens in your workspace, this might be because your workspace administrator has disabled tokens or has not given you permission to create or use tokens. See your workspace administrator or the following:

And, for example, to create a configuration profile named DEFAULT for your personal access token, do the following:

Note

The following procedure uses the Databricks CLI to create an Azure Databricks configuration profile with the name DEFAULT. If you already have a DEFAULT configuration profile, this procedure overwrites your existing DEFAULT configuration profile.

To check whether you already have a DEFAULT configuration profile, and to view this profile’s settings if it exists, use the Databricks CLI to run the command databricks auth env --profile DEFAULT.

To create a configuration profile with a name other than DEFAULT, replace the DEFAULT part of --profile DEFAULT in the following databricks configure command with a different name for the configuration profile.

  1. Use the Databricks CLI to create an Azure Databricks configuration profile named DEFAULT that uses Azure Databricks personal access token authentication. To do this, run the following command:

    databricks configure --profile DEFAULT

  2. For the prompt Databricks Host, enter your Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net.

  3. For the prompt Personal Access Token, enter the Azure Databricks personal access token for your workspace.

After you have your Azure Databricks configuration profile ready, with your project and the extension opened, do the following:

  1. In the Configuration pane, click Configure Databricks.

    Note

    If Configure Databricks is not visible, click the gear (Configure workspace) icon next to Configuration instead.

    Gear icon to configure workspace settings 1

  2. In the Command Palette, for Databricks Host, enter your per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net. Then press Enter.

  3. Select your target Azure Databricks configuration profile in the list for the URL.

The extension creates a hidden folder in your project named .databricks if it does not already exist. The extension also creates in this folder a file named project.json if it does not already exist. This file contains the URL that you entered, along with some Azure Databricks authentication details that the Databricks extension for Visual Studio Code needs to operate.

The extension also adds a hidden .gitignore file to the project if the file does not exist or if an existing .gitignore cannot be found in any parent folders. If a new .gitignore file is created, the extension adds a .databricks/ entry to this new file. If the extension finds an existing .gitignore file, it adds a .databricks/ entry to the existing file.

Continue with Select a cluster for the Databricks extension for Visual Studio Code.

Set up OAuth U2M authentication

Azure Databricks supports OAuth user to machine (U2M) authentication. This enables you to generate short-lived (one hour) OAuth access tokens, which eliminates the risk of accidentally exposing longer-lived tokens such as Azure Databricks personal access tokens through version control checkins or other means. This also enables better server-side session invalidation and scoping.

To complete the setup for OAuth U2M authentication, with your project and the extension opened, do the following:

  1. In the Configuration pane, click Configure Databricks.

    Note

    If Configure Databricks is not visible, click the gear (Configure workspace) icon next to Configuration instead.

    Gear icon to configure workspace settings 1

  2. In the Command Palette, for Databricks Host, enter your per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net. Then press Enter.

  3. Select OAuth (user to machine).

  4. Complete the on-screen instructions in your web browser to finish authenticating with your Azure Databricks account. If prompted, allow all-apis access.

The extension creates a hidden folder in your project named .databricks if it does not already exist. The extension also creates in this folder a file named project.json if it does not already exist. This file contains the URL that you entered, along with some Azure Databricks authentication details that the Databricks extension for Visual Studio Code needs to operate.

The extension also adds a hidden .gitignore file to the project if the file does not exist or if an existing .gitignore cannot be found in any parent folders. If a new .gitignore file is created, the extension adds a .databricks/ entry to this new file. If the extension finds an existing .gitignore file, it adds a .databricks/ entry to the existing file.

Continue with Select a cluster for the Databricks extension for Visual Studio Code.

Set up Azure CLI authentication

Azure Databricks supports authentication with the Azure CLI installed locally. This enables you to generate short-lived (one hour) Microsoft Entra ID (formerly Azure Active Directory) tokens, which eliminates the risk of accidentally exposing longer-lived tokens such as Azure Databricks personal access tokens through version control checkins or other means.

With your project and the extension opened, and the Azure CLI installed locally, do the following:

  1. In the Configuration pane, click Configure Databricks.

    Note

    If Configure Databricks is not visible, click the gear (Configure workspace) icon next to Configuration instead.

    Gear icon to configure workspace settings 1

  2. In the Command Palette, for Databricks Host, enter your per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net. Then press Enter.

  3. Select Azure CLI.

The extension creates a hidden folder in your project named .databricks if it does not already exist. The extension also creates in this folder a file named project.json if it does not already exist. This file contains the URL that you entered, along with some Azure Databricks authentication details that the Databricks extension for Visual Studio Code needs to operate.

The extension also adds a hidden .gitignore file to the project if the file does not exist or if an existing .gitignore cannot be found in any parent folders. If a new .gitignore file is created, the extension adds a .databricks/ entry to this new file. If the extension finds an existing .gitignore file, it adds a .databricks/ entry to the existing file.

Continue with Select a cluster for the Databricks extension for Visual Studio Code.