Authentication setup for the Databricks extension for VS Code
This article describes how to set up authentication between the Databricks extension for Visual Studio Code and your Azure Databricks workspace. See What is the Databricks extension for Visual Studio Code?
The Databricks extension for Visual Studio Code implements portions of the Databricks client unified authentication standard, a consolidated and consistent architectural and programmatic approach to authentication. This approach helps make setting up and automating authentication with Azure Databricks more centralized and predictable. It enables you to configure Azure Databricks authentication once and then use that configuration across multiple Azure Databricks tools and SDKs without further authentication configuration changes.
The following information assumes that you have already installed the Databricks extension for Visual Studio Code. See Install the Databricks extension for Visual Studio Code.
Note
If you have already completed the VSCode extension for Databricks tutorial: Run Python on a cluster and as a job, and want to keep using the Azure Databricks personal access token authentication that you set up in that tutorial, skip ahead to Select a cluster for the Databricks extension for Visual Studio Code.
Depending on the type of authentication that you want to use, complete the instructions for your target Azure Databricks authentication type.
- For the following authentication types, skip ahead to Set up authentication with a configuration profile:
- For OAuth user-to-machine (U2M) authentication, skip ahead to Set up OAuth U2M authentication.
- For Azure CLI authentication, skip ahead to Set up Azure CLI authentication.
Note
The Databricks extension for Visual Studio Code does not yet support authentication using Azure managed identities (formerly Azure Managed Service Identities (MSI)).
Databricks Connect supports authentication using Azure managed identities. However, you cannot use the Databricks Connect integration within the Databricks extension for Visual Studio Code to do Azure managed identity authentication. To use Databricks Connect with Visual Studio Code by itself, separate from the Databricks extension for Visual Studio Code, see Use Visual Studio Code with Databricks Connect for Python.
Set up authentication with a configuration profile
The following instructions assume that you have already set up an Azure Databricks configuration profile with the required fields for your Azure Databricks authentication type. See Azure Databricks configuration profiles and the Profile tab or section for your authentication type in Authentication for Azure Databricks automation - overview.
For example, if you want to authenticate by using Azure Databricks personal access token authentication, create your personal access token by doing the following:
- In your Azure Databricks workspace, click your Azure Databricks username in the top bar, and then select Settings from the drop down.
- Click Developer.
- Next to Access tokens, click Manage.
- Click Generate new token.
- (Optional) Enter a comment that helps you to identify this token in the future, and change the token’s default lifetime of 90 days. To create a token with no lifetime (not recommended), leave the Lifetime (days) box empty (blank).
- Click Generate.
- Copy the displayed token to a secure location, and then click Done.
Note
Be sure to save the copied token in a secure location. Do not share your copied token with others. If you lose the copied token, you cannot regenerate that exact same token. Instead, you must repeat this procedure to create a new token. If you lose the copied token, or you believe that the token has been compromised, Databricks strongly recommends that you immediately delete that token from your workspace by clicking the trash can (Revoke) icon next to the token on the Access tokens page.
If you are not able to create or use tokens in your workspace, this might be because your workspace administrator has disabled tokens or has not given you permission to create or use tokens. See your workspace administrator or the following:
And, for example, to create a configuration profile named DEFAULT
for your personal access token, do the following:
Note
The following procedure uses the Databricks CLI to create an Azure Databricks configuration profile with the name DEFAULT
. If you already have a DEFAULT
configuration profile, this procedure overwrites your existing DEFAULT
configuration profile.
To check whether you already have a DEFAULT
configuration profile, and to view this profile’s settings if it exists, use the Databricks CLI to run the command databricks auth env --profile DEFAULT
.
To create a configuration profile with a name other than DEFAULT
, replace the DEFAULT
part of --profile DEFAULT
in the following databricks configure
command with a different name for the configuration profile.
Use the Databricks CLI to create an Azure Databricks configuration profile named
DEFAULT
that uses Azure Databricks personal access token authentication. To do this, run the following command:databricks configure --profile DEFAULT
For the prompt Databricks Host, enter your Azure Databricks per-workspace URL, for example
https://adb-1234567890123456.7.azuredatabricks.net
.For the prompt Personal Access Token, enter the Azure Databricks personal access token for your workspace.
After you have your Azure Databricks configuration profile ready, with your project and the extension opened, do the following:
In the Configuration pane, click Configure Databricks.
Note
If Configure Databricks is not visible, click the gear (Configure workspace) icon next to Configuration instead.
In the Command Palette, for Databricks Host, enter your per-workspace URL, for example
https://adb-1234567890123456.7.azuredatabricks.net
. Then press Enter.Select your target Azure Databricks configuration profile in the list for the URL.
The extension creates a hidden folder in your project named .databricks
if it does not already exist. The extension also creates in this folder a file named project.json
if it does not already exist. This file contains the URL that you entered, along with some Azure Databricks authentication details that the Databricks extension for Visual Studio Code needs to operate.
The extension also adds a hidden .gitignore
file to the project if the file does not exist or if an existing .gitignore
cannot be found in any parent folders. If a new .gitignore
file is created, the extension adds a .databricks/
entry to this new file. If the extension finds an existing .gitignore
file, it adds a .databricks/
entry to the existing file.
Continue with Select a cluster for the Databricks extension for Visual Studio Code.
Set up OAuth U2M authentication
Azure Databricks supports OAuth user to machine (U2M) authentication. This enables you to generate short-lived (one hour) OAuth access tokens, which eliminates the risk of accidentally exposing longer-lived tokens such as Azure Databricks personal access tokens through version control checkins or other means. This also enables better server-side session invalidation and scoping.
To complete the setup for OAuth U2M authentication, with your project and the extension opened, do the following:
In the Configuration pane, click Configure Databricks.
Note
If Configure Databricks is not visible, click the gear (Configure workspace) icon next to Configuration instead.
In the Command Palette, for Databricks Host, enter your per-workspace URL, for example
https://adb-1234567890123456.7.azuredatabricks.net
. Then press Enter.Select OAuth (user to machine).
Complete the on-screen instructions in your web browser to finish authenticating with your Azure Databricks account. If prompted, allow all-apis access.
The extension creates a hidden folder in your project named .databricks
if it does not already exist. The extension also creates in this folder a file named project.json
if it does not already exist. This file contains the URL that you entered, along with some Azure Databricks authentication details that the Databricks extension for Visual Studio Code needs to operate.
The extension also adds a hidden .gitignore
file to the project if the file does not exist or if an existing .gitignore
cannot be found in any parent folders. If a new .gitignore
file is created, the extension adds a .databricks/
entry to this new file. If the extension finds an existing .gitignore
file, it adds a .databricks/
entry to the existing file.
Continue with Select a cluster for the Databricks extension for Visual Studio Code.
Set up Azure CLI authentication
Azure Databricks supports authentication with the Azure CLI installed locally. This enables you to generate short-lived (one hour) Microsoft Entra ID (formerly Azure Active Directory) tokens, which eliminates the risk of accidentally exposing longer-lived tokens such as Azure Databricks personal access tokens through version control checkins or other means.
With your project and the extension opened, and the Azure CLI installed locally, do the following:
In the Configuration pane, click Configure Databricks.
Note
If Configure Databricks is not visible, click the gear (Configure workspace) icon next to Configuration instead.
In the Command Palette, for Databricks Host, enter your per-workspace URL, for example
https://adb-1234567890123456.7.azuredatabricks.net
. Then press Enter.Select Azure CLI.
The extension creates a hidden folder in your project named .databricks
if it does not already exist. The extension also creates in this folder a file named project.json
if it does not already exist. This file contains the URL that you entered, along with some Azure Databricks authentication details that the Databricks extension for Visual Studio Code needs to operate.
The extension also adds a hidden .gitignore
file to the project if the file does not exist or if an existing .gitignore
cannot be found in any parent folders. If a new .gitignore
file is created, the extension adds a .databricks/
entry to this new file. If the extension finds an existing .gitignore
file, it adds a .databricks/
entry to the existing file.
Continue with Select a cluster for the Databricks extension for Visual Studio Code.
Feedback
https://aka.ms/ContentUserFeedback.
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see:Submit and view feedback for