Select a cluster for the Databricks extension for Visual Studio Code

This article describes how to select an Azure Databricks cluster for the Databricks extension for Visual Studio Code to use. See What is the Databricks extension for Visual Studio Code?.

The following information assumes that you have already set up the Databricks extension for Visual Studio Code for authentication with your Azure Databricks workspace. See Authentication setup for the Databricks extension for VS Code.

Note

If you have already completed the VSCode extension for Databricks tutorial: Run Python on a cluster and as a job, and you want to keep using the Azure Databricks cluster from that tutorial, skip ahead to Select a workspace directory for the Databricks extension for Visual Studio Code.

With the extension and your code project opened, select an existing Azure Databricks cluster that you want to use, or create a new Azure Databricks cluster and use it.

Use an existing cluster

If you have an existing Azure Databricks cluster that you want to use, do one of the following:

• In the Clusters pane, do the following:

  Note

  The Clusters pane appears only if the views.cluster experimental feature is enabled. See Settings for the Databricks extension for Visual Studio Code.

  1. Next to the cluster that you want to use, click the plug (Attach cluster) icon.

     

     Tip

     If the cluster is not visible in the Clusters pane, click the filter (Filter clusters) icon to see All clusters, clusters that are Created by me, or Running clusters. Or, click the arrowed circle (Refresh) icon next to the filter icon.

     

  This procedure is complete. Continue with Select a workspace directory for the Databricks extension for Visual Studio Code.

• In the Configuration pane, do the following:

  1. Next to Cluster, click the gear (Configure cluster) icon.

     

  2. In the Command Palette, click the cluster that you want to use.

  Continue with Select a workspace directory for the Databricks extension for Visual Studio Code.

Create a new cluster

If you do not have an existing Azure Databricks cluster, or you want to create a new one and use it, do the following:

1. In the Configuration pane, next to Cluster, click the gear (Configure cluster) icon.

   

2. In the Command Palette, click Create New Cluster.

3. When prompted to open the external website (your Azure Databricks workspace), click Open.

4. If prompted, sign in to your Azure Databricks workspace.

5. Follow the instructions to create a cluster.

Note

Databricks recommends that you create a Personal Compute cluster. This enables you to start running workloads immediately, minimizing compute management overhead.

1. After the cluster is created and is running, go back to Visual Studio Code.
2. Do one of the following:

   • In the Clusters pane, next to the cluster that you want to use, click the plug (Attach cluster) icon.

     Note

     The Clusters pane appears only if the views.cluster experimental feature is enabled. See Settings for the Databricks extension for Visual Studio Code.

     

     Tip

     If the cluster is not visible, click the filter (Filter clusters) icon to see All clusters, clusters that are Created by me, or Running clusters. Or, click the arrowed circle (Refresh) icon.

     

     This procedure is complete. Continue with Select a workspace directory for the Databricks extension for Visual Studio Code.

   • In the Configuration pane, next to Cluster, click the gear (Configure cluster) icon.

     

     In the Command Palette, click the cluster that you want to use.

     Continue with Select a workspace directory for the Databricks extension for Visual Studio Code.