Run shell commands in Azure Databricks web terminal

Azure Databricks web terminal provides a convenient and highly interactive way for you to run shell commands, including Databricks CLI commands, and use editors, such as Vim or Emacs, on the Spark driver node. The web terminal can be used by many users on one compute. Example uses of the web terminal include monitoring resource usage and installing Linux packages.

Web terminal is disabled by default for all workspace users. To enable it, see Enable the web terminal.

Warning

Azure Databricks proxies the web terminal service from port 7681 on the compute’s Spark driver. This web proxy is intended for use only with the web terminal. If the port is occupied when the compute starts or if there is otherwise a conflict, the web terminal may not work as expected. If other web services are launched on port 7681, compute users may be exposed to potential security exploits. Neither Databricks nor Microsoft is responsible for any issues that result from the installation of unsupported software on a compute.

Requirements

• Your Azure Databricks workspace must have web terminal enabled.
• CAN ATTACH TO permission on a compute.
• The compute must use an access mode of Single user or No isolation shared.

Launch the web terminal

You can launch the web terminal from the compute details page or from a notebook.

From a notebook

To launch the web terminal from a notebook:

1. Connect the notebook to compute.
2. At the bottom of the notebook’s right sidebar, click the Open bottom panel icon .
3. Alternatively, click the attached compute drop-down, hover over the attached compute, then click Web Terminal.

The web terminal opens in a panel at the bottom of the screen. The buttons at the upper-right of the panel allow you to:

• Open a new terminal session in a new tab .
• Reload a terminal session .
• Close the bottom panel . To reopen the panel, click at the bottom of the right sidebar.

From the compute details page

To launch the web terminal from the compute details page:

1. On the workspace’s sidebar, click Compute.
2. On the All-purpose compute tab, click the name of the compute.
3. Click Start to start the compute.
4. On the Apps tab, click Web Terminal.

A new tab opens with the web terminal UI and the Bash prompt.

Use web terminal

In the web terminal, you can run commands from root inside the container of the compute driver node.

Each user can have up to 100 active web terminal sessions (tabs) open. Idle web terminal sessions may time out and the web terminal web application will reconnect, resulting in a new shell process. If you want to keep your Bash session, Databricks recommends using tmux.

Run Databricks CLI commands

You can also use the web terminal to run Databricks CLI commands. The available CLI is always the latest version, and authentication is based on the current user. The compute must meet the following requirements:

• The cluster must have Databricks Runtime 15.0 or above installed.
• The workspace must not be enabled for Private Link.

Launch the web terminal and run the following command to output information about the current user:

   databricks current-user me

Bundle commands are also available, which allows you to create and manage your Databricks Asset Bundles directly from the web terminal within the Databricks workspace. For example, to create, deploy, and run a simple bundle using the default template:

1. From the web terminal root, navigate to your workspace home and run bundle init:

   cd /Workspace/Users/someone@example.com
   databricks bundle init
   

2. Accept the default template prompts, then change to the bundle directory and deploy it:

   cd my_project
   databricks bundle deploy
   

   You can view the deployed my_project bundle in the Databricks workspace UI.

3. Finally, run the default job in the bundle:

   databricks bundle run my_project_job
   

   Navigate to Job Runs to see the running job.

Limitations

• Azure Databricks does not support running Spark jobs from the web terminal.

• Azure Databricks web terminal is not available in the following compute types:

  • Job compute
  • Serverless compute
  • Compute launched with the DISABLE_WEB_TERMINAL=true environment variable set.
  • Compute launched with access mode set to Shared.
  • Compute launched with the Spark configuration spark.databricks.pyspark.enableProcessIsolation set to true.

• There is a hard limit of 12 hours since the initial page load, after which any connection, even if active, will be terminated. You can refresh the web terminal to reconnect. Databricks recommends using tmux to preserve your shell session.

• ARM compute resources cannot use web terminals to access workspace files, including files in Git folders.

• Enabling Docker Container Services disables web terminal.