Configuration profiles for the Databricks CLI

Note

Databricks CLI use is subject to the Databricks License and Databricks Privacy Notice, including any Usage Data provisions.

This article describes how to use the Databricks CLI with configuration profiles. It assumes that you have already installed the Databricks CLI and created a Databricks configuration profiles file. See Install or update the Databricks CLI and Azure Databricks configuration profiles.

Get information about configuration profiles

Adding multiple configuration profiles to the .databrickscfg file enables you to quickly run commands across various workspaces by specifying the target configuration profile's name in the command's --profile or -p flag. If you do not specify a profile, the CLI selects one in the following order:

  1. The DATABRICKS_CONFIG_PROFILE environment variable, if set.
  2. The default_profile setting in the [__settings__] section of .databrickscfg, if present. You can set this with databricks auth switch. See databricks auth switch.
  3. The DEFAULT profile.

Tip

You can press Tab after --profile or -p to display a list of existing available configuration profiles from which to choose.

For example, you could have a configuration profile named DEV that references an Azure Databricks workspace that you use for development workloads and a separate configuration profile named PROD that references a different Azure Databricks workspace that you use for production workloads.

By default, the Databricks CLI looks for the .databrickscfg file in your ~ (your user home) folder on Unix, Linux, or macOS, or your %USERPROFILE% (your user home) folder on Windows. To change the default path of the .databrickscfg file, set the environment variable DATABRICKS_CONFIG_FILE to a different path. See .databrickscfg-specific fields.

To get information about an existing configuration profile, run the auth describe command:

databricks auth describe --profile <configuration-profile-name>

For example, here is the output for a profile that is configured with Azure Databricks personal access token authentication:

Host: https://dbc-a1b2345c-d6e7.cloud.databricks.com
User: user@example.com
Authenticated with: pat
-----
Current configuration:
  ✓ host: https://dbc-a1b2345c-d6e7.cloud.databricks.com (from ~/.databrickscfg config file)
  ✓ token: ******** (from ~/.databrickscfg config file)
  ✓ profile: <configuration-profile-name> (from --profile flag)
  ✓ auth_type: pat

Note

The auth env command is deprecated. Use databricks auth describe to inspect a profile's configuration, or databricks auth token to retrieve a cached OAuth access token.

To get information about all available profiles, run the auth profiles command:

databricks auth profiles

Name                     Host                                            Valid
DEFAULT                  https://dbc-a1b2345c-d6e7.cloud.databricks.com  YES
Development              https://dbc-a1b2345c-d6e7.cloud.databricks.com  YES
Staging                  https://dbc-a1b2345c-d6e7.cloud.databricks.com  YES
Production               https://dbc-a1b2345c-d6e7.cloud.databricks.com  YES

The output of the auth profiles command does not display any access tokens. To display token details for a specific profile, run databricks auth describe --profile <profile-name> or databricks auth token --profile <profile-name>.

Important

The Databricks CLI does not work with a .netrc file. You can have a .netrc file in your environment for other purposes, but the Databricks CLI will not use that .netrc file.

Test your configuration profiles

To test your configuration profiles and verify that you have set up authentication correctly, run a command that connects to a workspace.

If you don't specify a profile, the default profile is used. For example, the following command lists the available Databricks Runtime versions for the Azure Databricks workspace that is associated with your DEFAULT profile.

Note

This command assumes that you do not have any environment variables set that take precedence over the settings in your DEFAULT profile. For more information, see Authentication order of evaluation.

databricks clusters spark-versions

To verify a specific configuration profile, provide the profile name using the -p flag.

databricks clusters spark-versions -p PROD
  • Last updated on