Share via

Set up Databricks Git folders

This article shows how to set up Databricks Git folders for version control. After you set up Git folders in your Databricks workspace, you can perform common Git operations such as clone, checkout, commit, push, pull, and branch management from the Databricks UI. You can also see diffs for your changes as you develop in Databricks.

Configure user settings

Databricks Git folders uses a personal access token (PAT) or equivalent OAuth credentials to authenticate with your Git provider to perform operations. To use Git folders, you must first configure your Git credentials in Databricks. See Configure Git credentials & connect a remote repo to Azure Databricks.

You can clone public remote repositories without Git credentials. To modify a public remote repository or to clone or modify a private remote repository, you must have a Git credential with Write (or greater) permissions for the remote repository.

Git folders are enabled by default. For more details on enabling or disabling Git folder support, see Enable or disable the Databricks Git folder feature.

Add or edit Git credentials in Databricks

  1. Select the down arrow next to the account name at the upper-right of your screen, and then select Settings.

  2. Select the Linked accounts tab.

  3. If you're adding credentials for the first time, follow the on-screen instructions.

    If you have previously entered credentials, click Config > Edit and go to the next step.

  4. In the Git provider drop-down, select the provider name.

  5. Depending on the provider selected, you might have the OAuth option and the personal access token (PAT) option. If you choose the OAuth option, complete the web authentication flow. If you choose the PAT option, enter your Git username or email. In the Token field, add the PAT from your Git provider. For details, see Configure Git credentials & connect a remote repo to Azure Databricks

    Important

    Databricks recommends that you use OAuth Git credentials. If you must use personal access tokens, you must set an expiration date.

  • For Azure DevOps, if you do not enter a token or app password, Git integration uses your Microsoft Entra ID token by default. If you enter an Azure DevOps personal access token, Git integration uses it instead. See Connect to an Azure DevOps repo using a token.

Note

After you update your Azure password, re-authenticate with Azure Databricks if you need the new authentication to work right away. If you do not re-authenticate, the Azure DevOps connection may not be validated for up to 24 hrs.

If your organization has SAML SSO enabled in GitHub, authorize your personal access token for SSO.

  1. Enter your username in the Git provider username field.
  2. Click Save.

You can also save a Git PAT token and username to Azure Databricks using the Databricks Repos API.

If you are unable to clone the repo and you're using Azure DevOps with Microsoft Entra ID authentication, see Issue with a conditional access policy (CAP) for Microsoft Entra ID.

Multiple Git credentials per user (Public Preview)

Azure Databricks supports multiple Git credentials per user, which enables easy switching between credentials for users on teams working with multiple Git providers or using multiple Git accounts for the same provider.

Databricks data intelligence platform

Limitations

  • You can use multiple Git credentials for jobs that use different Git providers.
  • If a job requires a Git credential that is not your default Git credential for that provider, you must use a service principal to schedule that job.
  • A service principal can only have one Git credential.
  • Only a single linked credential is allowed when connecting using the Databricks GitHub App.
  • A maximum of 10 Git credentials are allowed per user.

Network connectivity between Databricks Git folders and a Git provider

Git folders requires network connectivity to your Git provider to function. Ordinarily, this is over the internet and works without further configuration. However, you might have set up additional restrictions on your Git provider for controlling access. For example, you might have an IP allowlist in place, or you might host your own on-premises Git server using services like GitHub Enterprise (GHE), Bitbucket Server (BBS), or Gitlab Self-managed. Depending on your network hosting and configuration, your Git server might not be accessible via the internet.

Note

Security features in Git folders

Databricks Git folders have many security features. The following sections walk you through their setup and use:

  • Use of encrypted Git credentials
  • An allowlist
  • Workspace access control
  • Audit logging
  • Secrets detection

Bring your own key: Encrypt Git credentials

You can use Azure Key Vault to encrypt a Git personal access token (PAT) or other Git credentials. Using a key from an encryption service is referred to as a customer-managed key (CMK) or bring your own key (BYOK).

For more information, see Customer-managed keys for encryption.

Restrict usage to URLs in an allowlist

If you use Microsoft Entra ID for authenticating with Azure DevOps, the default allowlist restricts Git URLs to:

  • dev.azure.com
  • visualstudio.com

For Microsoft Entra ID with custom CNAMES or Git URL aliases, your workspace admin can configure a custom allowlist as shown in the following steps. If you use a custom allowlist, your workspace admin needs to add these URLs if you want to work with them: dev.azure.com and visualstudio.com.

A workspace admin can limit which remote repositories users can clone from and commit & push to. This helps prevent exfiltration of your code; for example, users cannot push code to an arbitrary repository if you have turned on the allowlist restrictions. You can also prevent users from using unlicensed code by restricting the clone operation to a list of allowed repositories.

To set up an allowlist:

  1. Go to the settings page.

  2. Click the Workspace admin tab (it is open by default).

  3. In the Development section, choose an option from Git URL allowlist permission:

    • Disabled (no restrictions): There are no checks against the allowlist.
    • Restrict Clone, Commit & Push to Allowed Git Repositories: Clone, commit, and push operations are allowed only for repository URLs in the allowlist.
    • Only Restrict Commit & Push to Allowed Git Repositories: Commit and push operations are allowed only for repository URLs in the allowlist. Clone and pull operations are not restricted.

    The Development pane under Admin Settings, used to set user Git access

  4. Click the Edit button next to Git URL allowlist: Empty list and enter a comma-separated list of URL prefixes.

    The Edit allow list button in the Development admin settings

  5. Click Save.

Note

  • The list you save overwrites the existing set of saved URL prefixes.
  • It can take up to 15 minutes for the changes to take effect.

Allow access to all repositories

To disable an existing allowlist and allow access to all repositories:

  1. Go to the settings page.
  2. Click the Workspace admin tab.
  3. In the Development section, under Git URL allowlist permission: select Disable (no restrictions).

Control access for a repo in your workspace

Note

Access control is available only in the Premium plan.

Set permissions for a repo to control access. Permissions for a repo apply to all content in that repo. You can assign five permission levels to files: NO PERMISSIONS, CAN READ, CAN RUN, CAN EDIT, and CAN MANAGE.

For more details on Git folder permissions, see Git folder ACLs.

Audit logging

When audit logging is enabled, audit events are logged when you interact with a Git folder. For example, an audit event is logged when you create, update, or delete a Git folder, when you list all Git folders associated with a workspace, and when you sync changes between your Git folder and the remote Git repo.

Secrets detection

Git folders scan code for access key IDs that begin with the prefix AKIA and warns the user before committing.

Move Git folder to trash (delete)

To delete a Git folder from your workspace:

  1. Right-click the Git folder, and then select Move to trash.

  2. In the dialog box, type the name of the Git folder you want to delete. Then, click Confirm & move to trash.

    Confirm Move to Trash dialog box.

Next steps