Clone a workspace group to a new account group

This page provides guidance for workspace administrators who want to grant Consumer access users in a Azure Databricks workspace. It includes a Python script to help manage groups using the Databricks SDK for Python.

Background

Workspaces are commonly configured such that all users in a workspace are assigned both the Workspace access and Databricks SQL entitlements automatically upon provisioning. Because entitlements are additive, users only see the simplified workspace when Consumer access is their sole entitlement. If a group is granted any entitlement with greater privileges than Consumer access, then none of the users in that group will receive only the consumer experience.

This page provides a workflow to clone the existing users system group to a new group that retains Workspace access and Databricks SQL access entitlements. It then explains how to reconfigure the users group so newly provisioned users are assigned only the Consumer access entitlement by default. With this setup, users who require a higher level access must be added to the new group when they are added to the workspace.

Grant consumer access

If you have used the users group to grant authoring privileges to all of the workspace's users, use the following steps to allow some users to be granted the consumer access entitlement only.

1. Duplicate the workspace’s users system group:

   • Create a new account group containing all current members of the workspace’s users group.
   • Assign higher-privilege entitlements, such as Workspace access or Databricks SQL access to this new group instead of the default users group.

2. Remove higher-privilege entitlements from the users group: This allows you to add new users as consumers without them being automatically upgraded by group entitlements.

3. Add new users as consumers: Add users who should only have the Consumer access experience to the workspace.

Use the Databricks SDK for Python to automate managing entitlements

The following Python script automates the process of duplicating the users group and assigning appropriate entitlements. It uses the Databricks SDK for Python and requires a service principal with administrator privileges for both the account and the workspace, authenticated using OAuth. See Authorize interactive access to Azure Databricks resources with a user account using OAuth.

Prerequisites:

• Service principal with admin rights
• Environment variables set:
  • DATABRICKS_ACCOUNT_ID (UUID from account console URL)
  • DATABRICKS_WORKSPACE_ID (numerical ID from workspace URL)
  • DATABRICKS_CLIENT_ID (service principal client ID)
  • DATABRICKS_CLIENT_SECRET (service principal client secret)

Example script

import os
import databricks.sdk as dbx
from databricks.sdk.service import iam

# Set the Databricks account host URL for your account's cloud
DATABRICKS_HOST = "https://accounts.azuredatabricks.net
"

Fetch credentials from environment variables
DATABRICKS_ACCOUNT_ID = os.getenv("DATABRICKS_ACCOUNT_ID")
DATABRICKS_WORKSPACE_ID = os.getenv("DATABRICKS_WORKSPACE_ID")
DATABRICKS_CLIENT_ID = os.getenv("DATABRICKS_CLIENT_ID")
DATABRICKS_CLIENT_SECRET = os.getenv("DATABRICKS_CLIENT_SECRET")

# Initialize Databricks account client
account_client = dbx.AccountClient(
host=DATABRICKS_HOST,
account_id=DATABRICKS_ACCOUNT_ID,
client_id=DATABRICKS_CLIENT_ID,
client_secret=DATABRICKS_CLIENT_SECRET,
)

print(f"Authenticated to Databricks account {DATABRICKS_ACCOUNT_ID}")

# Get workspace and initialize workspace client
workspace = account_client.workspaces.get(workspace_id=DATABRICKS_WORKSPACE_ID)
workspace_name = workspace.workspace_name
workspace_client = account_client.get_workspace_client(workspace)

print(f"Authenticated to Databricks workspace {DATABRICKS_WORKSPACE_ID}, '{workspace_name}'")

def get_workspace_group(group_name):
group = list(workspace_client.groups.list(filter=f"displayName eq '{group_name}'"))
print(f"Found workspace group: {group.display_name}")
print(f"Workspace {group.display_name} has {len(group.members)} members")
return group

def clone_workspace_group_to_account(workspace_group_name, new_account_group_name):
workspace_group = get_workspace_group(workspace_group_name)
group = account_client.groups.create(
display_name=new_account_group_name, members=workspace_group.members
)
print(f"Created account group: {new_account_group_name}")
print(f"Cloned workspace group {workspace_group.display_name} to account group {group.display_name}")
print(f"Account {group.display_name} has {len(group.members)} members")
return group

def add_account_group_to_workspace(account_group, workspace):
permissions = account_client.workspace_assignment.update(
workspace_id=workspace.workspace_id,
principal_id=account_group.id,
permissions=[iam.WorkspacePermission.USER],
)
print(f"Added account group {account_group.display_name} to workspace {workspace.workspace_id}, {workspace.workspace_name}")
return permissions

# Clone workspace 'users' group to new account group '{workspace_name}-contributors'
account_group = clone_workspace_group_to_account(
"users", f"{workspace_name}-contributors"
)

# Add account group '{workspace_name}-contributors' to the workspace
permissions = add_account_group_to_workspace(account_group, workspace)

After you have duplicated your exisitng groups and reassinged permissions, you can grant Consumer access to the users, or any other, group so that new users are automatically granted that access when they are added to the workspace.

Note

Adjust group names and entitlements for your organization’s policies and naming conventions. Always test changes in a non-production environment before applying them broadly.