Migrate SCIM API calls to IAM v2

This page describes how to move scripts that manage users, groups, service principals, group memberships, entitlements, and workspace assignments from the System for Cross-domain Identity Management (SCIM) APIs to the identity and access management (IAM) v2 APIs. It covers REST calls, the Databricks SDK for Python, the Databricks SDK for Go, the Databricks SDK for Java, and the Databricks CLI.

For Terraform configurations, see Migrate Terraform identity resources to IAM v2.

Requirements

  • Databricks SDK for Python 0.159.0 and above.
  • Databricks SDK for Go 0.192.0 and above.
  • Databricks SDK for Java 0.173.0 and above.
  • Databricks CLI 1.24.0 and above.

Key differences

  • One API per resource: Each SCIM resource maps to a set of create, get, list, update, and delete methods. Group memberships and workspace entitlements are separate resources, so you don't send SCIM PATCH operations.
  • Workspace-level and account-level methods: Call the workspace-level methods on a workspace URL and the account-level methods on the account console URL. Users, groups, and service principals belong to the account. The workspace-level methods manage the account that contains the workspace you call.
  • Workspace access is separate: Creating a user, group, or service principal adds it to the account only. To give it access to a workspace, create a workspace assignment.
  • Account groups only: The workspace-level SCIM API creates workspace-local groups. IAM v2 creates and manages account groups, including from the workspace-level methods. The caller also gets the group manager role on each group it creates, on a best-effort basis.
  • Resource names: Each object has a name, such as users/123 at the workspace level or accounts/<account-id>/users/123 at the account level. Get, update, and delete take the name.
  • Parents: Group member methods take the group's name as parent, such as groups/123. Account-level list and create methods take accounts/<account-id> as parent. Workspace-level list and create methods take an optional parent that defaults to your account.
  • Same IDs: user_id, group_id, service_principal_id, and principal_id are the same IDs that SCIM returns in id.
  • Field names: Fields use snake_case, for example username instead of userName and group_name instead of displayName. A user's email is its username, and a user's displayName is full_name.given_name followed by full_name.family_name.
  • Updates: Update methods take an update_mask that lists the fields to change, for example full_name.given_name,account_user_status.
  • Filters and paging: List filters use snake_case field names with eq, for example username eq "jane.doe@example.com". Users support username eq and external_id eq, and groups support group_name eq and external_id eq. Results are paged with page_size (1000 at most) and page_token, not startIndex and count.

Endpoint mapping

The following table maps each current call to the workspace-level IAM v2 call. In the table, /scim/v2 stands for /api/2.0/preview/scim/v2 in a workspace and /api/2.0/accounts/{account-id}/scim/v2 in the account. The last two rows are calls to the account permission assignment API.

To get the account-level IAM v2 call:

  • Add accounts/<account-id>/ after /api/2.0/identity/. For example, GET /api/2.0/identity/users becomes GET /api/2.0/identity/accounts/<account-id>/users.
  • For workspace assignments, add accounts/<account-id>/workspaces/<workspace-id>/ after /api/2.0/identity/ instead.

For the request and response fields of each call, see the IAM v2 API reference:

Current call IAM v2 call
POST /scim/v2/Users POST /api/2.0/identity/users
GET /scim/v2/Users?filter=userName eq "..." GET /api/2.0/identity/users?filter=username eq "..."
GET /scim/v2/Users/{id} GET /api/2.0/identity/users/{id}
PATCH or PUT /scim/v2/Users/{id} PATCH /api/2.0/identity/users/{id}?update_mask=...
DELETE /scim/v2/Users/{id} DELETE /api/2.0/identity/users/{id}
/scim/v2/Groups... /api/2.0/identity/groups...
/scim/v2/ServicePrincipals... /api/2.0/identity/service-principals...
PATCH /scim/v2/Groups/{id} with add on members POST /api/2.0/identity/groups/{id}/direct-members
PATCH /scim/v2/Groups/{id} with remove on members DELETE /api/2.0/identity/groups/{id}/direct-members/{member-id}
GET /scim/v2/Groups/{id} to read members GET /api/2.0/identity/groups/{id}/direct-members
GET /scim/v2/Users/{id} to read groups GET /api/2.0/identity/principals/{id}/transitive-parent-groups, which returns direct and nested parent groups
PATCH on entitlements for a user, group, or service principal PATCH /api/2.0/identity/workspace-assignments/<principal-id> to update, or POST /api/2.0/identity/workspace-assignments to create
PUT /api/2.0/accounts/{account-id}/workspaces/{workspace-id}/permissionassignments/principals/{id} POST /api/2.0/identity/accounts/{account-id}/workspaces/{workspace-id}/workspace-assignments
DELETE on the same permission assignment DELETE /api/2.0/identity/accounts/{account-id}/workspaces/{workspace-id}/workspace-assignments/{id}

Warning

Deleting a user, group, or service principal deletes it from the account and removes it from every workspace, even when you call the workspace-level method. To remove a principal from one workspace only, delete its workspace assignment.

The following table maps the SDK and Databricks CLI clients. Workspace-level IAM v2 methods end in _proxy or Proxy, such as create_user_proxy in Python and CreateUserProxy in Go.

Client SCIM clients IAM v2 client
Databricks SDK for Python users, groups, and service_principals, or users_v2, groups_v2, and service_principals_v2. workspace_assignment in the account client. workspace_iam_v2 in the workspace client, iam_v2 in the account
Databricks SDK for Go Users, Groups, and ServicePrincipals, or UsersV2, GroupsV2, and ServicePrincipalsV2. WorkspaceAssignment in the account client. WorkspaceIamV2 in the workspace client, IamV2 in the account
Databricks SDK for Java users(), groups(), and servicePrincipals(), or usersV2(), groupsV2(), and servicePrincipalsV2(). workspaceAssignment() in the account client. workspaceIamV2() in the workspace client, iamV2() in the account
Databricks CLI databricks users, databricks groups, and databricks service-principals. databricks account workspace-assignment in the account. databricks workspace-iam-v2, databricks account iam-v2

Entitlement mapping

Entitlements move from the user, group, or service principal to its workspace assignment, which lists every entitlement granted directly to the principal in that workspace.

SCIM entitlement or permission IAM v2 entitlement
workspace-access WORKSPACE_ACCESS
Permission USER on a permission assignment An assignment with the entitlements the principal needs, such as WORKSPACE_ACCESS
workspace-consume WORKSPACE_CONSUME
databricks-sql-access DATABRICKS_SQL_ACCESS
allow-cluster-create ALLOW_CLUSTER_CREATE
allow-instance-pool-create ALLOW_INSTANCE_POOL_CREATE
Permission ADMIN on a permission assignment WORKSPACE_ADMIN
Membership in the workspace admins group No change. Membership in the admins group still grants WORKSPACE_ADMIN.

Examples

The examples use the workspace-level methods. Replace <workspace-url> with your workspace URL, and authenticate as described in Authorize access to Azure Databricks resources.

Create a user

To create a user, run the following. IAM v2 requires username, full_name, and account_user_status on create, so a request that sends only username fails.

curl

curl -X POST "https://<workspace-url>/api/2.0/identity/users" \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  -d '{
    "username": "jane.doe@example.com",
    "full_name": {"given_name": "Jane", "family_name": "Doe"},
    "account_user_status": "ACTIVE"
  }'

Python

from databricks.sdk import WorkspaceClient
from databricks.sdk.service import iamv2

w = WorkspaceClient()

user = w.workspace_iam_v2.create_user_proxy(
    user=iamv2.User(
        username="jane.doe@example.com",
        full_name=iamv2.UserFullName(given_name="Jane", family_name="Doe"),
        account_user_status=iamv2.State.ACTIVE,
    )
)
print(user.name)  # users/<user-id>

Go

user, err := w.WorkspaceIamV2.CreateUserProxy(ctx, iamv2.CreateUserProxyRequest{
	User: iamv2.User{
		Username:          "jane.doe@example.com",
		FullName:          iamv2.UserFullName{GivenName: "Jane", FamilyName: "Doe"},
		AccountUserStatus: iamv2.StateActive,
	},
})

Java

User user = w.workspaceIamV2().createUserProxy(
    new CreateUserProxyRequest().setUser(
        new User()
            .setUsername("jane.doe@example.com")
            .setFullName(new UserFullName().setGivenName("Jane").setFamilyName("Doe"))
            .setAccountUserStatus(State.ACTIVE)));

CLI

databricks workspace-iam-v2 create-user-proxy --json '{
  "username": "jane.doe@example.com",
  "full_name": {"given_name": "Jane", "family_name": "Doe"},
  "account_user_status": "ACTIVE"
}'

Find a user by username

To look up a user by username, run the following:

curl

curl -G "https://<workspace-url>/api/2.0/identity/users" \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  --data-urlencode 'filter=username eq "jane.doe@example.com"'

Python

for user in w.workspace_iam_v2.list_users_proxy(filter='username eq "jane.doe@example.com"'):
    print(user.name, user.user_id)

Go

users, err := w.WorkspaceIamV2.ListUsersProxyAll(ctx, iamv2.ListUsersProxyRequest{
	Filter: `username eq "jane.doe@example.com"`,
})
for _, user := range users {
	fmt.Println(user.Name, user.UserId)
}

Java

for (User user : w.workspaceIamV2().listUsersProxy(
    new ListUsersProxyRequest().setFilter("username eq \"jane.doe@example.com\""))) {
  System.out.println(user.getName());
}

CLI

databricks workspace-iam-v2 list-users-proxy --filter 'username eq "jane.doe@example.com"'

The SDKs and the Databricks CLI fetch every page for you. In REST, repeat the call with page_token set to the previous next_page_token until it's empty.

Update a user

To change a user's name and deactivate the user in the account, run the following:

curl

curl -X PATCH "https://<workspace-url>/api/2.0/identity/users/<user-id>?update_mask=full_name.family_name,account_user_status" \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  -d '{"username": "jane.doe@example.com", "full_name": {"family_name": "Smith"}, "account_user_status": "INACTIVE"}'

Python

w.workspace_iam_v2.update_user_proxy(
    name="users/<user-id>",
    user=iamv2.User(
        username="jane.doe@example.com",
        full_name=iamv2.UserFullName(family_name="Smith"),
        account_user_status=iamv2.State.INACTIVE,
    ),
    update_mask="full_name.family_name,account_user_status",
)

Go

_, err := w.WorkspaceIamV2.UpdateUserProxy(ctx, iamv2.UpdateUserProxyRequest{
	Name: "users/<user-id>",
	User: iamv2.User{
		Username:          "jane.doe@example.com",
		FullName:          iamv2.UserFullName{FamilyName: "Smith"},
		AccountUserStatus: iamv2.StateInactive,
	},
	UpdateMask: "full_name.family_name,account_user_status",
})

Java

w.workspaceIamV2().updateUserProxy(
    new UpdateUserProxyRequest()
        .setName("users/<user-id>")
        .setUser(
            new User()
                .setUsername("jane.doe@example.com")
                .setFullName(new UserFullName().setFamilyName("Smith"))
                .setAccountUserStatus(State.INACTIVE))
        .setUpdateMask("full_name.family_name,account_user_status"));

CLI

databricks workspace-iam-v2 update-user-proxy users/<user-id> full_name.family_name,account_user_status --json '{
  "username": "jane.doe@example.com",
  "full_name": {"family_name": "Smith"},
  "account_user_status": "INACTIVE"
}'

Only the fields in update_mask change. The following rules also apply:

  • username is immutable, so a SCIM userName change has no IAM v2 equivalent. An update that includes username in the mask fails with INVALID_PARAMETER_VALUE.
  • The account owner and hidden users can't have their status changed.
  • You can't deactivate the user you're signed in as.
  • The SDKs require username in the request body even when the mask doesn't include it, so the examples include it.

Create a group and add a member

To create a group and add a user to it, run the following. Replace <user-id> with the user's ID from the create-a-user example, and use the group ID returned by the create call.

curl

curl -X POST "https://<workspace-url>/api/2.0/identity/groups" \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  -d '{"group_name": "data-engineers"}'

curl -X POST "https://<workspace-url>/api/2.0/identity/groups/<group-id>/direct-members" \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  -d '{"principal_id": <user-id>}'

Python

group = w.workspace_iam_v2.create_group_proxy(group=iamv2.Group(group_name="data-engineers"))

w.workspace_iam_v2.create_direct_group_member_proxy(
    parent=group.name,
    direct_group_member=iamv2.DirectGroupMember(principal_id=int(user.user_id)),
)

Go

group, err := w.WorkspaceIamV2.CreateGroupProxy(ctx, iamv2.CreateGroupProxyRequest{
	Group: iamv2.Group{GroupName: "data-engineers"},
})

_, err = w.WorkspaceIamV2.CreateDirectGroupMemberProxy(ctx, iamv2.CreateDirectGroupMemberProxyRequest{
	Parent:            group.Name,
	DirectGroupMember: iamv2.DirectGroupMember{PrincipalId: userId},
})

Java

Group group = w.workspaceIamV2().createGroupProxy(
    new CreateGroupProxyRequest().setGroup(new Group().setGroupName("data-engineers")));

w.workspaceIamV2().createDirectGroupMemberProxy(
    new CreateDirectGroupMemberProxyRequest()
        .setParent(group.getName())
        .setDirectGroupMember(new DirectGroupMember().setPrincipalId(userId)));

CLI

databricks workspace-iam-v2 create-group-proxy --group-name data-engineers
databricks workspace-iam-v2 create-direct-group-member-proxy groups/<group-id> <user-id>

To remove the member, delete groups/<group-id>/direct-members/<user-id>.

Grant entitlements in a workspace

To give a group workspace access and SQL access in this workspace, run the following:

curl

curl -X POST "https://<workspace-url>/api/2.0/identity/workspace-assignments" \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  -d '{"principal_id": <group-id>, "entitlements": ["WORKSPACE_ACCESS", "DATABRICKS_SQL_ACCESS"]}'

Python

w.workspace_iam_v2.create_workspace_assignment_proxy(
    workspace_assignment=iamv2.WorkspaceAssignment(
        principal_id=int(group.group_id),
        entitlements=[iamv2.Entitlement.WORKSPACE_ACCESS, iamv2.Entitlement.DATABRICKS_SQL_ACCESS],
    )
)

Go

_, err := w.WorkspaceIamV2.CreateWorkspaceAssignmentProxy(ctx, iamv2.CreateWorkspaceAssignmentProxyRequest{
	WorkspaceAssignment: iamv2.WorkspaceAssignment{
		PrincipalId:  groupId,
		Entitlements: []iamv2.Entitlement{iamv2.EntitlementWorkspaceAccess, iamv2.EntitlementDatabricksSqlAccess},
	},
})

Java

w.workspaceIamV2().createWorkspaceAssignmentProxy(
    new CreateWorkspaceAssignmentProxyRequest()
        .setWorkspaceAssignment(
            new WorkspaceAssignment()
                .setPrincipalId(groupId)
                .setEntitlements(
                    List.of(Entitlement.WORKSPACE_ACCESS, Entitlement.DATABRICKS_SQL_ACCESS))));

CLI

databricks workspace-iam-v2 create-workspace-assignment-proxy --json '{
  "principal_id": <group-id>,
  "entitlements": ["WORKSPACE_ACCESS", "DATABRICKS_SQL_ACCESS"]
}'

An assignment needs at least one entitlement. Use these calls to manage an existing assignment:

  • To change its entitlements, update workspace-assignments/<principal-id> with update_mask=entitlements and the full list of entitlements to keep.
  • To remove the principal from the workspace, delete the assignment.
  • To read every entitlement the principal holds, including through groups, get the assignment and read effective_entitlements.

Deactivate a principal in one workspace

To deactivate a user or service principal in one workspace without changing its account status, update principals/<principal-id>/workspace-identity-detail with update_mask=workspace_identity_status and set workspace_identity_status to INACTIVE. You can't activate or deactivate the user you're signed in as, and group principals aren't supported. See the Workspace Identity Detail API.

Identities from Microsoft Entra ID

When automatic identity management is enabled, users, groups, and service principals from Microsoft Entra ID sync to Azure Databricks when they're first used. You can't create them with external_id set. Instead, get the external principal by its ID in Microsoft Entra ID. The call syncs the principal's details and group memberships and returns its Azure Databricks ID in internal_id:

Principal Workspace-level call
User GET /api/2.0/identity/external-users/<external-id>
Group GET /api/2.0/identity/external-groups/<external-id>
Service principal GET /api/2.0/identity/external-service-principals/<external-id>

Service principals sync only from Microsoft Entra ID.

To filter on the external ID of a principal that already exists, use external_id eq "<external-id>". See Automatic identity management.

For a principal that syncs from Microsoft Entra ID, you can update only external_id. Its other fields come from Microsoft Entra ID. You can't add or remove direct members of a group that syncs from Microsoft Entra ID.

What IAM v2 doesn't cover

Keep using the SCIM APIs for the following:

Additional resources

Resource Description
Migrate Terraform identity resources to IAM v2 Move Terraform configurations from the SCIM-based identity resources to the IAM v2 resources.
Manage Azure Databricks users Add, update, and remove users in Azure Databricks as an account or workspace admin.
  • Last updated on