Manage personal access tokens
To authenticate to the Azure Databricks REST API, a user can create a personal access token and use it in their REST API request. Tokens have an optional expiration date and can be revoked. See Authentication using Azure Databricks personal access tokens.
The number of personal access tokens per user is limited to 600 per workspace.
The ability to create personal access tokens is enabled by default for all Azure Databricks workspaces that were created in 2018 or later. Workspace administrators can enable or disable the ability to create personal token access for any workspace, regardless of when the workspace was created.
When the ability to create personal access tokens is enabled for a workspace, workspace administrators can also monitor tokens, control which non-admin users can create tokens and use tokens, and set maximum lifetimes for new tokens.
Note
You can also enable Azure Databricks users to use Azure Active Directory tokens for REST API access instead of Azure Databricks personal access tokens. If your workspace uses Azure Active Directory tokens, the instructions in this article do not apply.
When the ability to generate personal access tokens is enabled for your workspace, by default all users in your Azure Databricks workspace can generate personal access tokens to access Azure Databricks REST APIs, and they can generate these tokens with any expiration date they like, including an indefinite lifetime.
As an Azure Databricks admin, you can use the Token Management API 2.0 and Permissions API 2.0 to control token usage at a more fine-grained level. The APIs are published on each workspace instance. To learn how to access and authenticate to the API, see Authentication using Azure Databricks personal access tokens. You must access the API as an Azure Databricks admin.
For some tasks, you can also use the Admin Console.
The following table indicates the tasks you can perform using the web application and those you can perform using the REST APIs. For the cells that say Yes, click the word to view the related documentation.
Task | Description | Admin Console | REST API |
---|---|---|---|
Enable/disable | Enable or disable all tokens for this workspace | Yes | Yes |
Control who can use tokens | Limit personal access token creation and usage to specified users and groups in this workspace. If you revoke a user’s permission to create and use tokens, that user’s existing tokens are also revoked. | Yes | Yes |
Max lifetime for new tokens. | Set the maximum lifetime of new tokens in this workspace | No | Yes |
Manage existing tokens | For existing tokens, get the token creator, expiration date, and user-provided token description. Revoke tokens for users who should no longer have access to Azure Databricks APIs. By monitoring and controlling token creation, you reduce the risk of lost tokens or long-lasting tokens that could lead to data exfiltration from the workspace. | No | Yes |
See the following diagram for the typical flow of token management using the REST API for administration and token usage. See the table in this topic for the set of tasks that you could alternatively use the Admin Console.
Requirements
This feature requires the Premium Plan.
Enable or disable token-based authentication for the workspace
Token-based authentication is enabled by default for all Azure Databricks workspaces that were created in 2018 or later. You can change this setting in the Admin Console. To specify which users are allowed to use tokens, see Control who can use or create tokens.
Warning
Partner Connect and partner integrations require personal access tokens to be enabled on a workspace.
To disable the ability to create personal access tokens for the workspace:
- Go to the Admin Console.
- Click the Workspace Settings tab.
- Click the Personal Access Tokens toggle.
- Click Confirm. This change may take a few seconds to take effect.
To use token-based authentication for a REST API request, see Authentication using Azure Databricks personal access tokens.
No tokens are deleted when you disable token-based authentication for a workspace. If tokens are re-enabled later, any non-expired tokens are immediately available for use.
If you want to disable token access for a subset of users, you can keep token-based authentication enabled for the workspace and then set fine-grained permissions for users and groups. See Control who can use or create tokens.
You can also use the REST API to make this change. To enable or disable the token management feature for a workspace, call the workspace configuration for tokens API (PATCH /workspace-conf
). In a JSON request body, specify enableTokensConfig
as true
(enabled) or false
(disabled). To render the OpenAPI in HTML or import into other tools, see Token Management API 2.0.
For example, to enable the feature:
curl -X PATCH -n \
https://<databricks-instance>/api/2.0/workspace-conf \
-d '{
"enableTokensConfig": "true",
}'
Control who can use or create tokens
A user can have one of the following token permissions:
No permissions
Can Use – Whether a user can use personal access tokens. For workspaces created after the release of Azure Databricks platform version 3.28 (Sept 9-15, 2020) by default no non-admin user or group is granted the Can Use permission. Admins must explicitly grant the Can Use permission to a user, a group, or to all users by granting it to the
users
group.Important
Workspaces created before 3.28 was released maintain the permissions that were already in place. The previous default was for all users to have the Can Use permission. Admins can revoke that group permission and grant it to other groups or to individual non-admin users. See Remove permissions.
Can Manage – Whether a user or group can manage personal access tokens and permissions for them. Users in the
admins
group have this permission by default and you cannot revoke it. No other groups can be granted this permission. The API enforces these rules.
This table lists the permissions required for each token-related task:
Task | No permission | Can Use | Can Manage |
---|---|---|---|
Create a token | – | Yes | Yes |
Use a token for authentication | – | Yes | Yes |
Revoke your own token | – | Yes | Yes |
Revoke any user’s token ** | – | – | Yes |
List all tokens ** | – | – | Yes |
Modify token permissions *** | – | – | Yes |
Actions that are marked with ** require the Token Management API 2.0.
Actions that are marked with *** can be performed in the Admin Console or with the Permissions API 2.0. See Control who can use or create tokens.
Manage token permissions using the admin console
To manage token permissions for the workspace using the admin console:
Go to the admin console.
Click the Workspace Settings tab.
Click the Permissions button next to Personal Access Tokens to open the token permissions editor.
Add, remove, or update permissions. For information about the meaning of a permission and the default permissions for a workspace, see Control who can use or create tokens.
If the
users
group has the Can Use permission and you want to apply more fine-grained access for non-admin users, remove the Can Use permissions from the users group by clicking the X next to the permission drop-down in the users row.To grant the permission to other entities, select each user or group to whom you want to grant access. Select a user or group from the Select User or Group… drop-down, select Can Use, and click the + Add button. In the following example, the admin has removed access for the users group and is granting access to the Data Science B2 group.
Click Save to save your changes.
Warning
After saving your changes, any users who previously had either Can Use or Can Manage permission but no longer have either permission are denied access to token-based authentication and their active tokens are immediately deleted (revoked). Deleted tokens cannot be retrieved.
Manage token permissions using the Permissions API
Get all token permissions for the workspace
To get token permissions for all Azure Databricks users, Azure Databricks groups, and Azure service principals for the workspace, call the get all token permissions for the workspace API (GET /permissions/authorization/tokens
). To render the OpenAPI in HTML or import into other tools, see Token Management API 2.0.
The response includes an access_control_list
array. Each element is a user object, a group object, or a service principal object. They each have an identity field appropriate to the type: users have a user_name
field, groups have a group_name
field, and service principals have a service_principal_name
field. All elements have an all_permissions
field that specifies which permission levels (CAN_USE
or CAN_MANAGE
) are granted.
For example:
curl -n -X GET "https://<databricks-instance>/api/2.0/preview/permissions/authorization/tokens"
Example response:
{
"object_id": "authorization/tokens",
"object_type": "tokens",
"access_control_list": [
{
"user_name": "jsmith@example.com",
"all_permissions": [
{
"permission_level": "CAN_USE",
"inherited": false
}
]
}
]
}
Set token permissions
You can set permissions on one or more users, groups, or service principals. For each user, you need to know the email address, which is specified in the user_name
request property. For each group, specify the group name in the group_name
property. For a service principal, specify the service principal applicationId
value in the service_principal_name
property.
To set token permissions, call the set token permissions API (PATCH /permissions/authorization/tokens
). To render the OpenAPI in HTML or import into other tools, see Token Management API 2.0.
You can set permissions on one or more user, groups, or Azure service principals. For each user, you need to know the email address, which is specified in the user_name
request property. For each group, specify the group name in the group_name
property. For an Azure service principal, specify the service principal applicationId
value in the service_principal_name
property.
Entities (such as users or groups) not mentioned explicitly are not directly affected by this request, although changes to group membership can indirectly affect user access.
You can only grant, not revoke, permissions with this API.
For example, the following example grants access to user jsmith@example.com and the group mygroup.
curl -n -X PATCH "https://<databricks-instance>/api/2.0/preview/permissions/authorization/tokens"
-d '{
"access_control_list": [
{
"user_name": "jsmith@example.com",
"permission_level": "CAN_USE",
},
{
"group_name": "mygroup",
"permission_level": "CAN_USE",
}
]
}'
Example response:
{
"access_control_list": [
{
"user_name": "jsmith@example.com",
"all_permissions": [
{
"permission_level": "CAN_USE",
"inherited": false
}
]
},
{
"group_name": "mygroup",
"all_permissions": [
{
"permission_level": "CAN_USE",
"inherited": false
}
]
}
]
}
If you want to set token permissions for all entities in the workspace in one request, use the Replace all token permissions for workspace API (PUT /permissions/authorization/tokens
). See Remove permissions. To render the OpenAPI in HTML or import into other tools, see Token Management API 2.0.
Remove permissions
Note
For workspaces created after the release of Azure Databricks platform version 3.28 (Sept 9-15, 2020) the default is for no users to have the Can Use permission. Admins must explicitly grant those permissions, whether to the entire users
group or on a user-by-user or group-by-group basis. Workspaces created before 3.28 was released maintain the permissions that were already in place. The default was for all users to have the Can Use permission. Admins can revoke that group permission assignment and grant it to other groups or to individual non-admin users.
To remove permissions from all or some non-admin users, use the update all permissions API (PUT /permissions/authorization/tokens
), which requires that you specify the complete set of permissions for all objects that are granted permissions for the entire workspace. To render the OpenAPI in HTML or import into other tools, see Token Management API 2.0.
If you want to authorize a subset of non-admin users to create and use tokens, do all three of the following:
- Grant the users, groups, and Azure service principals the
CAN_USE
permission. - Do not grant the
CAN_USE
permission to the built-inusers
group if you want to only authorize some non-admin users. You can optionally choose to assign the permission to this group, in which case all non-admin users can create and use tokens. - Grant the built-in
admins
group theCAN_MANAGE
permission. This is a requirement of the API.
Warning
After a successful request, if a user or Azure service principal does not have token permissions directly or indirectly through a group, their tokens are immediately deleted. Deleted tokens cannot be retrieved.
The following example grants Can Use tokens permission to group field-automation-group
, omits permissions for the users
(all users) group, and grants CAN_MANAGE
permission to the admins
group as required by the API. Any non-admin users that are not in the group field-support-engineers
will lose access to token creation and their existing tokens are immediately deleted (revoked).
curl -n -X PUT "https://<databricks-instance>/api/2.0/preview/permissions/authorization/tokens"
-d '{
"access_control_list": [
{
"group_name": "field-automation-group",
"permission_level": "CAN_USE",
},
{
"group_name": "admins",
"permission_level": "CAN_MANAGE",
},
]
}'
Set maximum lifetime of new tokens (REST API only)
Use the token lifetime management APIs to manage the maximum lifetime of new tokens in this workspace.
To set the maximum lifetime for new tokens, call the Set the maximum token lifetime for new tokens API (PATCH /workspace-conf
). Set maxTokenLifetimeDays
to the maximum token lifetime of new tokens in days, as an integer. If you set it to zero, new tokens are permitted to have no lifetime limit. To render the OpenAPI in HTML or import into other tools, see Token Management API 2.0.
For example:
curl -n -X PATCH "https://<databricks-instance>/api/2.0/workspace-conf"
-d '{
"maxTokenLifetimeDays": "90"
}'
Warning
This limit applies only to new tokens. To review existing tokens, see the get tokens API. To render the OpenAPI in HTML or import into other tools, see Token Management API 2.0.
To get the workspace’s maximum lifetime for new tokens, call the get the maximum token lifetime for new tokens API (GET /workspace-conf
) and pass keys=maxTokenLifetimeDays
as a query parameter. The response includes an maxTokenLifetimeDays
property that is the maximum token lifetime of new tokens in days, as an integer. If it is zero, new tokens are permitted to have no lifetime limit.
For example:
curl -n -X GET "https://<databricks-instance>/api/2.0/workspace-conf?keys=maxTokenLifetimeDays"
Example response:
{
"maxTokenLifetimeDays": "90"
}
Monitor and revoke tokens (REST API only)
Use the token management APIs to manage existing tokens in the workspace.
Get tokens for the workspace
To get the workspace’s tokens, call the get all tokens API (GET /token-management/tokens
). The response includes a token_infos
array. Each element represents a token and includes fields for ID (token_id
), creation time (creation_time
), expiry time (expiry_time
), description (comment
), and the user that created it (the ID is created_by_id
and the username is created_by_username
). To render the OpenAPI in HTML or import into other tools, see Token Management API 2.0.
You can learn more about the user using the SCIM get user API (GET /scim/v2/Users/{id}
).
To filter results by a user, set the request body property created_by_id
(for the ID) or created_by_username
(for the username). You can get a user ID from a display name using the SCIM get users API (GET /scim/v2/Users
)
For example:
curl -n -X GET "https://<databricks-instance>/api/2.0/token-management/tokens"
-d '{
"created_by_id": "1234567890"
}'
Example response:
{
"token_infos": [
{
"token_id": "<token-id>",
"creation_time": 1580265020299,
"expiry_time": 1580265020299,
"comment": "This is for ABC division's automation scripts.",
"created_by_id": 1234567890,
"created_by_username": "jsmith@example.com"
}
]
}
Alternatively, get a specific token using the get a token API (GET /token-management/tokens/{token_id}
). To render the OpenAPI in HTML or import into other tools, see Token Management API 2.0.
Delete (revoke) a token
- Find the token ID. See Get tokens for the workspace.
- Call the delete a token API (
DELETE /token-management/tokens
). Pass the token ID in the path.
For example:
curl -n -X DELETE "https://<databricks-instance>/api/2.0/token-management/tokens/<token-id>"
Terraform integration
You can manage permissions in a fully automated setup using Databricks Terraform provider and databricks_permissions as follows.
Warning
The following configuration contains the statement authorization = "tokens"
. There can be only one authorization = "tokens"
permissions resource per Azure Databricks workspace, otherwise there will be a permanent configuration drift. After applying the following changes, users who previously had either CAN_USE
or CAN_MANAGE
permission but no longer have either permission have their access to token-based authentication revoked. Their active tokens are immediately deleted (revoked).
resource "databricks_group" "auto" {
display_name = "Automation"
}
resource "databricks_group" "eng" {
display_name = "Engineering"
}
resource "databricks_permissions" "token_usage" {
authorization = "tokens"
access_control {
group_name = databricks_group.auto.display_name
permission_level = "CAN_USE"
}
access_control {
group_name = databricks_group.eng.display_name
permission_level = "CAN_USE"
}
}
Feedback
Submit and view feedback for