AD Authentication to Databricks REST API Returns 403 Unauthorized

M.Lo 1 Reputation point
2020-06-29T16:03:39.38+00:00

I'm attempting to call the Databricks API using the method described in Get an Azure Active Directory token using a service principal.

However when I use the management endpoint access token to access the Databricks REST API, I get HTTP ERROR 403. User Not Authorized.

The first 2 calls successfully return w/the appropriate access tokens. These are passed into the Databricks REST API per the documentation, however it returns 403. Help please.

# Attempting this   
#   * https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token  
  
tenantId="secret"  
subscriptionId="secret"  
resourceGroupId="my-test-group"  
clientId="secret"  
clientSecret="verysecret"  
dbResourceId="2ff814a6-3304-4ab8-85cb-cd0e6f879c1d"  
dbWorkspaceName="my-test-db"  
  
az login -u $1 -o table  
  
  
dbAccessToken=$(curl -X GET \  
-H 'Content-Type: application/x-www-form-urlencoded' -d  \  
'grant_type=client_credentials&client_id='${clientId}'&resource='${dbResourceId}'&client_secret='${clientSecret} \  
https://login.microsoftonline.com/${tenantId}/oauth2/token | jq -r '.access_token')  
  
echo Access Token  
echo $dbAccessToken  
mgtAccessToken=$(curl -X GET \  
-H 'Content-Type: application/x-www-form-urlencoded' -d \  
'grant_type=client_credentials&client_id='${clientId}'&resource=https://management.core.windows.net/&client_secret='${clientSecret} \  
https://login.microsoftonline.com/${tenantId}/oauth2/token | jq -r '.access_token')  
  
  
echo   
echo Management Access Token  
echo $mgtAccessToken  
echo  
result=$(curl -X GET \  
-H 'Authorization: Bearer '${dbAccessToken} \  
-H 'X-Databricks-Azure-SP-Management-Token: '${mgtAccessToken} \  
-H 'X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/'${subscriptionId}'/resourceGroups/'${resourceGroupId}'/providers/Microsoft.Databricks/workspaces/'${dbWorkspaceName} \  
https://"secretworkspaceid".azuredatabricks.net/api/2.0/clusters/list)  
  
echo  
echo $result  
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,059 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
20,446 questions
{count} votes

2 answers

Sort by: Most helpful
  1. Paweł Jankowski 1 Reputation point
    2020-10-02T07:54:00.36+00:00

    @M.Lo check whether you have a Contributor or Owner role assigned to your service principal

    az role assignment create --assignee $(clientId) --role "Contributor" --scope subscriptions/$(subscriptionId)/resourceGroups/$(resourceGroupName)/$(resourceName)  
    

  2. Paweł Jankowski 1 Reputation point
    2020-10-02T08:14:04.993+00:00

    I have similar issue but when using token generated with https://....azuredatabricks.net/api/2.0/token/create. My service principal have a Contributor role assigned, then I'm creating a token and saving this token to key vault. I'd like to consume this token in Azure Data Factory but then I'm getting 403 when testing connection via Databricks linked service. On the other hand when I create personal access token via Databricks portal and provide it to ADF then it works.

    The main issue is that we have this generation of access token automated via pipeline and manual way of getting it isn't an option for us.

    Any ideas?

    0 comments No comments