I am constantly getting the below error:"Secret does not exist with scope: kvmigrn and key: clientid". Tried creating an app two times from scratch. I already have a vault like this and a key . Attaching screenshot adb error.jpg vault.jpg

Mounting error in Gen 1 with adls: Secret scope does not exist

Accepted answer

Amira Bedhiafi 24,556

The error is clear there is a missing secret in the specified Databricks secret scope.

Ensure that the secret scope kvmigrn has been correctly created in Databricks. If you are using Azure Key Vault-backed secret scopes, you need to ensure that the Databricks workspace has permission to list and read secrets from the Key Vault.

Double-check that the secrets with the keys clientid, clientsecret, and tenantid exist in the kvmigrn scope. Typos and casing can sometimes be a cause of these types of issues.

Check this link to have an idea : https://community.databricks.com/t5/data-engineering/error-while-mounting-to-adls-gen2-error-secret-does-not-exist/td-p/28915

Also try this and tell us the result :

Navigate to your Azure Key Vault in the Azure Portal. Check to see if the secret with the key clientid is indeed present in the vault.

Each secret in AKV can have multiple versions. Ensure you're referencing the right version, or you're set to retrieve the latest version.

Ensure that the secret hasn’t expired or hasn’t been disabled in the Key Vault.

KEERTHANA JAYADEVAN 66

Hi @Amira Bedhiafi , thanks for that. It got resolved after i deleted everything and started from beginning, might have been some typo . But I am now stuck with another issue, could you please help me with that:

I am trying to mount my Gen 2 storage account in databricks. I have added permission of Storage blob data contributor to the storage account. But  I am getting below error:
Invalid permissions on the specified KeyVault https://kvmigrationnew.vault.azure.net/. Wrapped Message: Status code 403, {"error":{"code":"Forbidden","message":"Caller is not authorized to perform action on resource.\r\nIf role assignments, deny assignments or role definitions were changed recently, please observe propagation time.\r\nCaller: name=AzureDatabricks;appid=123abcdbhjdllajddaddd;iss=https://sts.windows.net/984433r41f5ssadfgfsf/\r\nAction: 'Microsoft.KeyVault/vaults/secrets/getSecret/action'\r\nResource: '/subscriptions/xxxx-xxx-xxx/resourcegroups/rsggen2demo/providers/microsoft.keyvault/vaults/kvmigrationnew/secrets/clientid'\r\nAssignment: (not found)\r\nDecisionReason: 'DeniedWithNoValidRBAC' \r\nVault: kvmigrationnew;location=eastus2\r\n","innererror":{"code":"ForbiddenByRbac"}}}
------------------------------------
Configuration
---------------
Resource group: rsggen2demo
Storage acc name: stggen2demo
Keyvault: kvmigrationnew
DB Scope: gen2mig
 
Code: 
adlsAccountName = "stggen2demo"
adlsContainerName = "output"
adlsFolderName = "schemas_new"
mountPoint = "/mnt/schemas_new"
 
# Application (Client) ID
applicationId = dbutils.secrets.get(scope="gen2mig",key="ClientID")
 
# Application (Client) Secret Key
authenticationKey = dbutils.secrets.get(scope="gen2mig",key="ClientSecret")
 
# Directory (Tenant) ID
tenandId = dbutils.secrets.get(scope="gen2mig",key="TenantID")
 
endpoint = "https://login.microsoftonline.com/" + tenandId + "/oauth2/token"
source = "abfss://" + adlsContainerName + "@" + adlsAccountName + ".dfs.core.windows.net/" + adlsFolderName
 
# Connecting using Service Principal secrets and OAuth
configs = {"fs.azure.account.auth.type": "OAuth",
           "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
           "fs.azure.account.oauth2.client.id": applicationId,
           "fs.azure.account.oauth2.client.secret": authenticationKey,
           "fs.azure.account.oauth2.client.endpoint": endpoint}
 
# Mount ADLS Storage to DBFS only if the directory is not already mounted
if not any(mount.mountPoint == mountPoint for mount in dbutils.fs.mounts()):
  dbutils.fs.mount(
    source = source,
    mount_point = mountPoint,
    extra_configs = configs)
 
 
 
 
 

keyv.jpg

Amira Bedhiafi 24,556 Reputation points

2023-09-18T10:44:53.28+00:00

You need to post a new question and paste here the link for it.

Share via

Mounting error in Gen 1 with adls: Secret scope does not exist

0 additional answers

Your answer