Error when mounting Azure Data Lake Storage Gen1 account

Question

I'm using the code below to mount gen1 account on azure databricks

configs = {"fs.adl.oauth2.access.token.provider.type": "ClientCredential",
"fs.adl.oauth2.client.id": <service-principal-appid>,
"fs.adl.oauth2.credential":<storage-account-access-key>,
"fs.adl.oauth2.refresh.url": "https://login.microsoftonline.com/<tenant-id>/oauth2/token"}
dbutils.fs.mount(
source = "adl://<storageaccountname>.azuredatalakestore.net/",
mount_point = "/mnt/<container>",
extra_configs = configs)
It mounts successfully, however, when I try to list the files using dbutils.fs.ls(/mnt/<container>), it throws the error below. The Service principal I'm using has owner role over the storage account.

An error occurred while calling z:com.databricks.backend.daemon.dbutils.FSUtils.ls.
: com.microsoft.azure.datalake.store.ADLException: Error enumerating directory /
Error fetching access token
Operation null failed with exception java.io.IOException : Server returned HTTP response code: 401 for URL:

Answer 1

@Anonymous If the hierarchical namespace is disabled then its not a DataLake, its a Blob Storage V2.
Could you try using blob storage mount:

dbutils.fs.mount(
source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net",
mount_point = "/mnt/<mount-name>",
extra_configs = {"fs.azure.account.key.{0}.blob.core.windows.net".format(storage_account_name): storage_account_key}
)

Answer 2

@Anonymous Thanks for using Microsoft Q&A !!

I believe you are not passing correct credentials while mounting ADLS account. You need to pass your service principal secret (client secret) to fs.adl.oauth2.credential and not <storage-account-access-key>. Can you please try again with the client secret and let me know if any issues.
You can get a client secret by going to Azure Portal > Azure Active Directory> App Registrations and select your registered application(service principal)

----------

Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.

Answer 3

Hi @Saurabh Sharma Thank you for your response and apologies for the delay. I passed the client secret to fs.adl.oauth2.credential and I'm still receiving the same error. The service principal has Contributor role assigned to the storage account.

Answer 4

Hi @Saurabh Sharma , I see that Allow access from all networks is checked. One quick question - under properties, I don't see the blob URL starting with adl:// . This is the blob service endpoint - https://xxxxxx.blob.core.windows.net/. And the account type is StorageV2 (general purpose v2). However, hierarchical namespace is disabled so I assumed it's gen1. Am I missing something here?
Appreciate your help!

Answer 5

@Saurabh Sharma , @Pranay I'm able to mount the blob storage successfully now :) sorry for the trouble and appreciate your help.