Failure mounting Azure datalake storage to DataBricks file storage

asked 2022-12-30T19:47:19.027+00:00
JV van Heeringen 1 Reputation point

I unsuccessfully tried mounting my Azure datalake storage account to an Azure Databricks workspace. Here's my code:

configs = {"fs.azure.account.auth.type": "OAuth",
"fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
"fs.azure.account.oauth2.client.id": "<<my-client-id>>",
"fs.azure.account.oauth2.client.secret": "<<my-secret-id>>",
"fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/<<my-token-id>>/oauth2/token"}

dbutils.fs.mount(
source = "abfss://<<folder>>@<<my-datalake-name>>.dfs.core.windows.net/",
mount_point = "/mnt/bronze",
extra_configs = configs)

Here's the raised error:

java.lang.NullPointerException

---------------------------------------------------------------------------

ExecutionError Traceback (most recent call last)
File <command-3688668228540591>:9
1 # mount my datalake to the Databricks File Storage
2 configs = {"fs.azure.account.auth.type": "OAuth",
3 "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
4 "fs.azure.account.oauth2.client.id": "05594a47-908a-4419-91aa-c6e58c801891",
5 "fs.azure.account.oauth2.client.secret": "12af1888-7b70-494f-a965-71122acccd21",
6 "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/9f8a1445-2e1b-406c-a1bb-6e042da5468a/oauth2/token"}
8 #display(configs)
----> 9
10 # Optionally, you can add <directory-name> to the source URI of your mount point.
12 endpoint = "wasbs://bronze@datalakejv1.dfs.core.windows.net/";
14 dbutils.fs.mount(
15 source = endpoint,
16 mount_point = "/mnt/bronze",
17 extra_configs = configs)

File /databricks/python_shell/dbruntime/dbutils.py:362, in DBUtils.FSHandler.prettify_exception_message.<locals>.f_with_exception_handling(*args, **kwargs)
360 exc.context = None
361 exc.cause = None

I read many posts regarding similar DataBricks issues, followed up on many suggestions (such as making sure to unmount : dbutils.fs.unmount("/mnt/bronze")). However, I ran out of options.

I appreciate your help and suggestions! Thank you

I made sure that

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,199 questions
{count} votes