Getting unauthorized response with DataLake Gen 2 link

Question

Getting unauthorized response with DataLake Gen 2 link

Jeff 1

I'm having trouble with some simple data queries. I'm using an account key to authorize connections to Data Lake storage Gen 2. First I set the account key (I know not best practice storing key in notebook but I'm just trying to get something running):

spark.conf.set(
  "fs.azure.account.key.mystorage.dfs.core.windows.net",
  "031...=")

But the following fails with an error:

df = spark.read.text("abfs://demo@mystorage.dfs.core.windows.net/sample_data.csv")

Py4JJavaError: An error occurred while calling o324.text.
: Operation failed: "This request is not authorized to perform this operation.", 403, HEAD, https:/mystorage.dfs.core.windows.net/demo/?upn=false&action=getAccessControl&timeout=90
    at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.execute(AbfsRestOperation.java:241)
    at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsClient.getAclStatus(AbfsClient.java:767)
    at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsClient.getAclStatus(AbfsClient.java:749)

I have been able to access this data from a Jupyter Notebook running on a VM using a SAS token. Any ideas?

Answer 1

ShaikMaheer-MSFT 30,041 Microsoft Employee

Hi @Jeff ,

Thank you for posting query in Microsoft Q&A Platform.

You can access ADLS Gen2 using OAuth 2.0 Or using SAS. When it comes to Blob storage then you can do that using Account key.

So, If in your case your storage is ADLS gen2 then kindly consider using OAuth 2.0 or SAS mechanism.

Click here to know about complete detailed steps to Access ADLS Gen2 using OAuth 2.0
Click here to know about complete detailed steps to Access ADLS Gen2 using SAS
Click here to know about accessing Azure Blob storage.

Hope this helps. Please let us know if any further queries.

--------------

Please consider hitting Accept Answer. Accepted answers helps community as well.

Jeff 1 Reputation point

2022-02-04T15:49:01.17+00:00

OK, I think I figured some of this out. I had to change network/firewall rules to allow access from "All Networks" and not just the 10.x.x.x subnet I had setup. However, doesn't this become a security issue? Is there some way to restrict the access to the storage account to just the Spark cluster?

Also, I had to add the service principal to the "Storage Blob Data Owner" role.
ShaikMaheer-MSFT 30,041 Reputation points Microsoft Employee

2022-02-11T16:09:57.263+00:00

Thanks for sharing your resolution. How about using Managed end points? Managed private endpoints are private endpoints created in a Managed Virtual Network associated with your Azure Synapse workspace. Managed private endpoints establish a private link to Azure resources. Click here to know more about it.

yes we should have "Storage Blob Data Owner".
ShaikMaheer-MSFT 30,041 Reputation points Microsoft Employee

2022-02-15T16:50:15.133+00:00

Hi @Jeff - Kindly consider marking answer as accepted answer which has workaround discussed. Accepted answers helps community as well.
ShaikMaheer-MSFT 30,041 Reputation points Microsoft Employee

2022-03-04T06:27:16.82+00:00

Hi @Jeff - Just checking if you get chance to mark it as Accepted Answer. Accepted answers helps community as well.

Answer 2

For some reason I couldn't comment on your post. I created a new app registration, generated a client secret, then gave the app "Contributor" role on the storage account. I then tried using OAuth2:

configs = {"fs.azure.account.auth.type": "OAuth",
       "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
       "fs.azure.account.oauth2.client.id": "xxx...",
       "fs.azure.account.oauth2.client.secret": "xxx...",
       "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/xxx.../oauth2/token",
       "fs.azure.createRemoteFileSystemDuringInitialization": "true"}

dbutils.fs.mount(
source = "abfss://demo@mystorage.dfs.core.windows.net/",
mount_point = "/mnt/demo",
extra_configs = configs)

This still throws an exeception:

ExecutionError: An error occurred while calling o298.mount.
: Operation failed: "This request is not authorized to perform this operation.", 403, PUT, https://mystorage.dfs.core.windows.net/demo?resource=filesystem, AuthorizationFailure, "This request is not authorized to perform this operation. RequestId:4838ca85-d01f-0025-55d2-194358000000 Time:2022-02-04T14:20:09.0319452Z"
    at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.execute(AbfsRestOperation.java:241)
    at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsClient.createFilesystem(AbfsClient.java:186)

Considering the other method, the SAS doc page is missing information on how to do the SAS auth in Python.

SQLArcher 81 Reputation points

2022-03-06T12:03:01.923+00:00

Hi @Jeff ,

Regarding the "all networks" you will need to follow either the instruction to create private endpoints or alternatively you follow this guidance on VNET-injection for Databricks here

Regarding the access for the service principal, you need to assign Storage Blob Data Contributor/Owner instead of the contributor role. Guidance on the roles is here

SAS tokens become "messy" to be used with Databricks, although this is use-case dependant, normally Databricks should have an ADLS account associated with it where processed data will be maintained. Data access will then either be based on Azure AD pass-through authentication or delegated to Databricks ACLs. A sample of using SAS with Databricks is documented here

If this answers your question or resolves your issue, please remember to mark this post as an answer.

Getting unauthorized response with DataLake Gen 2 link

2 answers

Activity