Synapse Notebook FileNotFoundException when linked service connect to Storage Account using Managed Identity

Question

Problem Description

I set up an Linked Service to Azure Blob Storage using Managed Identity (as below). The Synapse Workspace's System-Assigned Managed Identity has been assigned the "Storage Blob Data Reader" role in the target storage account (as below).

In my Synapse Notebook, my code is as following.

blob_sas_token = mssparkutils.credentials.getConnectionStringOrCreds(blob_linked_service_name)  
  
blob_path = 'wasbs://%s@%s.blob.core.windows.net/%s' % (blob_container_name, blob_account_name, blob_relative_path)  
  
spark.conf.set('fs.azure.sas.%s.%s.blob.core.windows.net' % (blob_container_name, blob_account_name), blob_sas_token)  
  
files = mssparkutils.fs.ls(blob_path)  

I got a FileNotFoundException when I run the code

Py4JJavaError: An error occurred while calling z:mssparkutils.fs.ls.  
: java.io.FileNotFoundException: wasbs://[my container name]@[my storage account name].blob.core.windows.net/[my path] is not found  
	at org.apache.hadoop.fs.azure.NativeAzureFileSystem.listStatus(NativeAzureFileSystem.java:2845)  
	at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.ls(MSFsUtilsImpl.scala:259)  
	at mssparkutils.fs$.ls(fs.scala:23)  
	at mssparkutils.fs.ls(fs.scala)  
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
	at java.lang.reflect.Method.invoke(Method.java:498)  
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)  
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)  
	at py4j.Gateway.invoke(Gateway.java:282)  
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)  
	at py4j.commands.CallCommand.execute(CallCommand.java:79)  
	at py4j.GatewayConnection.run(GatewayConnection.java:238)  
	at java.lang.Thread.run(Thread.java:748)  

However, the exactly same code works fine when the Linked Service is configure using Account Key. (so I am sure there is no typo and the container/folder/file exist).

Can anyone help us understand what maybe the problem and how to fix it?

Answer 1

Hi @Jerry Pan ,

Thanks for the question and using MS Q&A platform.

When the linked service is configured with account key, it returns a SAS token as connection string that can be passed in through fs.azure.sas.blob.core.windows.net.

As per the repro, when I used the same above code returns error message as shown above:

Note: In order to use a linked service configured with MSI for blob storage, the code would need to be changed quite a bit. Along the lines of:

spark.conf.set("spark.storage.synapse.linkedServiceName", blob_linked_service_name)  
spark.conf.set("fs.azure.account.oauth.provider.type", "com.microsoft.azure.synapse.tokenlibrary.LinkedServiceBasedTokenProvider")  
blob_path = 'abfss://%s@%s.dfs.core.windows.net/%s' % (blob_container_name, blob_account_name, blob_relative_path)  
files = mssparkutils.fs.ls(blob_path)  

When I changed the code to use the fs.azure.account.oauth.provider.type it works as excepted.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

Hi @Jerry Pan ,

Thank you for using the Microsoft Q&A forum.

Based on the information you have provided above, you will need to use the storage account key in order to sign the SAS token. If you print out the request provided above then you will see that the MSI generates a JWT eyJ0eXAiOiJKV1QiL.....

This how-to guide here also suggests using an account key for accessing blob storage.
My understanding is that it comes down to how the OAuth request is processed - and normally, being a two-step process, you are getting the token but not making the SAS-generate request to the storage account.

To find out more about this you can take a look at this article for access management in storage accounts.

If this answers your question, please consider marking this as an answer.