notebook issue with managed identity

Question

I am using managed identity to run the code mentioned in the document, but I a getting this error. Any one experienced the issue?

Py4JJavaError: An error occurred while calling o3892.csv. : HTTP Error -1CustomTokenProvider getAccessToken threw java.io.IOException : POST failed with 'Bad Request' (400) and message: {"result":"DependencyError","errorId":"BadRequest","errorMessage":"[Code=, Target=, Message=].

%%pyspark

Python code

source_full_storage_account_name = "teststorage.dfs.core.windows.net"

spark.conf.set(f"spark.storage.synapse.{source_full_storage_account_name}.linkedServiceName", "<LINKED SERVICE NAME>")

spark.conf.set(f"fs.azure.account.oauth.provider.type.{source_full_storage_account_name}", "com.microsoft.azure.synapse.tokenlibrary.LinkedServiceBasedTokenProvider")

df = spark.read.csv('abfss://<CONTAINER>@<ACCOUNT>.dfs.core.windows.net/<DIRECTORY PATH>')

df.show()

https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-secure-credentials-with-tokenlibrary?pivots=programming-language-python#adls-gen2-storage-with-linked-services

Answer 1

Hello John,

Welcome to the MS Q&A platform.

The error is due to a minor bug on the synapse UI.

You must publish the linked service before using it in the notebook.

I successfully ran the code with the managed identity after publishing the linked service.

Please see the screenshots for your reference.

I hope this helps. Please let me know if you still have any issues after publishing the linked service.