notebook issue with managed identity

John 111 Reputation points
2023-05-16T21:38:29.7533333+00:00

I am using managed identity to run the code mentioned in the document, but I a getting this error. Any one experienced the issue?

Py4JJavaError: An error occurred while calling o3892.csv. : HTTP Error -1CustomTokenProvider getAccessToken threw java.io.IOException : POST failed with 'Bad Request' (400) and message: {"result":"DependencyError","errorId":"BadRequest","errorMessage":"[Code=, Target=, Message=].

%%pyspark

Python code

source_full_storage_account_name = "teststorage.dfs.core.windows.net"

spark.conf.set(f"spark.storage.synapse.{source_full_storage_account_name}.linkedServiceName", "<LINKED SERVICE NAME>")

spark.conf.set(f"fs.azure.account.oauth.provider.type.{source_full_storage_account_name}", "com.microsoft.azure.synapse.tokenlibrary.LinkedServiceBasedTokenProvider")

df = spark.read.csv('abfss://<CONTAINER>@<ACCOUNT>.dfs.core.windows.net/<DIRECTORY PATH>')

df.show()

https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-secure-credentials-with-tokenlibrary?pivots=programming-language-python#adls-gen2-storage-with-linked-services

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,569 questions
{count} votes

Accepted answer
  1. BhargavaGunnam-MSFT 28,271 Reputation points Microsoft Employee
    2023-05-17T00:26:15.7933333+00:00

    Hello John,

    Welcome to the MS Q&A platform.

    The error is due to a minor bug on the synapse UI.

    You must publish the linked service before using it in the notebook.

    I successfully ran the code with the managed identity after publishing the linked service.

    Please see the screenshots for your reference.

    I hope this helps. Please let me know if you still have any issues after publishing the linked service.

    User's image

    User's image

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful