Use Microsoft Spark Utilities:
secret= mssparkutils.credentials.getSecret(KEYVAULT_NAME, SECRET_NAME)
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
I am getting a 401 error while using the below code
Error: NoAuthenticationInformation, "Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
%%pyspark
spark.conf.set("fs.azure.account.auth.type", "SAS")
spark.conf.set("fs.azure.sas.token.provider.type", "com.microsoft.azure.synapse.tokenlibrary.AkvBasedSASProvider")
spark.conf.set("spark.storage.synapse.akv", "<AZURE KEY VAULT NAME>")
spark.conf.set("spark.storage.akv.secret", "<SECRET KEY>")
df = spark.read.csv('abfss://<CONTAINER>@<ACCOUNT>.dfs.core.windows.net/<FILE PATH>')
display(df.limit(10))
Use Microsoft Spark Utilities:
secret= mssparkutils.credentials.getSecret(KEYVAULT_NAME, SECRET_NAME)
Hello @Arnfried Rixen , @Anisha Rayapati , @Sivagnana Sundaram, Krithiga ,
Thanks for the question and using MS Q&A platform.
This is a known issue with product. Our product team currently working on the solution. I will update this thread once it’s available.
Meanwhile workaround is to create a linked service against the Azure key vault.
Use get secret with linked service using the below documentation:
https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-secure-credentials-with-tokenlibrary?pivots=programming-language-csharp#getsecret
Hope this will help. Please let us know if any further queries.
------------------------------