Is there a way to connect ADLS gen2 container with sas token in Synapse notebook?

Question

Is there a way to connect ADLS gen2 container with sas token in Synapse notebook?

Jize Xu 10 Microsoft Employee

I have tried as this: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-secure-credentials-with-tokenlibrary?pivots=programming-language-python#adls-gen2-storage-without-linked-services But the synapse notebook said fs.azure.sas.token.provider.type is invalid. User's image

How can I solve it? By the way, I cannot link it with linked service, because I want to generate many different notebooks to connect different ADLS gen2.

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Jize Xu 10 Reputation points Microsoft Employee

2024-01-24T03:05:57.9833333+00:00
Finally, I solved it with these codes. Notice that we should use "fs.azure.sas.token.provider.type.<ACCOUNT>.dfs.core.windows.net" as the provider key. The document I mentioned above misses the part ".<ACCOUNT>.dfs.core.windows.net"

%%pyspark spark.conf.set("fs.azure.account.auth.type.<ACCOUNT>.dfs.core.windows.net", "SAS") spark.conf.set("fs.azure.sas.token.provider.type.<ACCOUNT>.dfs.core.windows.net", "com.microsoft.azure.synapse.tokenlibrary.ConfBasedSASProvider") spark.conf.set("spark.storage.synapse.<CONTAINER>.<ACCOUNT>.dfs.core.windows.net.sas", "<SAS KEY>")
Harishga 6,000 Reputation points Microsoft External Staff

2024-01-24T05:18:41.93+00:00

Hi @Jize Xu
Glad to know your issue has been resolved. Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others "I'll repost your solution in case you'd like to accept the answer.
Harishga 6,000 Reputation points Microsoft External Staff

2024-01-25T05:39:10.26+00:00

Hi @Jize Xu
Just checking in to see if the below answer helped. If this answers your query, do click "Accept the answer” for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

1 answer

Your answer

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Jize Xu 10 Reputation points Microsoft Employee

2024-01-24T03:05:57.9833333+00:00

Finally, I solved it with these codes. Notice that we should use "fs.azure.sas.token.provider.type.<ACCOUNT>.dfs.core.windows.net" as the provider key. The document I mentioned above misses the part ".<ACCOUNT>.dfs.core.windows.net"

%%pyspark spark.conf.set("fs.azure.account.auth.type.<ACCOUNT>.dfs.core.windows.net", "SAS") spark.conf.set("fs.azure.sas.token.provider.type.<ACCOUNT>.dfs.core.windows.net", "com.microsoft.azure.synapse.tokenlibrary.ConfBasedSASProvider") spark.conf.set("spark.storage.synapse.<CONTAINER>.<ACCOUNT>.dfs.core.windows.net.sas", "<SAS KEY>")
Harishga 6,000 Reputation points Microsoft External Staff

2024-01-24T05:18:41.93+00:00

Hi @Jize Xu
Glad to know your issue has been resolved. Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others "I'll repost your solution in case you'd like to accept the answer.
Harishga 6,000 Reputation points Microsoft External Staff

2024-01-25T05:39:10.26+00:00

Hi @Jize Xu
Just checking in to see if the below answer helped. If this answers your query, do click "Accept the answer” for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Answer 1

Hi @Jize Xu
Welcome to Microsoft Q&A platform and thanks for posting your question here. I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer.

Ask:
How to connect an ADLS Gen2 container with a SAS token in Synapse notebook without using a linked service. The user has tried a solution from a Microsoft documentation page but is receiving an error message.

Solution:
Finally, I solved it with these codes. Notice that we should use "fs.azure.sas.token.provider.type.<ACCOUNT>.dfs.core.windows.net" as the provider key. The document I mentioned above misses the part ".<ACCOUNT>.dfs.core.windows.net"

%%pyspark
spark.conf.set("fs.azure.account.auth.type.<ACCOUNT>.dfs.core.windows.net", "SAS") spark.conf.set("fs.azure.sas.token.provider.type.<ACCOUNT>.dfs.core.windows.net", "com.microsoft.azure.synapse.tokenlibrary.ConfBasedSASProvider") spark.conf.set("spark.storage.synapse.<CONTAINER>.<ACCOUNT>.dfs.core.windows.net.sas", "<SAS KEY>")

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

Hope this helps. Do let us know if you any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

Is there a way to connect ADLS gen2 container with sas token in Synapse notebook?

1 answer

Your answer