Synapse pyspark call rest api sample

Question

I have a rest api call class now needs to update to linked service to call rest api.

due to security issue, I have to use linked service to call rest api. Any sample code I can use?

After getting the token, I can use bearer token to call rest api. How can I do the similar in Synapse with linked service(MSI)?

        app_id = configuration.get('app_id');
        app_secret = configuration.get('app_secret');
        tenant_id = '';
        credential = ClientSecretCredential(tenant_id, app_id, app_secret);
		self.token = credential.get_token(resource).token;

Answer 1

Hi @Simon Zhang,

The documentation in question that you're looking for is https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-secure-credentials-with-tokenlibrary?pivots=programming-language-python. It provides various python snippets using the mssparkutils package. You can utilize this library to manage credentials and access tokens. For example,

from mssparkutils.credentials import getToken

# Get the token for the storage account
token = getToken("https://storage.azure.com/")

# Use the token to access the storage account
spark.conf.set("fs.azure.account.auth.type.<storage-account-name>.dfs.core.windows.net", "OAuth")
spark.conf.set("fs.azure.account.oauth.provider.type.<storage-account-name>.dfs.core.windows.net", "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
spark.conf.set("fs.azure.account.oauth2.client.id.<storage-account-name>.dfs.core.windows.net", token["accessToken"])

In order to leverage the getToken method, your resource needs to have a system identity assigned to it and that identity should have RBAC roles to your Synapse resource; see https://learn.microsoft.com/en-us/azure/synapse-analytics/security/synapse-workspace-understand-what-role-you-need for more details.

Answer 2

Thanks for posting your question in the Microsoft Q&A forum.

I think you need to create client after setting credential:

synapse_client = ArtifactsClient(credential, endpoint='https://<your-workspace-name>.dev.azuresynapse.net')

** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful **