synapse notebook error

Question

My synapse notebook, spark session configured to use service principle to storage account.

When I run this notebook from pipeline, getting the error.

Operation failed: "This request is not authorized to perform this operation using this permission.", 403

In the pipeline I am using the notebook activity to run the script.

As per the document, I have provided Storage Blob Data Contributor to access the storage account. What could be the reason for this error.

My second question, when I create the linked service with service principle the linked service works fine but the notebook failed. below error
: Status code: -1 error code: null error message: Auth failure: HTTP Error -1CustomTokenProvider getAccessToken threw java.io.IOException : POST failed with 'Bad Request' (400) and message: {"result":"DependencyError","errorId":"BadRequest","errorMessage":

Answer 1

Hello Hari Krishna
Welcome to the MS Q&A platform.

Synapse notebooks use Azure Active Directory (Azure AD) pass-through to access the ADLS Gen2 accounts. If you are running the notebook directly on the synapse, your account needs Storage Blob Data Contributor to access the ADLS Gen2 account (or folder).

If you are running the notebook via the pipeline, the synapse workspace managed service identity needs Storage Blob Data Contributor to access the ADLS Gen2 account (or folder).

For your other question, the error message seems like there was an authentication failure when trying to access a resource.

Please check if you provided the storage blob data contributor access

https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/using-the-workspace-msi-to-authenticate-a-synapse-notebook-when/ba-p/2330029

Follow the above tech community article and run the code using the service principle.

You can use either managed identity or the service principal

Please try and let me know if you still have any issues further.

Answer 2

Hi there @hari krishna
The error you are encountering indicates that the request being made from your notebook or pipeline is not authorized to perform the operation with the given permission. There could be a few reasons for this:

1. Insufficient or incorrect permissions: Double-check the permissions assigned to the service principal being used by your notebook or pipeline.Ensure that it has the necessary permissions.
2. Role assignment not propagated: Sometimes, it may take some time for the role assignments to propagate fully. Ensure that sufficient time has passed since the role assignment was made.

Regarding your second question, the error message suggests an authentication failure during the process of obtaining an access token for the service principal.

• Azure AD configuration issues: Verify that the AAD application associated with the service principal has the necessary permissions and configurations for the desired access. Additionally, ensure that the AAD tenant is accessible and functioning properly.
• Incorrect service principal credentials: Double-check that the service principal credentials (client ID, client secret, tenant ID) provided in the linked service configuration are accurate and match the corresponding Azure Active Directory (AAD) application.

Also you may want to review the detailed logs and error stack traces to identify any specific error codes or messages that could provide more insight into the cause of the issue.

Answer 3

One thing that is not mentioned in Microsoft's guides for setting up Synapse nor in most support responses is that Spark Pools cannot access firewall-enabled storage accounts. See:
https://learn.microsoft.com/en-us/troubleshoot/azure/synapse-analytics/spark/spark-jobexec-storage-access#unsupported-scenarios

Answer 4

duplicated answer