ADF Pipeline calling Synapse Notebook Insufficient Priveledges for microsoft.azure.synapse.tokenlibrary.TokenLibrary API

Question

I have created a notebook within synapse and would like to add it to an ADF pipeline for orchestration. I am using the Azure Token Library to access my key vault to retrieve secrets. When running in Synapse the notebook runs successfully.

When executing within my ADF pipeline I am met with the error:

Py4JJavaError: An error occurred while calling z:com.microsoft.azure.synapse.tokenlibrary.TokenLibrary.getSecret. : com.microsoft.azure.synapse.tokenlibrary.TokenLibrary$NonRetryableStatusException$1: POST failed with 'Unauthorized' (401) and message: {"result":"DependencyError","errorId":"Unauthorized","errorMessage":"[Code=AccessControlUnauthorized, Target=, Message=Insufficient permissions to call this API. 5d2dd050-db90-4c10-b216-d9f55a987e81 does not have Microsoft.Synapse/workspaces/read, Microsoft.Synapse/workspaces/linkedServices/useSecret/action on scope workspaces/fplukpd01-datalake-syn01/linkedServices/fplukpd01datalakekv01]. TraceId : 029f69c7-9f55-4a9c-a5c2-68ea1f435095. Error Component : LSR"}

ADF service principle has the Synapse Contributor RBAC Role to the synapse workspace.

I believe the solution is within the guid in the error line below, however I dont know what resource/service principle this refers to.

5d2dd050-db90-4c10-b216-d9f55a987e81 does not have Microsoft.Synapse/workspaces/read

Answer 1

Hi WZIAFP ,

Thankyou for using Microsoft Q&A platform and thanks for posting your question here.

As I understand your question , you are trying to Synapse notebook via ADF pipeline. However, you are facing error while doing that. Please let me know if that's not the case.

Since you mentioned that the Service principal already has 'Synapse contributor role' , I feel the permission for service principal which is required to retrieve the secret from key vault might be missing.

It is likely that the service principal associated with your Azure Data Factory (ADF) doesn't have the necessary permissions to access the secrets in your key vault. To resolve this issue, you will need to grant the ADF service principal access to the secrets in the key vault.

Here's how you can grant access:

1. Navigate to the key vault in the Azure portal.
2. Click on "Access policies" in the left-side menu.
3. Click on the "Add Access Policy" button.
4. Select the "Key Management" operation, and "Get" action.
5. In the "Select principal" section, select the ADF service principal.
6. Click on the "Add" button to add the policy, and then click on the "Save" button to save the changes to the key vault.

Please let us know if it helps. If giving the above-mentioned permission doesn't resolve the issue, then kindly revert back, I will check internally what's causing the issue. Thankyou