How to call Synapse Data Plane rest api from within Synapse?

Raffaele Brivio 101 Reputation points
2021-11-02T10:19:08.4+00:00

In my Synapse workplace I have an integration pipeline that needs to get information on pipeline runs (running or finished) by passing to it the pipelineRunId as a parameter. So Synapse should, in some sense, perform a search on itself and I found the perfect rest endpoint in this docs.
I build a simple Web Activity to test the api, using Managed Identity as auth method, but I keep getting this error:

{
"code":"InvalidTokenAuthenticationAudience",
"message":"Token Authentication failed with SecurityTokenInvalidAudienceException - IDX10214: Audience validation failed. Audiences: '[PII is hidden]'. Did not match: validationParameters.ValidAudience: '[PII is hidden]' or validationParameters.ValidAudiences: '[PII is hidden]'.",
"failureType":"UserError",
"target":"Get Info",
"details":[]
}

the redacted web activity:

{
"name": "Get Info",
"type": "WebActivity",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"url": "https://{MYWORKPLACE}.dev.azuresynapse.net/pipelineruns/{MYPIPELINERUNID}?api-version=2020-12-01",
"connectVia": {
"referenceName": "AutoResolveIntegrationRuntime",
"type": "IntegrationRuntimeReference"
},
"method": "GET",
"authentication": {
"type": "MSI",
"resource": "https://management.azure.com/"
}
}
}

This seems to be an issue with synapse itself since with the same method I can successfully call other endpoints like https://management.azure.com/subscriptions?api-version=2020-01-01

What am I missing?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,373 questions
0 comments No comments
{count} votes

Accepted answer
  1. Raffaele Brivio 101 Reputation points
    2021-11-02T14:38:23.56+00:00

    Managed to solve it, I just needed to pay more attention to the docs.

    Synapse has 2 type of api:

    • classic azure management rest api
    • data-plane rest pi

    synapse internal object management is done through data plane api, so things like pipelines, activities, triggers, notebooks etc are in the scope of data plane api and to use them with Managed Identity auth you just have to specify the correct Resource in order for the token issuer to include the right scope. So for Data Plane api you would wanna use
    "https://dev.azuresynapse.net/" as resource, opposing to the classic "https://management.core.windows.net/".

    see also here:
    https://dev.to/jayendran/azure-synapse-analytics-workspaces-deploy-and-debug-part-2-l5c
    https://learn.microsoft.com/en-us/rest/api/synapse/#common-parameters-and-headers

    0 comments No comments

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.