Azure Data Factory Web Connection with System Assigned Managed Identity is not working for API Authentication

Question

Azure Data Factory Web Connection with System Assigned Managed Identity is not working for API Authentication

Jonas Lomholdt 6

I have a ADF (Azure Data Factory) where my pipeline needs to do a Web request to an API that's secured by Azure AD. I have configured a linked service Web Connection for the endpoint and selected "System Assigned Managed Identity" for Authentication. In the "Resource" field I have added the "Application ID URI" for the app registration connected to the API I'm trying to call. In my pipeline I have added the "Web" Activity and try to invoke the API. It fails with a 401 saying "it lacks valid authentication credentials".

"Invoking endpoint failed with HttpStatusCode - '401 : Unauthorized', message - 'Client request has not been completed because it lacks valid authentication credentials for the requested endpoint(url).'"

I have control over the API so I enabled PII logging and pulled the token out it receives. So I can confirm a JWT token is being passed along in the headers. It even validates the aud claim correctly. But still it's not working.

If I run an `az account get-access-token --resource

ShaikMaheer-MSFT 38,546 Reputation points Microsoft Employee Moderator

2023-11-27T08:53:34.1166667+00:00

Hi Jonas Lomholdt,

Thank you for posting query in Microsoft Q&A Platform.

From the error message, it seems Managed Identity don't have required permissions. We need to consider granting permissions for Managed Identity in API. API admin team can help you to have permissions for your Managed Identity on API.

When you ran queries explicitly and got tokens, that tokens were something given to your identity or the identity which running the script. Hence, they may work.

Hope this helps. Please let me know how it goes. Thank you.
Jonas Lomholdt 6 Reputation points

2023-11-27T14:06:26.1233333+00:00

Thanks @ShaikMaheer-MSFT - API Admin team, do you mean someone here on learn.microsoft.com or internally in our org? The API is not using any roles or other claims and should be open for anyone in the AD/Entra org. I was hoping the Managed Identity would just be part of it as well, but I guess it's not. So question might be how I add a Managed Identity to the AD org so it can access the api?
ShaikMaheer-MSFT 38,546 Reputation points Microsoft Employee Moderator

2023-11-28T06:03:48.82+00:00

Hi Jonas Lomholdt, Could you please help to understand how API is created and secured with AAD? Is it Azure Function with http trigger type? Once we have little idea on who and why using which azure service exactly API got created and secured with AAD then we can research more on how to give access to system managed identity.
Jonas Lomholdt 6 Reputation points

2023-11-28T09:15:14.8566667+00:00

Thanks @ShaikMaheer-MSFT - It's a ASP.NET Core API using the Microsoft.Identity.Web package using the AddMicrosoftIdentityWebApiAuthentication() extension method. It's then configured to use our Azure AD tenant. It works if I use my own token to do a request towards the API.

ShaikMaheer-MSFT 38,546 Microsoft Employee Moderator

Hi Jonas Lomholdt, Thank you for details. So it's a code written in local, not in any azure service like Azure Function?

Could you please try using Azure.Identity as well and see if that helps for System managed identities?

Here is the sample code:

using Azure.Identity;
using Microsoft.Graph;
using Microsoft.Identity.Web;

// Create a new instance of the DefaultAzureCredential class
var credential = new DefaultAzureCredential();

// Use the credential to obtain an access token for the managed identity
var token = await credential.GetTokenAsync(new TokenRequestContext(new[] { "https://graph.microsoft.com/.default" }));

// Use the access token to create a new instance of the GraphServiceClient class
var graphClient = new GraphServiceClient(new DelegateAuthenticationProvider(async (requestMessage) =>
{
    requestMessage.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token.Token);
}));

In this example, the DefaultAzureCredential class is used to obtain an access token for the managed identity. The access token is then used to authenticate the user and create a new instance of the GraphServiceClient class.

I hope this helps! Let me know if you have any further questions.

Jonas Lomholdt 6 Reputation points

2023-11-29T15:26:11.2433333+00:00

Thanks @ShaikMaheer-MSFT - I'm trying to do it the other way around unfortunately ADF --> API. And as far as I know I have no control over what ADF put's in the auth. I just thought I would be able to use the Web activity in ADF to request an Azure AD secured API - but I don't know how to grant ADF's Managed Identity permission to invoke the API.
Patel, Siddhartha 0 Reputation points

2024-10-21T18:38:04.15+00:00

'm trying to do it the other way around unfortunately ADF --> API. And as far as I know I have no control over what ADF put's in the auth. I just thought I would be able to use the Web activity in ADF to request an Azure AD secured API - but I don't know how to grant ADF's Managed Identity permission to invoke the API.

Your answer

ShaikMaheer-MSFT 38,546 Reputation points Microsoft Employee Moderator

2023-11-27T08:53:34.1166667+00:00

Hi Jonas Lomholdt,

Thank you for posting query in Microsoft Q&A Platform.

From the error message, it seems Managed Identity don't have required permissions. We need to consider granting permissions for Managed Identity in API. API admin team can help you to have permissions for your Managed Identity on API.

When you ran queries explicitly and got tokens, that tokens were something given to your identity or the identity which running the script. Hence, they may work.

Hope this helps. Please let me know how it goes. Thank you.
Jonas Lomholdt 6 Reputation points

2023-11-27T14:06:26.1233333+00:00

Thanks @ShaikMaheer-MSFT - API Admin team, do you mean someone here on learn.microsoft.com or internally in our org? The API is not using any roles or other claims and should be open for anyone in the AD/Entra org. I was hoping the Managed Identity would just be part of it as well, but I guess it's not. So question might be how I add a Managed Identity to the AD org so it can access the api?
ShaikMaheer-MSFT 38,546 Reputation points Microsoft Employee Moderator

2023-11-28T06:03:48.82+00:00

Hi Jonas Lomholdt, Could you please help to understand how API is created and secured with AAD? Is it Azure Function with http trigger type? Once we have little idea on who and why using which azure service exactly API got created and secured with AAD then we can research more on how to give access to system managed identity.
Jonas Lomholdt 6 Reputation points

2023-11-28T09:15:14.8566667+00:00

Thanks @ShaikMaheer-MSFT - It's a ASP.NET Core API using the Microsoft.Identity.Web package using the AddMicrosoftIdentityWebApiAuthentication() extension method. It's then configured to use our Azure AD tenant. It works if I use my own token to do a request towards the API.
ShaikMaheer-MSFT 38,546 Reputation points Microsoft Employee Moderator

2023-11-29T06:09:49.99+00:00

Hi Jonas Lomholdt, Thank you for details. So it's a code written in local, not in any azure service like Azure Function?

Could you please try using Azure.Identity as well and see if that helps for System managed identities?

Here is the sample code:

using Azure.Identity; using Microsoft.Graph; using Microsoft.Identity.Web; // Create a new instance of the DefaultAzureCredential class var credential = new DefaultAzureCredential(); // Use the credential to obtain an access token for the managed identity var token = await credential.GetTokenAsync(new TokenRequestContext(new[] { "https://graph.microsoft.com/.default" })); // Use the access token to create a new instance of the GraphServiceClient class var graphClient = new GraphServiceClient(new DelegateAuthenticationProvider(async (requestMessage) => { requestMessage.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token.Token); }));

In this example, the DefaultAzureCredential class is used to obtain an access token for the managed identity. The access token is then used to authenticate the user and create a new instance of the GraphServiceClient class.

I hope this helps! Let me know if you have any further questions.
Jonas Lomholdt 6 Reputation points

2023-11-29T15:26:11.2433333+00:00

Thanks @ShaikMaheer-MSFT - I'm trying to do it the other way around unfortunately ADF --> API. And as far as I know I have no control over what ADF put's in the auth. I just thought I would be able to use the Web activity in ADF to request an Azure AD secured API - but I don't know how to grant ADF's Managed Identity permission to invoke the API.
Patel, Siddhartha 0 Reputation points

2024-10-21T18:38:04.15+00:00

'm trying to do it the other way around unfortunately ADF --> API. And as far as I know I have no control over what ADF put's in the auth. I just thought I would be able to use the Web activity in ADF to request an Azure AD secured API - but I don't know how to grant ADF's Managed Identity permission to invoke the API.

Share via

Azure Data Factory Web Connection with System Assigned Managed Identity is not working for API Authentication

Your answer