REST API Connection through ADF Error

Question

REST API Connection through ADF Error

S B R Vamsi Manda 36

Failure happened on 'Source' side. ErrorCode=RestCallFailedWithClientError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Rest call failed with client error, status code 401 Unauthorized, please check your activity settings.
Request URL: https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelines?api-version=2018-06-01.
Response: {"error":{"code":"AuthenticationFailedInvalidHeader","message":"Authentication failed. The 'Authorization' header is provided in an invalid format."}},Source=Microsoft.DataTransfer.ClientLibrary,'

I have provided respective subid, resourcegroup , datafactoryname and the Additional header as API-KEY and value as 1234

can you please let me know do we require a header to get rest api for pipeline information from rest call

MartinJaffer-MSFT 26,236 Reputation points

2022-08-18T16:10:40.317+00:00

Hello @S B R Vamsi Manda ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .

Thanks
Martin

1 answer

Your answer

MartinJaffer-MSFT 26,236 Reputation points

2022-08-18T16:10:40.317+00:00

Hello @S B R Vamsi Manda ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .

Thanks
Martin

Answer 1

MartinJaffer-MSFT 26,236

Hello @S B R Vamsi Manda and welcome to Microsoft Q&A.

From the second error message it sounds like you might not have given the proper authorization header.

It is expecting like

Header name: "Authorization"
Heaver value: "Bearer XXXXXXXXXXXXXXXXXXXXXX"

This bearer token is acquired in a separate step, using https://login.microsoftonline.com/common/oauth2/authorize

However this is only one of the ways to authorize. Another easier way is to use the System Assigned Managed Identity. For this you need to go to the Azure Portal, and on the resource group of the Data Factory, assign either "Reader" or "Data Factory Contributor" role to the Factory.

Mike Hill 5 Reputation points

2023-03-02T15:04:47.5966667+00:00
This approach (using System Assigned Managed Identity) does not appear to work when attempting to use a Cosmos DB SQL API call. I am using an Azure Data Factory pipeline with a web activity using the following settings:

As this results in error:

{ "code": "Unauthorized", "message": "Authorization header doesn't confirm to the required format. Please verify and try again.\r\nActivityId: 94aaf22e-3c6a-4a1d-be74, Windows/10.0.17763 cosmos-netstandard-sdk/3.18.0", "ADFWebActivityResponseHeaders": { "Transfer-Encoding": "chunked", "x-ms-activity-id": "94aaf22e-3c6a-4a1d-be74-", "x-ms-gatewayversion": "2.0.0", "Date": "Thu, 02 Mar 2023 14:53:28 GMT", "Server": "Compute", "Content-Type": "application/json" }, "effectiveIntegrationRuntime": "AutoResolveIntegrationRuntime (UK South)",

It should be noted that the Authorization header needed to access Cosmos SQL API differs from the standard OAuth2 header, in that it requires the following format:

authorization:type=aad&ver=1.0&sig=eyJ0eXAiOiJKV1QiLCJhbGciOiJ...........................

When I use this type of header in postman (same URL to Cosmos), I can get results. See below.

So is this a case of Azure Data Factory not setting the authorization header correctly?
MartinJaffer-MSFT 26,236 Reputation points

2023-03-14T16:47:35.7266667+00:00

That is a really good observation Mike. I wasn't aware of the different form. The Web activity doesn't know about different ways of structuring the Auth for each different service. It is generalized. Looks like you will need to explicitly set the Auth header.

Share via

REST API Connection through ADF Error

1 answer

Your answer