ADF Web Activity - Fails with custom Authorization Header

Question

ADF Web Activity - Fails with custom Authorization Header

TX Aggie 00 26

In ADF, we need to ingest some data from Amazon Vendor Central via the SP-API. This requires you to sign the headers. When using the custom Authorization header, the web activity fails with the following error:

Header:

"Authorization": "AWS4-HMAC-SHA256 Credential=ASIAUSIDVNGYYVMPNRPR/20221214/us-east-1/execute-api/aws4_request, SignedHeaders=host;x-amz-access-token;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=a7bf55552b65c231b54440ff5c543f851297214f2bd514d9301c6b3583d435c8"

Error:

Error calling the endpoint 'https://sellingpartnerapi-na.amazon.com'. Response status code: 'NA - Unknown'. More details: Exception message: 'NA - Unknown [ClientSideException] The format of value 'AWS4-HMAC-SHA256 Credential=ASIAUSIDVNGYYVMPNRPR/20221214/us-east-1/execute-api/aws4_request, SignedHeaders=host;x-amz-access-token;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=a7bf55552b65c231b54440ff5c543f851297214f2bd514d9301c6b3583d435c8' is invalid.'.  
  
Request didn't reach the server from the client. This could happen because of an underlying issue such as network connectivity, a DNS failure, a server certificate validation or a timeout.

Does something in this header value need to be escaped for ADF to successfully use this?

Thanks,
Scott

TX Aggie 00 26 Reputation points

2022-12-15T00:48:57.617+00:00

To reproduce this issue, create a new pipeline and add a web activity. Leave everything default, add "http://www.google.com" with method GET and add the authorization header on the previous post.

You will get the same error message

I found that if I take the header value bit by bit, it fails as soon as I add the comma

Keep in mind that this header value is a valid value and works in every API tool except ADF web activity

Thanks,
Scott
TX Aggie 00 26 Reputation points

2022-12-15T05:26:59.487+00:00

Also posted in stackoverflow

2 answers

Your answer

TX Aggie 00 26 Reputation points

2022-12-15T00:48:57.617+00:00

To reproduce this issue, create a new pipeline and add a web activity. Leave everything default, add "http://www.google.com" with method GET and add the authorization header on the previous post.

You will get the same error message

I found that if I take the header value bit by bit, it fails as soon as I add the comma

Keep in mind that this header value is a valid value and works in every API tool except ADF web activity

Thanks,
Scott
TX Aggie 00 26 Reputation points

2022-12-15T05:26:59.487+00:00

Also posted in stackoverflow

Answer 1

AnnuKumari-MSFT 34,551 Microsoft Employee Moderator

Hi @TX Aggie 00 ,

Thankyou for using Microsoft Q&A platform and thanks for posting your query.

As I understand your issue, you are facing network connectivity related error while using web activity in Azure data factory. Please let me know if that is not the correct understanding.

I tried to replicate your issue using google API, and was able to get through it . As you mentioned, you are getting error as soon as you add comma in the headers. You can add multiple headers instead of separating them by comma like below:

Please try this approach and let us know how it goes. Thankyou.

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.
Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

TX Aggie 00 26 Reputation points

2022-12-15T14:51:13.913+00:00
Thanks for the reply. I am surprised it worked for you. I belong to several different organizations (directories) and have tried the same across all of them. All error out. This is definitely not a network issue. Using the example above across different accounts,

This works

AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20130524/us-east-1/s3/aws4_request

This does not

AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20130524/us-east-1/s3/aws4_request,SignedHeaders=date;host;x-amz-content-sha256;x-amz-date;x-amz-storage-class,Signature=98ad721746da40c64f1a55b78f14c238d841ea1380cd77a1b5971af0ece108bd

I will test breaking them up into separate headers and see if AWS accepts that format. The AWS sigv4 documentation is very specific in the format. I guess the point is that I shouldn't have to do that since it is a valid header value as it is and the web activity should accept it as such.

Thanks,
Scott
TX Aggie 00 26 Reputation points

2022-12-15T15:54:27.053+00:00
Just to follow up on this. I did test breaking these into separate headers and Amazon will not accept that. Amazon returns the following error:

Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization=AWS4-HMAC-SHA256 Credential=ASIAUSIDVN************/20221215/us-east-1/execute-api/aws4_request

So unfortunately, this work-around will not work and I am back to square one. The only way I can see how to achieve integrating Amazon Selling Partner data in ADF is to write multiple functions to get the data into the datalake, unless the web activity gets fixed or someone else out there has found a way

Thanks,
Scott

Answer 2

Andres De Abreu 0

@TX Aggie 00 where you able to connect to the SP-API? looking for a solution

Thanks

Share via

ADF Web Activity - Fails with custom Authorization Header

2 answers

Your answer