Post file from Blob to Web Activity in Azure Data Factory

Question

Hi,
I have been working on this problem for days, weeks and I can't seem to get it right.
What I am trying to do is send a file from a blob storage to a website using Web activity on ADF.
Attached is what I have so far. I can't figure out what should go in the body and how do I reference the blob.

Thanks in advance
Danny V

Answer 1

Hello @Anonymous and welcome to Microsoft Q&A. Thank you for your question.

I will inquire internally as to how the linked service and dataset features of the web activity are supposed to be used.

In the meantime, I will provide you with a solution I am certain will work.

Use 2 Web activities. The first fetches the data from the blob. The second web activity takes the output of the first activity, and uses it as body.

In the below example I use another blob to stand in for the website. (It effectively copies the blob).
Here for fetching the blob the URL takes the form of :

https://<storageAccountName>.blob.core.windows.net/<containerName>/<folder>/<file>

I also use MSI authentication. For this to work, you will need to go to the storage account (or container) access control and assign the Data Factory permissions (Blob reader at minimum. I use blob contributor).

Answer 2

Thanks for the great tip. I tried various variation of your method and keep getting this error:
<?xml version="1.0" encoding="utf-8"?> <Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:55403b3b-001e-009a-44a1-c19684000000 Time:2020-11-23T14:02:22.0447390Z</Message></Error>
It seems that my issue is related on how I am authenticating. I can't seem to figure out the best way or how to do it effectivily.

D

Answer 3

Hi, I am about to give up. This is impossible. I have been on this for hours and days. It's not working. How do you get the MSI Authentication to work.
I am very new at this, so I don't understand. What URL am I typing in under the resource? HTTPS://stroage.azure.com does not work.

Thanks
Danny

Answer 4

@Anonymous I am sorry this has been a frustrating process. I hope the below has the missing piece.

If you are getting error like pictured below, saying

Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature ... Authentication scheme Bearer is not supported in this version.

Then you need to add the header x-ms-version and value 2017-11-09 . (this is shown in picture of earlier solution)

The resource should be https://storage.azure.com/

If you are getting error saying

AuthorizationPermissionMismatch</Code><Message>This request is not authorized to perform this operation using this permission.

Then you may need to go to storage account and give more permissions, like pictured below. Wait several minutes for the permissions to propagate, otherwise you might get a false negative.

Answer 5

Hi Martin, Thanks, I figure that out and was able to get that part to work. However, now I get an error message when I am trying to post a word doc to a website. The error message is not much help

The input is as follows. It seems to show the Word doc in it's binary format. I believe.

And the Web activity I have set up is as follows.

Any ides as to what I am missing?

Thanks