How to add cookie header in ADF copy activity?

Question

I am working with a SOAP API that uses cookies to authenticate calls. By calling the "Connect" endpoint in a Web Activity I am able to obtain the cookie and using Set-Variable I am able to use that cookie in a subsequent Copy Activity.

Here is where the problem starts however. In a Copy Activity you can only set "additional headers" and when I put my cookie in there using the required key-value pair (header name: "Cookie". Value: "ASP.NET_SessionId=xxxxxx") this cookie will somehow not be sent or used in the API call.

The other key-value pairs that I send in the additional headers do seem to work (Host, Content-Type, and Action).

In a Web Activity there is the option of adding headers and there it works perfectly. The problem with using a Web Activity is that you can't simply sent the output of the API call to file without using some workarounds that I don't want to have working in production (like using additional columns or a blob storage in a Web Activity for example).

So in short, the question is: Can I send a cookie in the header of an API call in the source of a Copy Activity?

Answer 1

Hello Sander Koster,

Welcome to the MS Q&A platform.

Unfortunately, Cookies (as output from HTTP Connector) are not supported with ADF, and PG is evaluating this as a feature request.

Here are a few feedback items submitted for this feature request.

https://feedback.azure.com/d365community/idea/b79ada56-7226-ec11-b6e6-000d3a4f032c

https://feedback.azure.com/d365community/idea/d4649a65-6e26-ec11-b6e6-000d3a4f032c

https://feedback.azure.com/d365community/idea/a01b3576-7126-ec11-b6e6-000d3a4f032c

All the feedback shared in this forum is actively monitored and reviewed by respective product owners. This would be open for the user community to upvote & comment on. This allows product teams to prioritize idea against our existing feature backlog effectively and gives insight into the potential impact of implementing the suggested feature

I would suggest leaving a comment and up-vote for this feature request.

I hope this helps. Please let me know if you have any further questions.