How to add cookie header in ADF copy activity?

Sander Koster 50 Reputation points
2023-05-27T09:04:56.46+00:00

I am working with a SOAP API that uses cookies to authenticate calls. By calling the "Connect" endpoint in a Web Activity I am able to obtain the cookie and using Set-Variable I am able to use that cookie in a subsequent Copy Activity.

Here is where the problem starts however. In a Copy Activity you can only set "additional headers" and when I put my cookie in there using the required key-value pair (header name: "Cookie". Value: "ASP.NET_SessionId=xxxxxx") this cookie will somehow not be sent or used in the API call.

The other key-value pairs that I send in the additional headers do seem to work (Host, Content-Type, and Action).

In a Web Activity there is the option of adding headers and there it works perfectly. The problem with using a Web Activity is that you can't simply sent the output of the API call to file without using some workarounds that I don't want to have working in production (like using additional columns or a blob storage in a Web Activity for example).

So in short, the question is: Can I send a cookie in the header of an API call in the source of a Copy Activity?

Additional Headers

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,085 questions
0 comments No comments
{count} votes

Accepted answer
  1. Bhargava-MSFT 28,936 Reputation points Microsoft Employee
    2023-05-30T21:21:35.4566667+00:00

    Hello Sander Koster,

    Welcome to the MS Q&A platform.

    Unfortunately, Cookies (as output from HTTP Connector) are not supported with ADF, and PG is evaluating this as a feature request.

    Here are a few feedback items submitted for this feature request.

    https://feedback.azure.com/d365community/idea/b79ada56-7226-ec11-b6e6-000d3a4f032c

    https://feedback.azure.com/d365community/idea/d4649a65-6e26-ec11-b6e6-000d3a4f032c

    https://feedback.azure.com/d365community/idea/a01b3576-7126-ec11-b6e6-000d3a4f032c

    All the feedback shared in this forum is actively monitored and reviewed by respective product owners. This would be open for the user community to upvote & comment on. This allows product teams to prioritize idea against our existing feature backlog effectively and gives insight into the potential impact of implementing the suggested feature

    I would suggest leaving a comment and up-vote for this feature request.

    I hope this helps. Please let me know if you have any further questions.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful