How to save complete web activity output to ADLS Gen2?

Venkatesh P 21 Reputation points
2022-06-27T16:32:59.477+00:00

Hello there,

I have a requirement to fetch the GET method output of web activity to ADLS Gen2. Can you please guide me on how to achieve this?

Thanks in advance.

Thanks,
Venkat

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
0 comments No comments
{count} votes

Answer accepted by question author
  1. AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator
    2022-06-28T09:13:47.12+00:00

    Hi @Venkatesh P ,

    Welcome to Microsoft Q&A platform and thanks for posting your question here.

    I assume you are trying to store the web activity output to a file in ADLS. Please let me know if that is not the case.

    • Create a variable and use set variable activity after the Web activity and use this expression : @{activity('Web1').output.value[0]} in the value of set Variable.
    • Use copy activity and have a dummy .csv file with one column as the source dataset . Add additional column and provide a new column named WebActvityOutput with value = @{variables('var1')} . In the sink , use the same dataset. In the mapping tab , click on Import schema and delete the column from dummy file and only keep the additionally added column.
    • Execute the pipeline and check if the webActivity output got stored in the .csv in the ADLS . 215570-webactivityoutput.gif

    Hope this will help. Please let us know if any further queries.


    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
      Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    2 people found this answer helpful.

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.