AzureData Factory getting output from WebActivity

Caycee 1 Reputation point
2020-12-16T19:29:26.707+00:00

With ADF I created web activity with Post method and got my output. I need to get the output that's in json to be stored in my blob storage. I tried copy data method but it gives me error of "Failure happened on 'Source' side." I tried by calling 2 web activity with PUT method to add to blob but getting this error. Verified the container and it exists.

Error code
2108

User configuration issue
Details
<?xml version="1.0" encoding="utf-8"?><Error><Code>ContainerNotFound</Code><Message>The specified container does not exist.</Message></Error>

I have added dynamic value as an input in the copy activity but gives me error in debug. I am able to see the desired output in the input(debug mode) but output in not the same. Here is what it shows as output:

{
"effectiveIntegrationRuntime": "DefaultIntegrationRuntime (West US)",
"executionDuration": 0,
"durationInQueue": {
"integrationRuntimeQueue": 1
},
"billingReference": {
"activityType": "ExternalActivity",
"billableDuration": [
{
"meterType": "AzureIR",
"duration": 0.016666666666666666,
"unit": "Hours"
}
]
}
}
Goal here is to get the output stored in a blob as csv so I can import that file to sql table. How do I get the output stored in blob storage?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,726 questions
{count} votes

2 answers

Sort by: Most helpful
  1. HimanshuSinha-msft 18,621 Reputation points Microsoft Employee
    2020-12-23T21:56:09.227+00:00

    Hello @Caycee ,

    I think you can use a copy activity only to change the JSON to .csv . This is what i did , i just pulled in an public API which gives JSON as a response .

    https://api.github.com/users/hadley/orgs

    I used a COPY activity and used the source as REST and sink as blob and was able to achieve the goal . I am putting the gif below hope this helps .

    50896-caycee.gif

    Thanks Himanshu
    Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

  2. Caycee 1 Reputation point
    2021-01-04T15:13:26.36+00:00

    Thank you @HimanshuSinha-msft This only works for get Method in API. I am trying to get post method as I am requesting values for data. Is there another way other than ADF?