how to capture output of append variable in adf and write to json or csv file in adls

Question

how to capture output of append variable in adf and write to json or csv file in adls

Raviyaramati-1837 1

i have a set up like below , i need to capture data from for each activity to json or csv

Answer 1

KranthiPakala-MSFT 43,602 Microsoft Employee

Hi @Raviyaramati-1837 ,

Welcome to Microsoft Q&A forum and thanks for posting your query.

One way to achieve this requirement is by utilizing "Add additional columns during copy" feature as explained in this stackoverflow thread - [Transfer the output of 'Set Variable' activity into a json file Azure Data Factory

After your append variable activity, have a subsequent Copy activity and define a column and pass the value of append variable as described in stackoverflow thread.

Hope this info helps. Do let us know if you have further query.

----------

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.

KranthiPakala-MSFT 43,602 Reputation points Microsoft Employee

2021-07-14T21:24:00.217+00:00

Hi @Raviyaramati-1837 ,

Just checking in to see if the above information was helpful. If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thank you
KranthiPakala-MSFT 43,602 Reputation points Microsoft Employee

2021-07-15T19:43:43.287+00:00

Hi @Raviyaramati-1837 ,

We still have not heard back from you. Just wanted to check if the above suggestion was helpful? If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.
Raviyaramati-1837 1 Reputation point

2021-07-17T06:29:21.97+00:00

Hello @KranthiPakala-MSFT

i have gone through stack overflow thread earlier as well, but no luck , as it say point dataset to json , but to create dataset we need linked service point connection ,

1) where should we place or point dataset link service to point to

2) if we tried placing source adls link service and created json dataset , but this seems like we to place output of variable to json (sink) , but source , json seems invalid

can you suggest if there is any other way , other than this

Regards

Ravi
KranthiPakala-MSFT 43,602 Reputation points Microsoft Employee

2021-07-21T20:30:51.977+00:00

Hi @Raviyaramati-1837 ,

Thanks for your response and apologies for the delayed reply.

1) where should we place or point dataset link service to point to

You will have to create a linked service pointing to the sink data store where you would want to write the sink data (csv/json based on the requirement).

2) if we tried placing source adls link service and created json dataset , but this seems like we to place output of variable to json (sink) , but source , json seems invalid - Sorry I didn't get this ask here, could you please elaborate a bit so that we can better assist on this?

Thank you
KranthiPakala-MSFT 43,602 Reputation points Microsoft Employee

2021-07-22T21:46:45.413+00:00

Hi @Raviyaramati-1837 ,

We still haven't heard back from you. Could you please confirm the above details for further assistance on this.

Thank you

how to capture output of append variable in adf and write to json or csv file in adls

1 answer

Activity