Share via

Rest API post request file creation

Priya Jha 871 Reputation points
2022-10-11T09:56:07.153+00:00

Hi All,

Currently i am leveraging Rest API as a source and blob as a sink in Azure Data Factory. The Rest API output is triggered on post request

249283-microsoftteams-image-2.png

Preview Data:

249267-microsoftteams-image-3.png
249343-microsoftteams-image-4.png

Now when i am importing the mappings and executing the job only 1st row is getting copied in the file.

249279-image.png

How do i ensure that all the rows of the rest API get coped into the file.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,198 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Nandan Hegde 31,511 Reputation points MVP
    2022-10-11T10:55:41.993+00:00

    Based on the output, it seems to be Object type.
    Not sure whether the approach is correct but you can map it to array based on something like below :

    activity.output.results[0].tables[0].rows
    and then use that to load into file

    @AnnuKumari-MSFT @ShaikMaheer-MSFT : any thoughts ? As this might be a haphazard way :(

    0 comments
  2. AnnuKumari-MSFT 32,161 Reputation points Microsoft Employee
    2022-10-12T08:29:50.033+00:00

    Hi @Priya Jha ,

    Thankyou for using Microsoft Q&A platform and thanks for posting your question here.

    As per my understanding, you are trying to copy data from REST API to ADLS file. However, the results are not coming as expected. Kindly let me know if my understanding is incorrect.

    Since the source data is of hierarchical type. You need to use 'row[]' array as collection reference in copy activity , based on which iterations can be made and data can be dynamically fetched.

    For more details, kindly check the below resources:
    Hierarchical source to tabular sink
    How to load Tabular and Hierarchical JSON data from Rest API to SQL Table

    If this doesn't help in your case, you need to go for Mapping data flow where we have the capability to use Flatten transformation that can easily perform flatten over complex data.

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
      Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators