Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
How to copy all and update API data
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 57.00000000000001%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Raphael Koning
21
Reputation points
I'm trying to copy data from an API to blob storage but because of performence reasons it is not possible to return all the data at once. It only gives 10 elements and it is possible to get other pages using the 'page' parameter. In the first screenshot you can see the data it returns when copying 'meters'. Column _embedded row meters contains the 10 rows. How is it possible to copy and afterwards update all the data to blob storage and maybe combine different dataframes in the process(just copying it all at once every few minutes might not be possible because one of the data sources contains 10252096 elements)? I'm new to azure so I'm having some difficulties with the basics like adding parameters(like 'page') and that kind of stuff. I also put some screenshots of how I copy the data to make it a bit more clear what i did.
Feel free to ask me some more information if anything is unclear!
Thanks :)
Azure Synapse Analytics
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
HimanshuSinha 19,527 Reputation points Microsoft Employee Moderator2021-03-17T20:51:24.757+00:00 Hello @Raphael Koning ,
Thanks for the ask and using the Microsoft Q&A platform .
As I understand you have the data in the blob in many different JSON and I understand the intent is to aggregare all the JSON in one as if you were response of a single request .
I am not sure if you have already read this . I think this read , can help youPlease do let me know how it goes .
Thanks
Himanshu -
HimanshuSinha 19,527 Reputation points Microsoft Employee Moderator
2021-03-24T20:32:02.83+00:00 Hello @Raphael Koning ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu
Sign in to comment
Sign in to answer