Pagination Issue With Rest Request

Nicholas Mead 6 Reputation points

I'm struggling to get pagination to work for a rest request in data factory. Here's a preview of the data that gets returned by the rest call;


For my settings, I've applied 'AbsoluteURL', with Value 'Body' and the query $

I've tried various other tweaks, but always get the same result... only 50 records return. What am I missing?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,092 questions
0 comments No comments
{count} vote

2 answers

Sort by: Most helpful
  1. Evgeny Grishchenko 486 Reputation points

    $.paging… does not work, because the response is json array [], and not object {}.

    "The response body should contain only one JSON object. The JSONPath expression should return a single primitive value, which will be used to issue next request."

    0 comments No comments

  2. ShaikMaheer-MSFT 38,326 Reputation points Microsoft Employee

    Hi @Nicholas Mead ,

    Thank you for posting query in Microsoft Q&A Platform.

    In your case it seems you are getting array and inside array its json object. Hence you may seeing it not working.

    If we see official documentation example from below link, here API response was in json object.

    Still, kindly try using below syntax and see if that helps.
    $.[0] or $[0]

    If not helps, please try to make that API call using web activity and try to take the next URL in to some variable and then frame new URL and make again API call. That way it will work. If possible please also share the output json screenshot of API call from web activity to understand the API response better here.

    Kindly consider checking below video, there I explained a different way of implementing pagination and see that fits in your case.
    Copy data from REST API which sends response in Pages using Azure data factory

    Check this video for understanding web activity.

    Hope this helps. Please let me know how it goes.


    Please consider hitting Accept Answer button. Accepted answers help community as well.