Pagination in Azure data Factory(By REST API) using nextpagetoken

teja gudipati 21 Reputation points

Hi Team,
Can someone help me with it?
I am trying to pull data through Rest API into an azure data lake,
In the copy activity, the API response I am getting has the next page token column, I am using this column in the Pagination rules, Just like below.
But the API is not fetching data from the next pages.
I don't know what I am doing wrong here


Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,444 questions
{count} votes

2 answers

Sort by: Most helpful
  1. MartinJaffer-MSFT 26,011 Reputation points

    I think I see where the confusion is coming from now, @teja gudipati .

    So the token comes in the body of the response, and needs to be placed into the header of the next request, if I understand correctly.
    In this case, "AbsoluteURL" is not what you want to use.

    The Key should be Headers['header_name_for_next_page_token'] , renames as appropriate.
    The Value should be $.nextPageToken

    The Key specifies where to insert pagination in the next request.
    The Value specifies where to get the pagination info from the response.

  2. MartinJaffer-MSFT 26,011 Reputation points

    @teja gudipati
    For when the next page token goes into the URL, you need to use

    So for


    it would be


    This inserts nextPageToken into the URL query parameters.