Pagination in Azure ADF

Venkat 6 Reputation points

I am running into issues not being able to fetch data from pages beyond 1.

The REST API returns data like this.

I have setup the Mappings section as follows:


I have setup the Pagination as follows:

This does not work and I do not get beyond the 1st page. There are 7 pages of data to be iterated through.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,473 questions
{count} votes

2 answers

Sort by: Most helpful
  1. Venkat 6 Reputation points

    Figured it out. The AbsoluteUrl needed some tweak.

    1 person found this answer helpful.

  2. MarkKromer-MSFT 5,186 Reputation points Microsoft Employee

    We have to update the REST API docs for this, which we are working on currently.

    Basically, the rules in Copy can support both "." and [''], e.g., AbsoluteUrl: $.{@odata.nextLink} and AbsoluteUrl: $['@odata.nextLink']

    However, rules in dataflow only support "." for json path. e.g., AbsoluteUrl: $.{@odata.nextLink}

    0 comments No comments