RESP API Pagination rules in ADF

Ashutosh Yadav 1 Reputation point
2022-03-15T10:29:17.15+00:00

Hi All,

I have load all pages from rest api through adf but I am getting only record of only page that is 1000 record.
pagination rule not working please help some one.

183190-image.png

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,726 questions
Microsoft Partner Center API
Microsoft Partner Center API
Microsoft Partner Center: A Microsoft website for partners that provides access to product support, a partner community, and other partner services.API: A software intermediary that allows two applications to interact with each other.
317 questions
{count} votes

1 answer

Sort by: Most helpful
  1. ShaikMaheer-MSFT 38,126 Reputation points Microsoft Employee
    2022-03-17T13:15:29.213+00:00

    Hi @Ashutosh Yadav ,

    Thank you for posting query in Microsoft Q&A Platform.

    From my understanding you are trying to use pagination in copy activity but it's copying only one page. Kindly correct me if I am wrong.

    AbsoluteUrl key can be used in pagination rules when URL to the next page request comes as part of your previous page response.

    If your next page URL is as part of previous response then use $ syntax. $ indicates root of JSON.

    For example if below is our last response then kindly use pagination as "AbsoluteUrl": "$.paging.next"

    {  
        "data": [],  
        "paging": {  
            "cursors": {  
                "after": "MTAxNTExOTQ1MjAwNzI5NDE=",  
                "before": "NDMyNzQyODI3OTQw"  
            },  
            "previous": "https://graph.facebook.com/me/albums?limit=25&before=NDMyNzQyODI3OTQw",  
            "next": "Same with Last Request URL"  
        }  
    }  
    

    Kindly check MS documentation about pagination and different examples with detailed steps.

    Hope this helps. Please let us know how it goes and if any further queries. Thank you.

    ----------------

    Please consider hitting Accept Answer button. Accepted answers helps community as well.

    0 comments No comments