Pagination error invalid token for Rest API Data Factory

Aditya Aranya 1 Reputation point
2021-10-24T20:40:30.74+00:00

Hi,

I am trying to use rest api to do pagination as it is just sending the first page in Azure ADF going to blob storage. I am currently using AbsoluteUrl and $['@odata.nextLink'] to get over all the pages, the issue is I am getting this error, I have used first used the token activity to get the token and then used it in copy activity where the source is rest api dataset with headers dynamically coming from token activity and then used pagination. Can you point me in the right direction on if this is the correct approach or am I missing something.

![143201-image.png]1

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,603 questions
Microsoft Partner Center API
Microsoft Partner Center API
Microsoft Partner Center: A Microsoft website for partners that provides access to product support, a partner community, and other partner services.API: A software intermediary that allows two applications to interact with each other.
316 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. svijay-MSFT 5,201 Reputation points Microsoft Employee
    2021-10-25T17:45:59.193+00:00

    Hello @Aditya Aranya ,

    Thanks for the question and using MS Q&A platform.

    If I understand correctly. the copy activity fails - when the pagination happens.

    If there is a long duration for copy activity - for first iteration to complete. There is a possibility the token getting expired before the second request.

    This is causing a 401 - authentication issue.

    Note :
    You could quickly check your token here - https://jwt.ms - decoded.

    145773-image.png

    The expiry is in epoch, and you could convert the epoch time here. See whether the error is happening after this time.