Can't make pagination work in Copy Activity

KoA 0 Reputation points
2023-11-17T17:32:40.59+00:00

I'm using confluence rest api in Copy activity and used AbsoluteURL.

My base url is defined as https://loremipsum.atlassian.net/wiki/

and I'm getting next page from the $._links.next but for some reason when ADF is creating the request it is dropping wiki from the base url.

How to make this work?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,170 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Amira Bedhiafi 19,946 Reputation points
    2023-11-18T10:39:45.13+00:00

    It is either your AbsoluteUrl needs to be checked.

    Or in your pagination settings, you need to specify how ADF should retrieve the next page of data. Since you're using $._links.next to get the next page URL, make sure that this is set up correctly in the pagination rules of the Copy Activity.

    Use the expression builder in ADF to concatenate the base URL with the relative URL if necessary. If $._links.next returns a relative URL, you may need to explicitly concatenate the base URL and the relative URL in the pagination rule.

    https://stackoverflow.com/questions/67885066/azure-data-factory-pagination-doesnt-work

    https://www.youtube.com/watch?v=EbURwy2a8dM

    0 comments No comments