Azure Data Fatory Copy activity not passing pagination parameter correctly to api source

Question

Azure Data Fatory Copy activity not passing pagination parameter correctly to api source

Rolando Chavez 5

Data provider has an api that requires passing the page number as a url parameter. We are using the copy action with an api source.

The api source has this as a relative url:

User's image Where I tried to follow the documentation example 1 on adding a pagination parameter using brackets, in this case the parameter is "{page}".

Then in the copy activity I configured the pagination as following:

User's image

I've tried also with the absolutUrl instead of queryParameter option. When running on debug, it runs successfully, but it seems to run with page=1, 31 times. I can verify this in the output, where it says the 31 lines of results are all for the page 1:

User's image

2 answers

Your answer

Answer 1

Rolando Chavez 5

I have found a solution to my problem. If I change the name of the variable to "offset" it runs correctly. Note that this is without the brackets that are used in the documentation example. As if I leave it as "{offset}" it tries to call the api as "{1}". I have no idea why it substitutes "{page}" to "1" in the setup used in the question, nor why it substitutes "{offset}" to "{1}".

At the end, this is how my source looked:

User's image

And this is how the copy activity source tab looked:

User's image

Answer 2

Deleted

This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

4 deleted comments

Comments have been turned off. Learn more

Share via

Azure Data Fatory Copy activity not passing pagination parameter correctly to api source

2 answers

Your answer