Issue with API pagination using a Data Flow inside Azure Data Factory

Question

I'm trying to implement pagination rules on multiple api calls following the documentation provided by Data Factory.
By implementing it on a simple pipeline with a copy data node it works.

But when trying the same configuration on a data flow that has other transformation after getting the source data, Fetching the source data fails with the error :
'{"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: at Source 'GetOpportunities': java.net.URISyntaxException: Illegal character in query at index 78:...'

The source is configured as :

Pagination configuration :

Tried to do it with QueryParameters using relative url but it is not an option to choose from like in a copy node :

Answer 1

Hi @Ismail Akrim ,

Thanks for posting query in Microsoft Q&A Platform.

This kind of error usually will come when Issue with URL encoding and decoding for special characters. Kindly try using URL encoding and decoding using replace function as explains in below post and see if that helps?
https://stackoverflow.com/questions/65950642/java-net-urisyntaxexception-illegal-character-in-query-at-index-177

Please let us know how it goes. Thank you.