Azure Data Factory REST API Pagination Rules

Question

Azure Data Factory REST API Pagination Rules

Rafael Silveira 21

Hi there,

I've built a simple pipeline with 2 activities: One "Web" and one "Copy Data"

The web activity generates a token to access a REST API and Copy Data uses the token to copy the result to a SQL database.

I can preview the data from the Source in the "Copy data" activity, but when I run the pipeline I get this error:

Failure happened on 'Source' side. ErrorCode=UserErrorInvalidPaginationRule,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Invalid PaginationRule, RuleKey=supportRFC5988, RuleValue=true,Source=Microsoft.DataTransfer.ClientLibrary,'

This is not a multiple page response. I have no values set for Pagination Rules

I understand that "supportRFC5988" is a default added by ADF, correct?

I only find articles helping users with multiple pages, but this is not the case.

Should I suppress that default parameter passed? How would I do that?

Any help is deeply appreciated.

Thanks.

Accepted answer

0 additional answers

Your answer

Answer 1

ShaikMaheer-MSFT 38,546 Microsoft Employee Moderator

Hi @Rafael Silveira ,

Thank you for posting query in Microsoft Q&A Platform.

I tried a sample dummy API in copy activity with pagination rules. Its working fine for me.

Could you please share your copy activity source tab settings with screenshot to understand issue better and help on resolution?

Rafael Silveira 21 Reputation points

2022-01-10T15:10:52.943+00:00

Hi @ShaikMaheer-MSFT , thanks for taking the time to help me, deeply appreciated.

Let me share some screenshots: This is the pipeline:

Here are the settings for the "Web" activity, which successfully gets a token:

And here are the settings for the "Copy activity", Source tab, where it fails:

Last Friday I was able to obtain a "Preview data", but now that I tried again (new token), I'm getting the same error even for preview data:

And this is the data I'm expecting (all of it, using postman)

I'm afraid this pagination setting supportRFC5988 is not compatible the REST API I'm using?

Let me know if there's something else you need to assist me, and sorry for the "super advanced blurring technique".

Thanks once again,

Rafael
Rafael Silveira 21 Reputation points

2022-01-10T23:23:45.703+00:00

I found a solution, will post here to help others.

Inside the "Copy data" activity, the source dataset was a REST object. I replaced it with an HTTP object with the same configuration and it worked.

Thanks.
ShaikMaheer-MSFT 38,546 Reputation points Microsoft Employee Moderator

2022-01-12T16:31:36.297+00:00

Hi @Rafael Silveira - Awesome. Thank you for sharing your work around. Could you please consider making it as accepted answer. Accepted answers helps community as well.
Rafael Silveira 21 Reputation points

2022-01-12T16:43:12.61+00:00

How do I accept my own answer? I don't see the option, just for answers from someone else.
ShaikMaheer-MSFT 38,546 Reputation points Microsoft Employee Moderator

2022-01-13T16:49:58.203+00:00

Oh okay. I merged all discussion as single answer. Now could you please try if you can accept it as answer?

Share via

Azure Data Factory REST API Pagination Rules

0 additional answers

Your answer