FetchXml query in ADF Copy Activity with count and page fails

Timothy Allott 1

I have the following Fetch XML in my copy activity Source query:

I get the following error when running the pipeline

Failure happened on 'Source' side. ErrorCode=DynamicsOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Dynamics operation failed with error code: DynamicsOperationFailed, error message: The Fetch Xml query specified is invalid..,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,''Type=System.Xml.XmlException,Message=Should not specify paging related arributes in your Fetch Xml query.,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,'

I have tested the FetchXml in XRM Toolbox using FetchXml Builder and it works just fine.

HimanshuSinha-msft 19,386 Reputation points Microsoft Employee

2021-08-19T19:42:29.253+00:00

Hello @,
Thanks for the ask and using the Microsoft Q&A platform .

Can you please remove the page attribute and check if this really works ? From the exception is looks to be that the error is coming from the Dynamics side and so I looked for any documents which says if this is supported by Dynamics at all .

Please do let me know how it goes .
Thanks
Himanshu
Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members
Timothy Allott 1 Reputation point

2021-08-20T05:32:24.253+00:00

Thanks for your response...

Yes it definitely works to return rows without the page and count - no error but of course only 5000 rows are ever returned due to the limitation of fetchXml.

I have not tried just count by itself.

As I have said the supplied FetchXml worked perfectly from XRM Toolbox using FetchXml Builder and therefore its more likely an issue with the ADF CDS Connector?

Thanks
Tim
HimanshuSinha-msft 19,386 Reputation points Microsoft Employee

2021-08-24T18:51:08.42+00:00

Hello @Timothy Allott ,

ADF does not allow customer setting the paging attributes because ADF will internally paging the query to retrieve results. So when you are not passing the page attribute and just say you have 10K , it should make two iteration and bring the records . Please do let me know if it is working differently .

Thanks
Himanshu
HimanshuSinha-msft 19,386 Reputation points Microsoft Employee

2021-08-27T18:12:11.547+00:00

Hello @Timothy Allott ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu
Timothy Allott 1 Reputation point

2021-08-30T00:52:45.603+00:00

Thanks Himanshu

Is this documented somewhere?

I will revert my code to remove pagination and do some record counts to see if I get the expected rows past the first 5000.

Regards
Tim
HimanshuSinha-msft 19,386 Reputation points Microsoft Employee

2021-08-31T16:58:19.097+00:00

@Timothy Allott : I diont think at this time it is documented anywhere . Let me know how it goes for you . Once I hear from you I can work with the right contact , to get the document updated .

Thanks
Himanshu
HimanshuSinha-msft 19,386 Reputation points Microsoft Employee

2021-09-03T17:40:04.657+00:00

Hello @Timothy Allott ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu
Jef Reyniers 0 Reputation points

2024-03-14T09:20:54.63+00:00

Does this also work for a Lookup activity in ADF?

Share via

FetchXml query in ADF Copy Activity with count and page fails