Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,432 questions
FetchXml query in ADF Copy Activity with count and page fails
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 8%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anonymous
I have the following Fetch XML in my copy activity Source query:
<fetch count="5000" page="39" >
<entity name="contact" >
<all-attributes/>
<filter>
<condition attribute="modifiedon" operator="gt" value="2021-08-16 9:45:33 PM" />
</filter>
<order attribute="contactid" />
</entity>
</fetch>
I get the following error when running the pipeline
Failure happened on 'Source' side. ErrorCode=DynamicsOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Dynamics operation failed with error code: DynamicsOperationFailed, error message: The Fetch Xml query specified is invalid..,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,''Type=System.Xml.XmlException,Message=Should not specify paging related arributes in your Fetch Xml query.,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,'
I have tested the FetchXml in XRM Toolbox using FetchXml Builder and it works just fine.
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft 19,481 Reputation points Microsoft Employee2021-08-19T19:42:29.253+00:00 Hello @,
Thanks for the ask and using the Microsoft Q&A platform .Can you please remove the page attribute and check if this really works ? From the exception is looks to be that the error is coming from the Dynamics side and so I looked for any documents which says if this is supported by Dynamics at all .
Please do let me know how it goes .
Thanks
Himanshu
Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members -
Anonymous
2021-08-20T05:32:24.253+00:00 Thanks for your response...
Yes it definitely works to return rows without the page and count - no error but of course only 5000 rows are ever returned due to the limitation of fetchXml.
I have not tried just count by itself.
As I have said the supplied FetchXml worked perfectly from XRM Toolbox using FetchXml Builder and therefore its more likely an issue with the ADF CDS Connector?
Thanks
Tim -
HimanshuSinha-msft 19,481 Reputation points Microsoft Employee
2021-08-24T18:51:08.42+00:00 Hello @Anonymous ,
ADF does not allow customer setting the paging attributes because ADF will internally paging the query to retrieve results. So when you are not passing the page attribute and just say you have 10K , it should make two iteration and bring the records . Please do let me know if it is working differently .
Thanks
Himanshu -
HimanshuSinha-msft 19,481 Reputation points Microsoft Employee
2021-08-27T18:12:11.547+00:00 Hello @Anonymous ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu -
Anonymous
2021-08-30T00:52:45.603+00:00 Thanks Himanshu
Is this documented somewhere?
I will revert my code to remove pagination and do some record counts to see if I get the expected rows past the first 5000.
Regards
Tim -
HimanshuSinha-msft 19,481 Reputation points Microsoft Employee
2021-08-31T16:58:19.097+00:00 @Anonymous : I diont think at this time it is documented anywhere . Let me know how it goes for you . Once I hear from you I can work with the right contact , to get the document updated .
Thanks
Himanshu -
HimanshuSinha-msft 19,481 Reputation points Microsoft Employee
2021-09-03T17:40:04.657+00:00 Hello @Anonymous ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu -
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJR%3C/text%3E%3C/svg%3E)
Jef Reyniers 0 Reputation points2024-03-14T09:20:54.63+00:00 Does this also work for a Lookup activity in ADF?
Sign in to comment
Sign in to answer