FetchXML drops columns, how else can I work with MS CRM Dataverse?

Christian Loveridge 0 Reputation points
2023-11-30T20:21:04.5033333+00:00

I am trying to copy data from MS Dynamics CRM to an Azure SQL Database, but the table is large, resulting an a very expensive and slow Copy Data upsert. I switched the source from "Entity Name" to "Query" and wrote a FetchXML query to pull the data, but Data Factory's FetchXML drops columns if the first 10 records are all null.

User's image

Is there a way to get around this idiocy artificial limitation and pull data from CRM as-is? Another language or tool to use instead? A specific command or keyword in FetchXML?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,150 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Amira Bedhiafi 19,781 Reputation points
    2023-12-01T15:48:40.73+00:00

    Your request is not that clear, but based on what you provided so far, instead of relying on FetchXML to automatically include all columns, explicitly state each column you need in your FetchXML query.Dynamics CRM supports OData, which is a more flexible data access protocol and can be used to retrieve data.OData queries can be more detailed and less restrictive compared to FetchXML.

    The Dynamics CRM SDK provides a rich set of APIs to interact with CRM data. You can write a custom application or script using the SDK to extract the data you need. This allows for more control and can bypass some of the limitations of FetchXML.

    0 comments No comments