Using Change Tracking token with Azure Data Factory on Source for Incementals - how can I do this ?

Question

Hi all.  I've used CozyRoc for years in SSIS to pull data from D365 using the Change Tracking token to just pull changes since last run, but now I need to start updating some of these processes to use Azure Data Factory.  How can I use the Change Tracking token in the ADF Source to D365 online to pull incrementals? I've played with connecting to the service directly using odata but this is so much data I'm not sure how this works with paging through the data.  And the Dynamics 365 nor the Dataverse connector I've found anyway to use a change tracking token.

 

Any thoughts?  Thanks.

Answer 1

Hi there Alexander, Sam

Thanks for using QandA platform

Try enabling Change Tracking on the entities in D365. the ADF’s connector doesn’t directly support Change Tracking tokens, you can use the OData connector to interact with the Change Tracking API. First, use the /GetChanges endpoint to retrieve a deltaLink token. Store this token.

You’ll also need to manage large datasets by configuring pagination using OData parameters like $top and $skip, which ADF can handle efficiently.

https://learn.microsoft.com/en-us/azure/data-factory/connector-odata?tabs=data-factory

If this helps kindly accept the answer thanks much.