question

HSever-0909 avatar image
0 Votes"
HSever-0909 asked KranthiPakala-MSFT commented

Export Dynamics 365 (Dataverse) tables to csv files with Azure Data Factory

I need to run an export of 20 custom tables located in Dynamics 365/PowerApps (with Dataverse backend) to blob storage csv files. Each table to be copied 1:1 to the csv file. The total data volume for all tables is less than 2 GB. I plan to use Azure Data Factory.
1. Can I use the ADF copy functionality for the table to csv copy (please point me to couple of good links for this)?
2. Do you see limitations?

Many thanks for your input !


azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered KranthiPakala-MSFT commented

Hello @HSever-0909,

Thanks for the question and using MS Q&A platform.

Yes, you can copy data from Dynamics 365 (Microsoft Dataverse) to any supported sink data store using ADF Dynamics 365 connector in ADF copy activity.

Please note that the ADF Dynamics connector supports Dynamics versions 7 through 9 for both online and on-premises. More specifically:

  • Version 7 maps to Dynamics CRM 2015.

  • Version 8 maps to Dynamics CRM 2016 and the early version of Dynamics 365.

  • Version 9 maps to the later version of Dynamics 365.


For Dynamics 365 specifically, the following application types are supported:

  • Dynamics 365 for Sales

  • Dynamics 365 for Customer Service

  • Dynamics 365 for Field Service

  • Dynamics 365 for Project Service Automation

  • Dynamics 365 for Marketing This connector doesn't support other application types like Finance, Operations, and Talent.


Note: To copy data from Dynamics 365 Finance and Operations, you can use the Dynamics AX connector.


When you copy data from Dynamics, the following document has a table which shows mappings from Dynamics data types to interim data types within the ADF service. Configure the corresponding interim data type in a dataset structure that is based on your source Dynamics data type by using the mapping table mentioned in this document: Data type mapping for Dynamics


Limitation: Please note the Dynamics data types AttributeType.CalendarRules, AttributeType.MultiSelectPicklist, and AttributeType.PartyList aren't supported.


In case if you have a requirement to copy tables to ADLS Gen2 then instead of ADF, there is an out of box service called Azure Synapse Link for Dataverse with Azure Data Lake which can be used to export the D365 data to ADLS Gen2 directly. But it has below prerequisites and limitations:

200742-image.png

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators



image.png (43.1 KiB)
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi KranthiPakala-MSFT, thank you for your reply!

Yes, I need to run an export on a general purpose v2 storage account. I know the direct export from Dataverse to data lake exists but not interested due to limitations you mentioned.
But ADF would work for tables export from Dynamics 365 (Dataverse) to csv files on this storage account, correct? As mentioned in original post, it is a 1:1 data export , full whole table export to csv.

Thank you!!!

0 Votes 0 ·

Hi @HSever-0909,

Yes, you can use ADF to export tables from Dynamics 365 to CSV files in a storage account.

You will have to follow below documentation for setting up the connectors for your source and sink datasets.

Source - Dynamics 365 Connector
Sink - Azure Blob storage Connector

Hope this info helps. Do let us know if you have further query.



Please do consider clicking on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.


0 Votes 0 ·

Hi @HSever-0909,

Just checking in to see if the above suggestion was helpful. If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

0 Votes 0 ·

Hi KranthiPakala and all, I have seen in ADF there is both the Dynamics365 connector as you mentioned in this post, BUT ALSO a Dataverse connector. While changing both the Dynamics365 and Dataverse to use Query (instead of full table), I see there is fetchXML syntax on both. So to me these 2 connectors look like there are the same. Is there any difference between a Dynamics365 and Dataverse connector or are the same?

Thank you so much for your clarification/reply.

0 Votes 0 ·

Hello @HSever-0909, Infact both of the connectors are same and to simplify Microsoft Dataverse is a database hosted in Microsoft Dynamics 365. Hence the connectors are same but when you use them in your specific dataset then you will see the difference in configurations.

Hope this clarifies. Let me know if you have further query.

Thanks

0 Votes 0 ·