How to extend entities of Common Data Model in Azure data lake

Shafqaat Ahmad 0

Hi,

We have CDM (common data model folders) in Azure Data lake. Is there any easy way to extend the entities so that they can meet our business requirements? Do we have to extend them manually, like adding attributes, adding tables?

Regards

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Shafqaat Ahmad 0 Reputation points

2023-06-08T15:04:32.8266667+00:00

Thank you very much. I wonder I got same answer from ChatGPT.
BhargavaGunnam-MSFT 26,496 Reputation points Microsoft Employee

2023-06-08T19:31:45.6466667+00:00

Hello Shafqaat Ahmad,

Using Spark common data model connector for synapse, you can read and write CDM entities in a CDM folder on ADLS using Spark data frame. This allows us to extend the entities programmatically rather than manually.

To extend the entities, you can read the data from an entity in a CDM folder into a Spark dataframe, modify the data frame schema as needed (by adding attributes or tables), and then write the modified data frame back to an entity in the CDM folder.

I hope this helps.
Shafqaat Ahmad 0 Reputation points

2023-06-08T19:53:09.2166667+00:00

Thank you, but we are not using spark. Is there anyway to extend entities or create attributes in ADLS gen 2 without using spark?
BhargavaGunnam-MSFT 26,496 Reputation points Microsoft Employee

2023-06-12T17:39:39.3566667+00:00

Hello Shafqaat Ahmad,

I am checking to see if the above response is helpful. Please let me know if you have any further questions.

1 answer

BhargavaGunnam-MSFT 26,496 Reputation points Microsoft Employee

2023-06-10T00:21:56.7333333+00:00

Hello Shafqaat Ahmad,

The other option is by using CDM SDK to programmatically create and manage CDM folders, entities, and attributes.

https://learn.microsoft.com/en-us/common-data-model/sdk/from-modeljson-to-the-future#common-data-model-sdk

https://community.dynamics.com/365/mbas/f/microsoft-business-applications-summit-gallery-2020/389466/advanced-cdm-learn-the-benefits-of-cdm-sdk

If you can use serverless Apache Spark pool in Azure Synapse Analytics, then using FSSPEC library to read and write data to ADLS gen2 using a linked service

https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/tutorial-spark-pool-filesystem-spec

Here is a video tutorial

I hope this helps.
Please sign in to rate this answer.

2 people found this answer helpful.
Shafqaat Ahmad 0 Reputation points

2023-06-12T17:42:07.1766667+00:00

Hi Thank you BhargavaGunnam, yes your answer is helpful, thank you very much

BhargavaGunnam-MSFT 26,496 Reputation points Microsoft Employee

2023-06-12T19:00:53.09+00:00

Hello Shafqaat Ahmad,

Thank you. In case if you don't have any further questions, please consider accepting the answer by hitting the Accept answer and up-vote as it helps the community look for answers to similar questions.

QuantumCache 20,031 Reputation points

2023-06-13T18:01:45.4766667+00:00

Hello @Shafqaat Ahmad, how are you doing!

This is in continuation of the follow up earlier!

@BhargavaGunnam-MSFT

If the response is helpful, please click "Accept Answer" and upvote it. So that we can close this thread.

Happy to help you further on this matter!

@BhargavaGunnam-MSFT

Azure CXP.
Sign in to comment