Ingest exported Dataverse data with Azure Data Factory with Option Set Metadata issues

Question

Dear All,
We are doing export the data from dataverse to azure data lake with help of azure synapse link. we can do the export data and sync to data lake.
after that we are trying to do the data ingest and transformation using azure data factory with help of data flow.

now i can do the transformation all the tables with help of model.json to generate the schema header but if we want to do the OptionSet MetaData folder. it is not accessible and not able to do the any transformation.

I have followed below microsoft url and i can see the normal tables like account..etc tranformation using inline common data model.

how do we do the similar ADF transformation in OptionSetMetaData. kindly advise and here i have attached links and images.

https://learn.microsoft.com/en-us/power-apps/maker/data-platform/export-to-data-lake-data-adf?source=docs

Thanks

Answer 1

Hello @Karthik Elavan ,

Welcome to the MS Q&A platform.

When you select model.json, you are not able to do any transformations on your files in the folder 'OptionSetMetaData'.

Did you try with manifest option? If not, Can you point to the manifest and subsequent entity that you wanted to extract data and schema?

Answer 2

We are stuck on the same issue , can you please guide to extract the csv file and link to the entity table. When I tried to create external table , i got an error saying you cannot create external table in the replicated lakehouse.
Without linking entity to the metadata csv file we are stuck with just code in the columns and not the actual value related to the code. please help