Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
https://learn.microsoft.com/en-us/azure/data-factory/format-common-data-model
How to use files from CDM folder in adf ?
Amar Agnihotri
926
Reputation points
Hi,
In my data lake storage account there is a CDM folder adn it contains data and headers in the separate files. I want to do some transformation on those files in adf but not able to do that. Can anyone suggest how can i pull files from CDM folders in adf.
Thanks in Advance
Azure Data Factory
{count} votes
-
ShaikMaheer-MSFT 38,631 Reputation points Microsoft Employee Moderator2022-07-21T17:45:38.113+00:00 Hi @Amar Agnihotri ,
Glad to know your issue resolved. Thanks for sharing details. Could you please consider marking it as
Accepted Answer? Accepted answers help community as well.
Sign in to comment
2 answers
Sort by: Most helpful
-
MarkKromer-MSFT 5,231 Reputation points Microsoft Employee Moderator2022-07-06T03:55:37.45+00:00 -
Amar Agnihotri 926 Reputation points
2022-07-06T13:59:25.773+00:00 Hi @MarkKromer-MSFT ,
Thanks for the reply. I tried that approach and created this dataflow
My source folder is containing model.json file as -
which i am using inside data flow to get the schema
This is my sink flowlet in data flow and i want to save the file as .csv in the datalake
In the data preview of dataflow all the columns and data is showing
But when i am calling this dataflow inside adf pipeline and trying to save the output in .csv file not headers are showing there. Can you please help me out getting headers in the output file. I am not able to understand from where can i save those headers with the output data in .csv file.Thanks
-
Amar Agnihotri 926 Reputation points
2022-07-07T06:46:45.847+00:00 hi @MarkKromer-MSFT ,
The issue is resolved. Actually, i forgot to select First row as header option in sink dataset . Thanks for your support. -
ShaikMaheer-MSFT 38,631 Reputation points Microsoft Employee Moderator
2022-08-01T09:59:51.327+00:00 Hi @Amar Agnihotri - Glad to know issue resolved. Could you please consider marking it as
Accepted Answer? Accepted answers help community as well. -
Amar Agnihotri 926 Reputation points
2022-08-01T13:48:45.573+00:00 hi @ShaikMaheer-MSFT
The proper answer is given in the link that i shared in my answer but i can't select my answer as "Accepted Answer". Please suggest what should i do now?
Sign in to comment -
-
Amar Agnihotri 926 Reputation points
2022-07-07T06:48:22.477+00:00 Here is the proper solution
https://www.linkedin.com/pulse/working-common-data-mo-azure-factory-morten-hannibalsen-olsen/
-
MarkKromer-MSFT 5,231 Reputation points Microsoft Employee Moderator
2022-07-21T20:15:06.853+00:00 Thank you for sharing this with the community !!
Sign in to comment -