Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,858 questions
https://learn.microsoft.com/en-us/azure/data-factory/format-common-data-model
How to use files from CDM folder in adf ?
Amar Agnihotri
921
Reputation points
Hi,
In my data lake storage account there is a CDM folder adn it contains data and headers in the separate files. I want to do some transformation on those files in adf but not able to do that. Can anyone suggest how can i pull files from CDM folders in adf.
Thanks in Advance
{count} votes
-
ShaikMaheer-MSFT 38,456 Reputation points Microsoft Employee2022-07-21T17:45:38.113+00:00 Hi @Amar Agnihotri ,
Glad to know your issue resolved. Thanks for sharing details. Could you please consider marking it as
Accepted Answer? Accepted answers help community as well.
Sign in to comment
2 answers
Sort by: Most helpful
-
MarkKromer-MSFT 5,211 Reputation points Microsoft Employee2022-07-06T03:55:37.45+00:00 -
Amar Agnihotri 921 Reputation points
2022-07-06T13:59:25.773+00:00 Hi @MarkKromer-MSFT ,
Thanks for the reply. I tried that approach and created this dataflow
My source folder is containing model.json file as -
which i am using inside data flow to get the schema
This is my sink flowlet in data flow and i want to save the file as .csv in the datalake
In the data preview of dataflow all the columns and data is showing
But when i am calling this dataflow inside adf pipeline and trying to save the output in .csv file not headers are showing there. Can you please help me out getting headers in the output file. I am not able to understand from where can i save those headers with the output data in .csv file.Thanks
-
Amar Agnihotri 921 Reputation points
2022-07-07T06:46:45.847+00:00 hi @MarkKromer-MSFT ,
The issue is resolved. Actually, i forgot to select First row as header option in sink dataset . Thanks for your support. -
ShaikMaheer-MSFT 38,456 Reputation points Microsoft Employee
2022-08-01T09:59:51.327+00:00 Hi @Amar Agnihotri - Glad to know issue resolved. Could you please consider marking it as
Accepted Answer? Accepted answers help community as well. -
Amar Agnihotri 921 Reputation points
2022-08-01T13:48:45.573+00:00 hi @ShaikMaheer-MSFT
The proper answer is given in the link that i shared in my answer but i can't select my answer as "Accepted Answer". Please suggest what should i do now?
Sign in to comment -
-
Amar Agnihotri 921 Reputation points
2022-07-07T06:48:22.477+00:00 Here is the proper solution
https://www.linkedin.com/pulse/working-common-data-mo-azure-factory-morten-hannibalsen-olsen/
-
MarkKromer-MSFT 5,211 Reputation points Microsoft Employee
2022-07-21T20:15:06.853+00:00 Thank you for sharing this with the community !!
Sign in to comment -