Converting csv to parquet format with correct datatypes

Question

Hi,

I am facing issue while converting the .csv file to parquet files.

As per my requirement, I've converted the parquet to .csv format and added two new columns of string datatype. Now I'm trying to convert this .csv file back to parquet format with original parquet file datatypes using mapping data flows but the datatype conversion in not happening. All the column datatypes are shown as string only.
Is there any workaround to handle this scenario so that my final parquet file shows up the correct data types?

Answer 1

Hello @Madugundu Somashekara, R (Roopa) ,

Thanks for the question and using MS Q&A platform.

As @MarkKromer-MSFT called out, in your source transformation, under Projection you will see Detect data type button which can be used to auto detect the datatypes from your delimited text files. In case if you want to manually update it to some other data type other than autodetected, you will have a drop-down menu to choose from. In addition, you can also specify a format which is optional.

For more clarification, please see below GIF:

To explore about these features, please refer to this product documentation: ADF Mapping Dataflow - Source Transformation - Projection Tab

Here is a related helpful video demonstration by @MarkKromer-MSFT for shaping large dataset projections - Shape large dataset projections w/ADF Select transformation

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators