TIMESTAMP_MICROS in Parquet file is not supported in ADF

Question

I am receiving a parquet file from customer with one of the columns has timestamp_micros value(Ex: 1640997340000000). When creating a linked service and inferSchema of Parquet file, it shows error as "Type of column 'checkuptime' is not supported" and not allowing to perform field mappings. I am use Copy task, trying to move data from Parquet to Csv file format, to use that data in compute engine further. Could anyone help what is wrong and why Timestamp_Micros type is not supported in ADF?

Answer 1

Hello @Saravanan Ponnaiah ,

Thanks for the question and using MS Q&A platform.

As per my understanding your source schema has TIMESTAMP_MICROS and you are receiving above error while trying to import schema in copy activity. Please correct if I'm not accurate.

As per my analysis, this is a limitation with Copy activity where TIMESTAMP_MICROS isn't supported. Hence as a workaround, product team recommends using Mapping Dataflow. Mapping dataflow supports it.

Here is an existing feature request logged by another user in IDEAS forum, please feel free to up-vote and add your comments to help increase the priority of the feature request: Support "TIMESTAMP_MICROS" for Parquet file to be compatible for datetime in Copy Activity

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

Answer 2

Thanks @KranthiPakala-MSFT for the clarification.