reading .parquet column in correct datatype

arkiboys 9,706 Reputation points
2021-09-15T07:48:27.64+00:00

Hi,
In the source tab of the copy activity, I add a new column to show the date, i.e.
CurrentDate --> utcnow()

Then in the parquet file, when I read it using the openrowset query, I put WITH for the column name, etc...
I put the datatype as datetime but it errors.
If I use varchar(50), the view does not error.
Any suggestions?
Thank you

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
0 comments No comments
{count} votes

Answer accepted by question author
  1. ShaikMaheer-MSFT 38,631 Reputation points Microsoft Employee Moderator
    2021-09-16T14:27:08.19+00:00

    Hi @arkiboys ,

    Thank you for posting query on Microsoft Q&A Platform.

    utcnow() function returns current timestamp in string format. Hence your parquet file will be having datatypes as UTF8 for CurrentDate column.

    Please check below screenshot of parquet file which shows current_date column type as UTF8.

    132718-image.png

    According to Type mappings for parquet in Synapse Analytics for binary UTF8 supported sql data type is varchar only. Hence in your case when you are using varchar its working fine.
    Please check below screenshot for same. Click here to know more about Type mappings for parquet in Synapse Analytics.

    132794-image.png

    Hope this will help. Please let us know if any further queries.


    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    0 comments No comments

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.