Share via

Additional column of type `Datetime` when using Copy Activity to write to Parquet

Martin B 21 Reputation points
2023-05-05T18:15:26.5333333+00:00

I'm reading data from a DB2 database using ODBC and Copy Activity and writing it to a parquet file.

I'd like to add an additional column with the UTC timestamp of the execution:User's image

This basically works, but I found that the data type in the resulting parquet file column is "String" and not "Datetime".

I also tried explicitly defining a Mapping, but this has no effect either. The colums is still "String" and not "Datetime" in the parquet file.

User's image

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,696 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,199 questions
0 comments
Accepted answer
  1. Suba Balaji 11,206 Reputation points
    2023-05-06T12:06:17.6333333+00:00

    Hello Martin B

    Thanks for using MS Q&A Portal for posting your query

    For additional columns, we wont be able to change the datatype of the column in copy activity. By default it will be string only.

    If you want to change the data type, we should only use Derived column in data flow.

    Please let us know if that answers your question!

    BR

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. KranthiPakala-MSFT 46,442 Reputation points Microsoft Employee
    2023-05-09T04:12:17.4533333+00:00

    Hi @Martin B - Thanks for using MS Q&A forum and posting your query here.

    As @Subashri Vasudevan called out the data type of additional column will be of type String . I agree that it would good to improve the feature by providing the flexibility for users to select the preferred data type while configuring the additional column. Please feel free to log a feature request item for the same in IDEAS forum here: https://feedback.azure.com/d365community/forum/1219ec2d-6c26-ec11-b6e6-000d3a4f032c All the feedback shared in this forum are actively monitored and reviewed by respective product owners. Kindly share the feedback link once it is posted so that I can share it with respective owners to further review it.

    Regarding the documentation note, I don't think it is mentioned in the public documentation. I will work with respective document owners to have a note about the data type of additional column.

    Thank you @Subashri Vasdudevan for sharing your inputs and looping us for confirmation. We will work with document owners to have a note about the same.

    Hope this info help. Let me know if you have any further questions.

    0 comments