I am facing same issue. Could you pls let me know how did you solve it.
Thanks in advance
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Hello Team,
I am getting an error when copying parquet type file to Azure synapse using polybase option. I am getting the below error.
Code": 21001, "Message": "ErrorCode=ParquetInvalidFile,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=File is not a valid parquet file.,Source=Microsoft.DataTransfer.Richfile.ParquetTransferPlugin,'", "EventType": 0,
When i tried for the same parquet file using BulkInsert option, it worked.
Any Ideas?
Thank You
I am facing same issue. Could you pls let me know how did you solve it.
Thanks in advance
@HimanshuSinha-msft
Hi , But with polybase selected option, if we write "*.parquet" , then we got error that polybase only support . or * wildcard path.
How we can resolve this issue ? I am still getting same issues and not using any complex types . All columns are string types and writing the parquet file through databricks.
Hello @Parthivelu ,
Thanks for question and also for using this new forum .
I did tried to do the same using the wildcard option and it worked for me , please do try this out .
I did had 3 paraquet files and by doing this I was able to insert into the SQL table .
Thanks Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members
Hello @Parthivelu ,
Thanks for question and also for using this new forum .
I am not sure if you have gone through this link , there are some limitation of copy activity when handling paraquet file .
https://learn.microsoft.com/en-us/azure/data-factory/format-parquet#data-type-support . May be you have this issue .
"Parquet complex data types (e.g. MAP, LIST, STRUCT) are currently supported only in Data Flows, not in Copy Activity"
Thanks Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members