How to copy files from SSMS using SHIR to DataLAke Gen 2 in parquet format?

Question

How to copy files from SSMS using SHIR to DataLAke Gen 2 in parquet format?

nidhi staning 0

I am trying to copy csv files from on-prem using SSMS. I created linked services and set up the SHIR to get the data from on-prem. I created the source as SQL SERVER and then I created sink as Data Lake storage Gen 2 and format as Parquet.

When I run the pipeline, I am getting this error. But when I make the sink format as csv, the pipeline runs. Please anybody can explain and help me with this? Thanks

Error details

Copy address table

Error codeUnknown

Failure typeUser configuration issue

DetailsErrorCode=Unknown,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An unknown error occurred.,Source=Microsoft.DataTransfer.Common,''Type=JNI.JavaExceptionCheckException,Message=Exception of type 'JNI.JavaExceptionCheckException' was thrown.,Source=Microsoft.DataTransfer.Richfile.HiveOrcBridge,'

Activity ID3a46d433-a6b4-4d02-a17b-14bdfb79e614 User's image

Smaran Thoomu 25,175 Reputation points Microsoft External Staff Moderator

2023-11-24T07:37:20.1866667+00:00

@nidhi staning We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

1 answer

Your answer

Smaran Thoomu 25,175 Reputation points Microsoft External Staff Moderator

2023-11-24T07:37:20.1866667+00:00

@nidhi staning We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Answer 1

Hi @nidhi staning

Welcome to Microsoft Q&A and thanks for reaching out.

The error message you received indicates that there was an issue with the pipeline when trying to copy data from SQL Server to Data Lake Storage Gen2 in Parquet format. However, the pipeline runs successfully when the sink format is set to CSV.

One possible reason for this issue could be that the Parquet format is not supported by the SHIR (Self-hosted Integration Runtime) version you are using. Please check the version of SHIR you are using and ensure that it supports Parquet format.

Another possible reason could be that the Parquet format is not compatible with the data you are trying to copy. Please ensure that the data you are trying to copy is compatible with the Parquet format.

To troubleshoot this issue, you can check the following:

Check the SHIR version you are using and ensure that it supports Parquet format.
Check the compatibility of the data you are trying to copy with the Parquet format.
Check the pipeline settings and ensure that they are configured correctly.

You can also refer: How to copy data to and from Azure Data Lake Storage Gen2 using Azure Data Factory

I hope this helps! Let me know if you have any further questions.

Smaran Thoomu 25,175 Reputation points Microsoft External Staff Moderator

2023-11-27T17:27:22.37+00:00

nidhi staning Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

How to copy files from SSMS using SHIR to DataLAke Gen 2 in parquet format?

1 answer

Your answer