Azure pipeline failure while coping data to parquet from SFTP

Ashu Anand 31 Reputation points
2022-12-30T09:58:23.167+00:00

I am facing an issue while copying data from a SFTP foder to parquet file while using ADF pipelines and SHIR. The same integration runtime is running perfectly fine while copying data from on-premise SQL database. The issue is coming while copying from a SFTP source.

Error Message : ErrorCode=JniException,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An error occurred when invoking Java Native Interface.,Source=Microsoft.DataTransfer.Common,''Type=JNI.JniReturnException,Message=Cannot create JVM: JNI return code [-1][JNI call failed due to unknown JNI error.],Source=Microsoft.DataTransfer.Richfile.HiveOrcBridge,'

Any idea what the issue could be?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,269 questions
{count} votes

Accepted answer
  1. AnnuKumari-MSFT 19,116 Reputation points Microsoft Employee
    2023-01-02T05:11:20.053+00:00

    Hi @Ashu Anand ,

    Welcome to Microsoft Q&A platform and thanks for posting your question here.

    As I understand your issue, you are facing JNIException issue while copying data from SFTP to ADLS as parquet file. Please let me know if my understanding is incorrect.

    If the error message contains "Cannot create JVM" , the possible cause is that JVM can't be created because some illegal (global) arguments are set.

    Log in to the machine that hosts each node of your self-hosted integration runtime. Check to ensure that the system variable is set correctly, as follows: _JAVA_OPTIONS "-Xms256m -Xmx16g" with memory bigger than 8G. Restart all the integration runtime nodes, and then rerun the pipeline.

    Source : Troubleshoot guide for Error code: JniException

    275219-image.png

    Additional resource for troubleshoot: Using Self-hosted Integration Runtime

    Hope this will help. Please let us know if any further queries.


    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
      Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

0 additional answers

Sort by: Most helpful