JNIException during copy activity

Question

Hello,

I encountered the follow error in Copy activity regarding JNIException:

1. Operation on target ArchiveFile failed: ErrorCode=JniException,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An error occurred when invoking Java Native Interface.,Source=Microsoft.DataTransfer.Richfile.ParquetTransferPlugin,''Type=JNI.JniReturnException,Message=Cannot create JVM: JNI return code [-1][JNI call failed due to unknown JNI error.],Source=Microsoft.DataTransfer.Richfile.HiveOrcBridge,'
2. Operation on target Source to ADLS failed: ErrorCode=JniException,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An error occurred when invoking Java Native Interface.,Source=Microsoft.DataTransfer.Common,''Type=JNI.JniReturnException,Message=Cannot create JVM: JNI return code [-4][JNI call failed: memory error.],Source=Microsoft.DataTransfer.Richfile.HiveOrcBridge,'

Some additional information , trying to copy data from Azure SQL Managed Instance using SHIR to ADLS. Tested connection to both Azure SQL MI and ADLS are working fine

Can anyone help me with this error?

Answer 1

Hi @Jacky ,

Thanks for using Microsoft Q&A forum and posting your query.

For error message 2 (Cannot create JVM: JNI return code [-4][JNI call failed: memory error.]): JNI return code -4 usually means there is not enough memory for JVM to get started.
Possible resolution:

1. First, please check whether there is enough RAM on the SHIR VM. If you have set _JAVA_OPTIONS before, double check the Xms value to make sure it is not too big (reasonable value is 256M 512M). For more information about _JAVA_OPTIONS setting while copying parquet format please refer to this document: Parquet format - Using SHIR In ADF
2. Secondly, check whether there are other heavy loads running in the VM where SHIR is hosted around the time the pipeline is failing. Generally, parquet related jobs have a high memory footprint and hence, it is recommended to reduce the concurrent parquet jobs count. Reserve at least 2GB memory for each parquet job, a reasonable concurrent job count could be (Available RAM)/2. In case you cannot decrease the concurrent jobs then you may want to use a more powerful VM with more RAM. Or when processor usage is high and available memory is low on the self-hosted IR, add a new node to help scale out the load across machines. For more information about scaling considerations please refer to this doc: ADF SHIR Scale considerations

For the error message 1: It says unknown error and as per my internal analysis, it requires a deeper analysis, and the support engineer may need SHIR logs for further investigation. If you are seeing the error message 1 consistently or frequently, I recommend filing a support ticket as this is related to SHIR performance troubleshooting. And incase if you don't have a support plan, please let me know so that I can work with you offline in creating a onetime free support ticket to investigate on this issue.

In addition, you can also refer to this SHIR Diagnostics tool documentation and see if it provides any additional information about the SHIR performance issue - Diagnostic tool for self-hosted integration runtime

Additional helpful resources: Troubleshoot self-hosted integration runtime

Hope this info helps.

---

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.