HDInsight cluster is failing when upgrading the version to 4.0

Kajol Dhosewan 1 Reputation point
2021-08-02T10:50:45.45+00:00

Hi Team,

I am getting this error :
Operation on target Txxxxxx failed: Hadoop job failed with exit code '2'. See 'wasbs://adfjobs@sc1admadstg01.blob.core.windows.net/HiveQueryJobs/c269912c-700d-4907-a4d9-4e51ae0096ca/02_08_2021_06_47_21_708/Status/stderr' for more details. Alternatively, open the Ambari UI on the HDI cluster and find the logs for the job 'job_1627886449866_0002'. Contact HDInsight team for further support.

this error is random and usually happens when i manually run the pipeline.

HDinsight cluster creates the cluster but i guess then it fails while running hive query.

the pipeline runs successfully at 3.6 version of HDInsight cluster Linked Service and it gives the above mentioned error when i upgrade the version to 4.0.

Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
198 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,521 questions
Azure R Server for HDInsight
Azure R Server for HDInsight
An Azure service that provides predictive analytics, machine learning, and statistical modeling for big data.
13 questions
{count} votes