HDinsight spark livy server stopping as soon as It starts

Question

I am a new user of Azure HDInsight. I have created a 4 node spark cluster. When the cluster is successfully created, I saw spark history server and livy both are in a stopped state. And when I try to run them from Ambri it gets stopped by hdinsightwatchdog as soon as it gets started. Can anyone help me understand what is happening here.

Answer 1

Hello @Gourav Sharma ,
Thanks for the ask and using Microsoft Q&A platform .

As I understand at this time , the ask is why the Livy and history server is in stopped state . These two services does not run on all the nodes but one of the nodes in the clsuter but to maintain high avalaiblity it is installed on all the nodes . So when you want to start the services and the services is already installed & running on any other nodes , the watchdog kicks in and it stops the services on the node . ( I am sharing the screenshot below ) .

As you mentioned you are running a 4 node cluster , this service must be running on one of them , request you please check that .

Thanks
Himanshu

-------------------------------------------------------------------------------------------------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators