Hello,

Last night around 5:15am UTC (10:15pm local), our server stopped functioning and was unreachable until deallocating and reallocating. This totaled to 10 hours of downtime and loss of data, which we want to avoid in the future. I was hoping for some advice to troubleshoot the root cause and potential solutions.

I've attached a journalctl log.txt from that time frame and it appears that some software updates were applied automatically - shortly thereafter the server became unresponsive. Looking at the memory allocation, it's clear that we ran out of RAM but it never recovered. I understand that there's a way for the server to automatically heal itself, but I haven't been able to find the option.

Additionally, the server now has its health state as "Unhealthy" but the troubleshooters can't find any issue.

Any recommendations?

Well in such scenario you can schdule a task for every 5 min which will run powershlle script to check server utiliation and give you alert at 85 % and restart your server if its tochues to 90 % of server RAM, Please find attached Unexpected Server Outage.txtsuggested powershell script.

Please accept as answer if it does help.

Hi @BHaist,

If an answer has been helpful was provided by VasimTamboli, please consider accepting the answer to help increase visibility of this question for other members of the Microsoft Q&A community. If not, please let us know what is still needed in the comments so the question can be answered. Thank you for helping to improve Microsoft Q&A!

We ended up resizing the VM to allocate more RAM since our machine was pushing its limits as-is. I neglected to mention it's a linux VM, so unfortunately a powershell script wouldn't work. I also didn't want to restart the server mid-update and risk system errors.

I wasn't able to find a way to make the server "heal" itself, but I did set up notifications when free RAM dropped below a certain %. This was in the VM>Monitoring>Metrics and selecting RAM % free, then selecting "New Alert Rule".

Hi @BHaist,

We have noticed that you have rated an answer as not helpful. We appreciate your feedback and are committed to improving your experience with the Q&A.

I have noticed that you are using the basic standard VM (B1ms). Could you please increase the size of the VM and try again?

That has already been done and was the solution to this problem.

@BHaist ,

Checking into see that if your issue has been resolved or do you have any further queries, please do let us know. We are happy to assist you.

Hi @BHaist,

Good catch! Glad the issue is resolved for you finally. I will have this answer promoted by reposting it. As an Original Poster BHaist will not be able to accept your own answer.

This is in the attempt to help others looking for a solution for a similar issue.

The issue ended up resizing the VM to allocate more RAM. also set up notifications when free RAM dropped below a certain %. This was in the VM>Monitoring>Metrics and selecting RAM % free, then selecting "New Alert Rule".

Thanks again for sharing the solution here. Have a good day!