CPU 100% utilisation

Question

Hi

We have a windows 2019 server which runs a single application. Every 2 weeks, this VM reaches 100% CPU usage and the application falls over. Rebooting the server doesnt make much difference, the CPU usage rockets on reboot. The only way was are able to sort the issue is to redeploy the VM - this knocks the CPU usage back down to less than 10%.

We have looked at upping the VM size but the fact that the application runs fine for a week before suddenly going nuts is puzzling - there is no extra processing every 2 weeks. we are beginning to wonder if there is something the ubderlying hypervisor doesnt like seeing as redeploy sorts it. Can anyone give any suggestions as to how we can troubleshoot this or has anyone seen this behaviour before?
Thanks

Sam

Answer 1

Hi @Bailey, Sam

Thanks for reaching out to Microsoft Q&A.

The following factors are common in a high-CPU situation:

• A recent code change or deployment that is mostly applicable to apps such as Internet Information Services (IIS), Microsoft SharePoint, Microsoft SQL Server, or third-party applications.
• A recent update that might be related to an OS-level update, or to application-level cumulative updates and fixes.
• A query change or outdated indexes. SQL Server and Oracle data tier applications also have a query plan optimization as another factor. Data changes or a lack of appropriate indexes can cause several queries to become more compute-intensive.
• Azure VM-specific. There are certain processes such as RDAgent, and extension-specific processes such as Monitoring Agent, MMA agent, or Security client, that can cause high-CPU consumption. These processes must be viewed from either a configuration or known issues perspective.

Here is the document to troubleshoot on this issue- https://learn.microsoft.com/en-us/troubleshoot/azure/virtual-machines/windows/troubleshoot-high-cpu-issues-azure-windows-vm?source=recommendations

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

I would start by RDPing to the server and analyzing the cpu usage of running processes. You need to determine if the cpu usage is caused by the virtual machine infrastructure or some application program running on the OS.

In the details tab of task manager sort the CPU column. Do you see a process using a lot of cpu? Is it your application?

If it's the application that the server runs, then you need to figure out what it is doing. Contact the developer and ask them to enable debugging within the app so that you can troubleshoot the cpu usage.

If they look at you with a blank stare and reply that they don't know how to do that, then you can help them along by using the Sysinternals tools.

Process Explorer will allow you look inside of the runaway program.

https://learn.microsoft.com/en-us/sysinternals/downloads/process-explorer

Double click on the program and look at the threads tab. This might point you to the function that is using the cpu. But again, you would need to show this to your developers because no tool is going to tell you WHY the program is doing what it's doing.

Another tool that I use quite frequently is Process Monitor.

https://learn.microsoft.com/en-us/sysinternals/downloads/procmon

This will allow you to trace the registry/file/network calls that a given process is making.

That will tell you if the program is in a hard cpu loop or if its actually trying to do something like read a file.