This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Hi everyone,
I'm trying to resolve a high CPU condition on a 2 node Windows Server 2012R2 cluster hosting a SQL Server 2016 Availability group.
On the virtual servers with 4 vCPUs nothing really special happens . But the clussvc.exe causes 2 vCPUs to stay at 50% usage each.
When I deactivate the cluster state monitoring in SCOM the CPU usage drops a few minutes later to almost 0% for clussvc.exe
Did anyone had the same issue or has an idea what exactly causes clussvc.exe to go crazy?
I am not a SCOM expert (more the SQL Server guy) but I'd like to get this solved / explained. I do not want to burn CPU time for nothing.
Regards,
Dirk
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 30%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECM%3C/text%3E%3C/svg%3E)
@Dirk Hondong , From your description, I notice SCOM cluster state monitoring cause high CPU on clussvc.exe. And we have deactivate cluster state monitoring and find the CPU usage returns normal. Before going on, I would like to confirm if we disable on related monitor in SCOM to deactivate the cluster state. If yes, Could you let us know what is the monitor name? Meanwhile, I would like to know the MP name and version which include this monitor.
In addition, could you also let us know what is the version of our SCOM server?
Please collect the above information and if there's any update, feel free to let us know.
Hi Crystal,
I am not sure if the cluster state probe is really the one causing clussvc.exe to spike.
I'm trying to pin it down. The thing is: if I set the cluster state probe to maintenance mode all attached monitors are also going to maintenance mode.
So, not that easy for me, eg which other Maintenance Packs are involved or if it is really a Cluster Mgmt Pack or if it is SQL Server (Always On) Mgmt Pack.
So, the SCOM Server is Verison 1807
Windows Server 2012R2 Cluster Mgmt Libraries have Version 10.0.6.6
SQL Server 2016 Libs have Version 7.0.15.0
Regards
Dirk
@Dirk Hondong , Thanks for the reply. I notice when put cluster state probe into maintenance status, all the monitors will be not working. Here, I would like to know id there's any monitor that will query event logs using expression. If yes, I suggest to disable this monitor or rule to see if the CPU usage will be normal. Meanwhile, could you check if there's any custom rule or monitor we create may cause this issue? we can disable them one by one to see if there's any abnormal one
In addition, please also check if the windows server of the cluster is with the latest update. If not, try to install the latest update to see if it will help.
Hope it can help.
Hi @Crystal-MSFT ,
thank you vor reaching out again.
I'll check the monitors but it will take some time.
I will come back to you later.
Regards,
Dirk
@Dirk Hondong , Thanks for the reply. we will wait for the update. If there's anything we can help, feel free to let us know.
Hi again,
since my colleague, who is responsible fot SCOM, is out of office I cannot ask/check how to disable specific monitors.
However, I stumbled across the FailoverClustering Log in the event viewer of one of the nodes.
It is full with errors and warnings which occurs multiple times in jsut a second.
However, if I put the node in SCOM maintenance mode the errors and warnings dissappear.
So, first of all I have an Error message Event ID 2051 saying
[API] AccessCheck[AndAuditAlarm] failed. status = 0x00000005 (maybe access is denied?
followed by 2 warnings:
[API] s_ApiOpenClusterEx: Client AccessCheck failed with 5 EventID 2050
[API] s_ApiOpenClusterEx: failed with error = 5 EventID 2050
Ok, I think I have found the answer / solution by myself.
As I have stated out I had a lot of errors and warnings in the failover cluster log which you can access through event manager.
The next hint was this old technet post
https://social.technet.microsoft.com/Forums/en-US/e7e95c36-8f82-45b6-a88b-bdd423a4a1b5/access-denied-errors-api-sapiopenclusterex-failed-with-error-5?forum=virtualmachinemgrclustering
where Erland stated out that NT Authority\network service did not had sufficient rights.
I checked the properties of my cluster object from Failover cluster manager and the network service only had Read Access.
With the powershell cmd Grant-ClusterAccess "NT Authority\Network Service" -Full
(https://learn.microsoft.com/en-us/powershell/module/failoverclusters/grant-clusteraccess?view=windowsserver2019-ps)
the high CPU usage turned from clussvc to the WMI Provider. Putting both nodes then in maintenance mode for a few minutes and then let Health Monitor resume its work did the trick.
No high CPU usage anymore.
Regards
Dirk
@Dirk Hondong , Thanks for the reply and sorry for the late response during our weekends.
I am glad to hear that our issue is resolved by granting NT Authority\network service the sufficient rights. Congratulations! And thanks for sharing the solution here, it can help others who has the same issue. I appreciate it.
Thanks again for your time and have a nice day!