Since Sunday, 02.06.2022, SenseNdrPktmon maximum size warning occurs on all virtual machines

Question

Hello,

Ever since Sunday, the 6th of February, all of our Azure virtual machines have been reporting the following warning:

The message is:

The backing-file for the real-time session "SenseNdrPktmon" has reached its maximum size. As a result, new events will not be logged to this session until space becomes available. This error is often caused by starting a trace session in real-time mode without having any real-time consumers.

From what we can tell, it is an issue with Windows Defender performance monitors. However, we cannot figure out how to fix the issue. We have patched and rebooted several virtual machines, but the event is still logged.

There are also some additional "Sense" events logged:

Session "SenseEventLog" failed to start with the following error: 0xC0000035

Session "SenseIRDebugLogger" failed to start with the following error: 0xC0000035

Any assistance with figuring out how to stop these events from occurring is greatly appreciated.

Thank you,

Mathew

Answer 1

Not from my end.

Answer 2

I thought I was the only one! I have been suspecting this to be involved with a crashing server of mine. These events are generated on most, if not all, my Azure VMs, but the one crashing generates significantly more per day than others. Healthy server has about 180-200 events a day. The unhealthy server has over 500 a day.

My metrics mimic you behavior. CPU spikes to 100% over the course of 1-2 minutes and stays pegged there. Network traffic drops off the earth. The server has crashed roughly 5 times now and each time same behavior. Each time I check my sysmon logs, its always Windows Defender events kicking off at the same time the CPU spikes.

I have Windows Defender disabled in the VM as I used my own, third party. I do have Microsoft Defender for Cloud enabled on the server - as I do all my servers.

The server is running IIS with a special third party module called DNN or DNNGo.

Sharing all of this in hopes we can find a common ground and help MS pinpoint. I've raised two tickets with MS so far and unless I can provide the perfx logs - there is not much they can do for me. Funny thing is, the server locks up and performance diagnostics will not complete in a timely manner. This server hosts something that cannot tolerate a long downtime. Longest I've tried was 15 minutes. Normally should only take 5 minutes.

Redeployed the VM. Recreated it. Recreated NIC.

Answer 3

Does anyone have found the log I should give for storage size ? Can;t find it

Answer 4

Same issue here. We sometimes see 30+ of these warnings in a few minutes time, not limited to reboots or windows updates. Affected are Windows 10, Windows 11, Windows Server 2016, Server 2019, Server 2022 running Physical and as VMs. The warnings seem to accumulate specially, but not exclusively, when a network connection is lost. Maybe it has something to do with Defender for Endpoint? Google does not show a lot about SenseNdrPktmon so I'm hesitant to contact support about this (we don't have support for the windows clients). While investigating other (possibly unrelated) issues, the amount of these warnings makes it very difficult to troubleshoot anything.

Has anyone found out what backing-file is affected and how to reset/change those settings?

Answer 5

I've found daily log files under
C:\ProgramData\Microsoft\Windows Defender Advanced Threat Protection\Temp\SenseNdr_28_06_2022.log
but they are all empty.
Also it seems that the packet monitor is started automatically at startup

PktMon.exe status  
  

I cant access the trace directory under C:\ProgramData\Microsoft\Windows Defender Advanced Threat Protection\ - it seems to be locked down even for admins.

I don't know whether the above things are supposed to be this way or not. Maybe someone with more knowledge of pktmon can give some advice.