Since Sunday, 02.06.2022, SenseNdrPktmon maximum size warning occurs on all virtual machines

Question

Hello,

Ever since Sunday, the 6th of February, all of our Azure virtual machines have been reporting the following warning:

The message is:

The backing-file for the real-time session "SenseNdrPktmon" has reached its maximum size. As a result, new events will not be logged to this session until space becomes available. This error is often caused by starting a trace session in real-time mode without having any real-time consumers.

From what we can tell, it is an issue with Windows Defender performance monitors. However, we cannot figure out how to fix the issue. We have patched and rebooted several virtual machines, but the event is still logged.

There are also some additional "Sense" events logged:

Session "SenseEventLog" failed to start with the following error: 0xC0000035

Session "SenseIRDebugLogger" failed to start with the following error: 0xC0000035

Any assistance with figuring out how to stop these events from occurring is greatly appreciated.

Thank you,

Mathew

Answer 1

same issue with one of my customers on their VMs hosted on VMware ESX, after they Activated MDATP/MDE and Sentinel.
Seems situation is worst on Print Server.

Updating Drivers, removing virtual print queues and adding CPU didnt resolve the issue.
Got the Event ID 1 every few seconds...

Answer 2

Has anyone been able to resolve this issue?

I am seeing it on Server 2022, and the log files have 0 size.

Answer 3

I have spent way to much time looking into this and I gave up.

I think we can agree that the amount of these warnings and errors, often in quick succession, strongly indicate some kind of issue with pktmon when Defender for Endpoint is active / the client is onboarded. Defender being one of our primary defenses on all of our windows clients, this situation is not something to be ignored.
After contacting support and being told that this is an issue with a corrupted windows installation (which, conveniently, is not included in m365 support) I freshly installed windows 10 pro on a dell client (manually, no domain join) and onboarded it on defender for endpoint. This clean, updated, licensed windows 10 pro installation without any additional software or configuration shows the same errors and warnings after just one day. Therefore this is most probably not an issue with a corrupted windows install.
I have also verified that these warnings and errors only appear when defender for endpoint is active (because pktmon trace is only started by defender for endpoint which can be verified by the "pktmon status" command), and these logs were also sent to support. But at this point I just expect this issue to silently being resolved with a future defender or windows update. Until then I just hope these messages are not indicating some serious issue we now overlook.

Oh and microsoft 365 support is not going to help you, don't bother wasting your time.

Answer 4

EDIT: right after I posted this answer, the "Microsoft-Windows-Kernel-EventTracing/Admin" log started getting chatty again. So still no luck, but maybe the below could be part of finding a permanent solution?

I was able to resolve this on a W10 Enterprise v21H1 PC by taking ownership of this folder:

"C:\Windows\System32\LogFiles\WMI\RtBackup"

It all seemed to stem from this error, logged in the Kernel-EventTracking log.

"Session "SenseNdrPktmon" failed to write to log file "C:\WINDOWS\system32\Logfiles\WMI\RtBackup\EtwRTSenseNdrPktmon.etl" with the following error: 0xC000007F"

I got quite a few more of those before it eventually (less than 10 min later) started churning out the Warning event above (Event ID 1) within the same log source.

I was getting events recorded pretty much every second until I took ownership of the above "RtBackup" folder. It's now been almost 15 minutes since I made the change and the "Microsoft-Windows-Kernel-EventTracing/Admin" log has been completely silent. Hopefully this will work for someone else!

Answer 5

While I'm not sure the reason for the malfunction of the event logs, this issue is caused by the "Windows Defender Advanced Threat Protection - Sense NDR module".
This is the "SenseNdr.exe" process in task manager.

According to the document below, the SenseNDR.exe process is actually part of the "Device Discovery" Process.
https://learn.microsoft.com/en-us/microsoft-365/security/defender-endpoint/device-discovery?view=o365-worldwide#discovery-methods

According to this FAQ about Device Discovery, you can disable the discovery, however the SenseNdr.exe process will still run on the machine.
I'm testing disabling that now, but so far as long as the process is running, I'm getting the event viewer entries, so I'm not sure if the change has not taken effect yet, or if the process running causes the Event Viewer entries regardless of Device Discovery being Disabled or Enabled.

You can temporarily stop the event log entries by going into task manager and end the SenseNDR.exe process, however it starts itself back up within a few minutes.
Theoretically you could create some script or something to end the process any time it starts, or possibly use some kind of access restrictions to prevent it from starting itself up.

https://learn.microsoft.com/en-us/microsoft-365/security/defender-endpoint/device-discovery-faq?view=o365-worldwide#can-i-disable-basic-discovery