Why is the Windows 11 Spoolsv.exe process sending out SNMP and PDL (ports 161, 9100 respectively) to a private IP that does not exist on the network?

Question

While looking at the network traffic we have, from the perspective of our firewall, the logs were showing what I considered to be a problem that needed to be investigated more.

There were WWW IPs showing up as being on our LAN and IPs that had our internal LAN prefix octets but do not exist on our LAN.

The WWW IPs on the internal LAN was worked out to be the way the log files handled entries for the return traffic from the WWW, so no 

foul there.

The issue of internal IPs that don't exist was more perplexing.

Using packet captures by the firewall and local Wireshark installations along with 'netstat -ab' commands and TCPView I was able to determine that across all of our Windows 11 devices the spoolsv.exe, otherwise known as 'Print Spooler' system service, is the culprit.

On roughly 10 second intervals the spoolsv.exe process is sending out an SNMP packet to the IP address of 192.168.1.202 over UDP with a target port of 161.

Concurrent with the above, but on much more random intervals ranging from 1 to 15 seconds+, there is a PDL packet to 192.168.1.202 targetting TCP port 9100. According to web pages, port 9100 is used for streaming raw data to a printer.

The weird parts of this are:

- We have no device at 192.168.1.202 and never had one in the past.

- We have never used an IP address in the 192.168.1.* space.

- There is no network tunnel to some other network. Our network is a standalone island within the walls of our building.

- This happens on Windows 11 devices that were upgraded mid last year from Windows 10 and brand new Dell devices that were delivered less than a month ago and put on the network.

- All devices are patched to current releases for OS and firmware.

Our firewall doesn't know what to do with packets for 192.168.1.202 so we defined an access rule within the firewall to block them from going anywhere but the LAN segment the desktops are on.

I have not found any references to this combination of network traffic and spoolsv.exe on the WWW and am wondering if others have encountered this, know why it is happening and how to stop it? 

I did find a reference to a malware payload that was trying to be a replacement for spoolsv.exe and exfiltrating data as a print data stream out of the facility, which is seriously sneaky.

Questions, comments and opposing views are welcomed.

Paul

***moved from Windows / Windows 11 / Devices and drivers***

Answer 1

Try to disable SNMP on all network printer-ports:

Answer 2

Hello Paul

Hope you have a lovely day!

The behavior you're observing with the spoolsv.exe process sending SNMP and PDL data to a non-existent IP address on your network is indeed unusual and warrants thorough investigation. Here are some steps and considerations that may help you diagnose and potentially resolve the issue:

1. Check Printer Configuration and History:

- Printers & Scanners settings: Verify if there were any printers previously installed or configured that might have used the IP address `192.168.1.202`. Even if the printer has been removed from the system, some remnants of its configuration might still be triggering the spooler service. 

- Registry Check: Look into the Windows Registry to see if there are any entries related to this IP address. You can use `regedit` and search for `192.168.1.202`. Be very cautious when making changes to the registry. 

2. Examine Group Policies and Scripts:

- If your organization uses Group Policies to manage printer configurations, check if any policies might be pushing out configurations that include the mysterious IP. 

- Also, check any login scripts or configuration scripts that might run at startup or at regular intervals. 

3. Network Configuration Check:

- Ensure there are no DHCP settings or static routes that could be misdirecting traffic to `192.168.1.202`. 

- Double-check any VLAN configurations or isolated segments that might have overlapping or hidden network configurations. 

4. Security Audit and Malware Scan:

- It's prudent to run a thorough security scan using a reputable antivirus and malware detection tool. Consider the possibility of a compromised `spoolsv.exe` or other system files, especially given the potential reference to malware you found. 

- Verify the integrity of the `spoolsv.exe` file by comparing its properties, such as file size and digital signature, with known good copies from similar systems or directly from Microsoft sources. 

5. Monitor and Block Unnecessary Traffic:

- Since you've already started blocking this traffic at the firewall, continue to monitor these blocks to see if any new or changed behaviors occur as a result of these rules. 

- Consider implementing additional monitoring at the switch or router level if possible, to capture any traffic attempts to or from `192.168.1.202`. 

6. Reset or Reconfigure Print Spooler:

- As a more drastic measure, you could try stopping the print spooler service, clearing out the spooler and printer configuration files (from `C:\Windows\System32\spool\`), and then restarting the service or even rebooting the system. This should essentially reset the print spooler configuration. 

Best regards

Rosy

Answer 3

Rosy..

My thanks or your detailed reply!

Going through the sections you documented, in order, the following is offered.

1. Check Printer Configuration and History:

Unfortunately, I took over when the previous admin died of a heart attack and left zero documentation and zero user creds (admin or user) for the site. It has been a challenge, ugly at times, to get access. The huge downside is there is no history of what he did, how he did it and his thoughts at the time. I am fixing the documentation issue (OneNote) and creds (password mgr).

Since you reply I have done a lot more digging just within the print spooler and the printer definitions.

What I found is that there was a duplicate printer entry for a RICOH device. One went to 192.168.25.202, which is the subnet for our printers and desktops while the other was for 192.168.1.202!

I spent a considerable amount of time doing:

• deletion of the 'bad' duplicate RICOH printer from Settings page.
• Created restore point and then used Regedit to search through everything, multiple times, looking for 'Ricoh', 'C3500', 192.168.1.202, and whatever other unique strings I could think of to find 'bad/bogus' RICOH device entries. Did a lot of deletes.
• Rebooted the desktop many times to see what remained gone and what 'came back'. Eventually got to the point where there was a single RICOH printer device pointing at the correct IP.

2. Examine Group Policies and Scripts:

• We use an AD WorkGroup here and no GPOs based on what I can find.
• I checked the Run and RunOnce registry keys for scripts and found nothing but .EXE files.
• Looking at the Task Scheduler, I did not find anything suspicious. The 'Microsoft>Windows>Printers' folder has the 3 standard entries.

3. Network Configuration Check:

• Checked the firewall, which also does DHCP for all of the subnets. Nothing there for the 192.168.1.* subnet in any regard. No static routes
• VLANs are all unique with no cross boundary issues. The ones that should talk between others is allowed and ones that only talk with WWW only get to the WWW.

4. Security Audit and Malware Scan:

• Interesting that you include this aspect. I am currently running both MalwareBytes and MS Defender. I have some doubts about MalwareBytes and started a comparison effort between the two. Long term, I am leaning towards Defender, XDR and Sentinel from MS and dropping MalwareBytes. We get a more complete package that SHOULD all talk among themselves, correctly, to show what is going on. Time will tell if that is a going to work out well or not.
• Bottom line is that both AV products show no problems. Don't get me started on run-times, 'file counts' and other mismatches.
• Spoolsv.exe has the MS signature, and matches properties with the same file on other desktops. Yes, I know, if all the desktops are infected via spoolsv.exe then that would be the case. ;(

5. Monitor and Block Unnecessary Traffic:

• Asked questions of others to see if any of the printers NEED to talk with something on the WWW and am awaiting answers. One thought was if they are reporting to the leasing company to report pages printed for billing purposes.
• Part of me wants to just put in firewall access rules blocking all printer IPs from getting to the WWW and be done with it. Hopefully none of the printers need to talk to a lease company.

6. Reset or Reconfigure Print Spooler:

• There is nothing in the spool directory. Stopping/starting/rebooting did nothing to the problem except test my ability to constantly put in user creds and MFA strings.

With all of this said, I came to find some things out.

1. I installed or used a range of tools to see what is going on. Tools such as DriverStore, Print Management, PowerShell Get-Printers, Settings, Device Manager, Printui, Regdit, Wireshark, SonicWall packet Capture, netstat, arp, Resmon and maybe a couple of others. A fair number of these tools replicated the same basic output, which was good to prove that there is consistency. For the MOST part. More on the inconsistency below.
2. As mentioned earlier, there is no SNMP package installed or configured on the desktops. Yet a number of the desktops were sending out SNMP packets on port 161, confirmed by Wireshark and firewall packet capture.
3. When I got rid of the 'duplicate RICOH printer' device that was configured to 192.168.1.202, which does not exist, the SNMP traffic stopped!!!! Apparently the print spooler can do some SNMP on its own for whatever it needs to do.
4. After a lot of effort, per desktop, the printer device list, shown by the 'Settings>Printers' web page, was whittled down to printers that actually exist and print jobs can complete on.
5. And that printer device list has persisted over 2 days and many desktop reboots without modification.
6. When I use any of the MS Office products and print a document, the printer devices that show up in the list match what shows up in 'Settings>Printers'! Yea!
7. Here is the latest wrinkle.

If I open a document in either Notepad or Wordpad and do a CTRL-P there is an EXTRA Printer Device in the list AND it is the duplicate RICOH queue that would have gone to 192.168.1.202!!!!!!! Argggggg....

I went back through the registry, DriverStore, Print Management and Settings and *none* of them show the bad RICOH printer queue/device!!!

Yet Notepad and Wordpad show the bad RICOH device!

I deinstalled and then re-installed Notepad on the desktop to see if that would cause Notepad to regenerate the printer device list. No joy in Mudville. The good devices continue to show up as well as the bad RICOH printer device that no longer exists!!!

I have NO IDEA where/how/why Notepad and Wordpad manage the printer device entries!

I wondered about something socked away in appdata but I did not find any files there about Notepad/Wordpad.

Bottom line is that the issue is better understood, in at least general terms, with a lengthy process to fix the SNMP and PDL packets to a missing printer on the network. I will work through those steps per desktop. Tedious, but doable and needs to be done.

I am open to all suggestions on how to get Notepad/Wordpad to drop the printer device that does not exist!

My thanks for your help and that from any others that want to chime in!

Take care.

Paul

Answer 4

Rosy, et al...

I got to wondering about how pervasive the 'bad/removed' RICOH printer queue entry is on the desktop, since I can't find a method to remove it, at the moment.

So I created a new, local, non-admin account on the same desktop.

Then logged into that new account and went through the machinations of getting a new account to the point where it was usable to me.

Fired up Notepad, keyed some garbage into it, hit CTRL-P and Presto!

The same dead, non-existent, RICOH printer queue showed up in the list to select!

Along side the good RICOH print queue and other physical and soft devices!

I was seriously hoping that the new account would not have the dead/removed RICOH queue.

I looked at GPOs using gpedit, rsop and PowerShell for printer info and found nothing there about any of the printer devices defined on the node.

What I deduce from this test is that it is something 'persistent' across accounts, even brand new accounts!

The mechanisms to do that, that I can think of, are:

• Registry, which is clean
• GPOs, which is clean
• *.INF files, which have no entries for dead/removed RICOH queue
• *.ini files, no matches to dead/removed queue
• physical files on disk, search of all (hidden or otherwise) C: files for "58387928607B" showed 126 hits. All but 4 hits were within my admin account and would not be used for building a new account. The ones for my account were spread over multiple Google cache/historyshortcut files. The remaining 4 hits were in Windows Event Viewer.

I am at a loss of what to check next.

Paul

Answer 5

Hello Paul,

Thank you for your detailed feedback and thorough documentation of the situation! It sounds like you've been managing a very challenging situation quite adeptly, and I commend your efforts.

Regarding the issue with the printer settings, especially the persistent "bad" RICOH printer device that appears in Notepad and Wordpad, it indeed presents a perplexing problem. Here are some steps you might consider to potentially resolve the issue of the unwanted printer device still showing up:

Steps to Remove the Unwanted Printer Device from Notepad/Wordpad

1. Clear System Cache: Occasionally, outdated printer device entries can linger in system caches. Try clearing the print spooler cache and restarting the service:

• Stop the Print Spooler service via Services.msc or command line (net stop spooler).
• Navigate to C:\Windows\System32\spool\PRINTERS and delete all files within this folder.
• Restart the Print Spooler service (net start spooler).

2. Registry Cleanup:

Since you've already done some registry cleaning, it might be worth revisiting this with a focus specifically on Notepad and Wordpad entries:

• Use the Registry Editor (regedit) and carefully search for entries related to Notepad and Wordpad under HKEY_CURRENT_USER and HKEY_LOCAL_MACHINE.
• Look for any entries that might relate to printer configurations specifically used by these applications.

3. Reinstall Notepad and Wordpad:

Although you've already tried reinstalling Notepad, doing a more thorough removal and then reinstalling might help:

• Uninstall Notepad and Wordpad from the Optional Features settings in Windows.
• After uninstallation, check again for any leftover files or registry entries related to these apps and remove them.
• Reinstall Notepad and Wordpad through the Optional Features menu.

4. Check Application Data:

Sometimes applications store configuration files in hidden directories within your user profile:

• Check the AppData folder within your user directory for any Notepad or Wordpad specific settings or cache files. These might be located in subfolders like Local, LocalLow, or Roaming.

5. System File Checker (SFC):

Run the System File Checker to ensure that all Windows system files are correct and intact, which could potentially affect how applications handle peripheral devices like printers:

• Open Command Prompt as Administrator.
• Run the command: sfc /scannow.
• Allow the process to complete and then restart your computer.

Your situation clearly shows a lot of diligence and persistence. I hope these steps might offer some relief. I'm open to any further questions or details you might want to discuss!

Take care,

Rosy