I found what the issue was. It was related to the "Packaged" setting of the driver and whether it was true or false for one given printer driver. We had one specific model of Zebra printer of which there are about 30 different printers of the exact same model that are installed for each user. Typically when we seed the drivers on a server that someone works from in a Remote Desktop Session Host environment, we Install the printer and driver first on the print server, then we browse to the print server as an admin from the RDSH server and seed the driver by installing the printer from the share on the print server once so we can get the driver pulled over from the print server (we block users from installing drivers, so an admin needs to seed the driver initially on the servers).
The behavior that we found, and it seemed to only be happening with this Zebra print driver, was that it was changing the "Packaged" flag on the driver from true (on the print server it was true) to be false on the Remote Desktop Session Host servers when we seeded the driver like this. Mind you, other different printers didn't seem to exhibit this behavior when seeding the driver from the print server by browsing to the print server as an admin and adding the printer once for a particular given printer model / driver. When I removed the print driver from the RDSH servers that was seeded like this and then manually re-added the driver back to the server via using the extracted source drivers (the same ones I used to set up the printer and drivers initially on the print server), the flag was changed back to true for the "Packaged" setting so that it was now true both on the print server and the RDSH servers.
After making this change, the 50+ printers being added to new users logging in went from taking about 4 minutes and using far more CPU cycles went down to taking about 20-40 seconds, depending on other variables. The higher CPU spike is still there during the GPO phase where printers are being added, which is expected for adding that many printers, but at least now it's getting the printers added much more quickly and is less disruptive to the server.