Take a look at Part 3 of my 8 part blog series on How to Setup, Manage, and Maintain WSUS.
Part 4 is also important because it sets the proper GPOs. One other thing to make sure of is that your WSUS server has the correct Mimetypes for UUP
https://www.ajtek.ca/guides/how-to-prepare-for-on-prem-wsus-uup-updates/
Theoretically, if you have the latest CU on your WSUS server, it should be good, but just in case - that's how you need to add them.