Are you performing the proper WSUS maintenance including but not limited to running the Server Cleanup Wizard (SCW), declining superseded updates, running the SQL Indexing script, etc.?
https://www.ajtek.ca/wsus/how-to-setup-manage-and-maintain-wsus-part-8-wsus-server-maintenance/
Have you made any of the optimizations to WSUS and IIS that is recommended by Microsoft?