you can use unique permission for the document library first.
then grant designer permission for the user.
so that the user just can design the site without library permission.
Step as below:
- create a SharePoint group for manage library
- Set library permission: Stop Inheriting Permission -> Check all permissions and remove User Permission
- Grant permission for the group
- Grant design permission for the designer developer