1.Based on your description, I understand that you want to users can access document library but cannot access files in the document library. You cannot achieve this through custom permission level. You could achieve this through set unique permissions for files. Select the file -> Manage access -> Advanced -> Stop Inheriting Permissions -> Remove Users Permissions.
2.Please go to Library settings -> Permissions for this document library -> Check permissions.
3.You cannot modify default permission level. You can use Read permission instead of Restricted View permission.
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.