There is no way to set the permission on the Recycle bin rather it is control by the documents / items permission. Typically user with contribute access on Document library / list can access deleted stuff from Recycle-Bin. On Side note, Site collection owner/admin can access the recycle bin of the site collection.
If you want to restrict that ability then you have to lower the permissions of that user on the Document library ( less than edit permission).
If an Answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.