Hi Debbie,
It's feasible to create a specific permission document library in your SharePoint Online Site.
Please follow my steps below to proceed:
1.Create a new Document Library in your site. And open it, click on the Gear on the top-right corner > Library Settings.
2.Under Permissions and Management tag, Click on Permissions for this document library.
3.Click on Stop Inheriting Permissions button on the top left corner.
4.Choose the people or group whoever you don't want them to have permission to this library, and click Remove User Permissions.
Or you could remove all the existing people or group. And then, grant the permission for the 3 people.
Hope this information could help.
Regards,
Alex Chen