Hi Marten,
Thanks for your query in this community.
Based on my test results, the user who don't have SharePoint Online license could access to the SharePoint site without issue.
You just need to share this site to their email address and grant them with correct permission level.
Share a site
In you scenario, I would like to confirm if those user have their corresponding UPN in your Tenant? Or they are just using their Outlook/Gmail account? Either will do. But please notice, if they do not have the Office Online license or Desktop Office license,
they won't be able to open the documents in the SharePoint document library. So in short, they might have limited access to some features provided in SharePoint Online.
You can also refer to the Subscription options and
SharePoint Online Enterprise external users section in the following article:
SharePoint Online Service Description
It's not necessary to add those users to Azure AD manually, per my experience, if you share the site to them, they will be added to the guest user in Azure AD (If user doesn't have an User Principe Name) once they open the site in their mailbox.
Regards,
Alex Chen