Hi Krbot,
I would like to clarify what is an external user in SharePoint Online. They are users that are not part of your organization and usually uses a work/school account of a different tenant, or a personal Microsoft account.
If the user is part of your organization (same tenant), then he/she is not an external user. Can you please let us know what the user's account type is?
Regarding the permission issue, we'd first like to confirm the detailed steps you did to add the user/group to the site (if possible, please add screenshots for each step).
Based on your description, I do not see an issue with your steps, which is why I would like some screenshots to see whether it may be an issue with the site.
Please also use your browser's private mode to make sure that the user logged in is the user you have invited to the site.
Regards,
Sheen