The External Access is for Direct/Instant Messaging or Collaboration. You can turn that off.
Meetings are a separate feature and you can invite anyone and you get a link for the Meeting. A phone number is included if you have the calling plan. You can also manage whether you allow anonymous access. That setting is in the Teams Admin > Meetings > Meeting Settings. Attached images of the Meeting Settings and how an anonymous user can sign in through a web browser if they get the link from your invite.