Hi @Mikaela Caron ,
@MarileeTurscak is referring to the Web Redirect URI that you mentioned. You'll have to toggle that between localhost and your deployed app or you can create a separate app registration for your localhost environment. That doesn't mean adding an app registration here will be automatically linked your app. Think of the App Registration as a registry of entries where the entry says for applications that have this GUID, the Application (client) id, send the token of an authenticated user back to this URI.
Now, from the app service standpoint, you'll use the Authentication/Authorization blade to say my app will use this app registration entry because this app has the URI that will receive the token. You can also create app registration from the app service from this blade and see it appear in the App Registration blade of the Azure Active Directory because your saying I want my app to receive the token from my tenant when a user successfully signs in.
Hope that clears up the confusion of why you can't configure app service registration from AAD in the portal. If you have any further questions, let us know.