You need to create a login page and enable anonymous access to the page, and on this page, you can display different identity providers. For ASP.Net Core, you can specify Login Path or Access Denied Path, both these pages will have anonymous access.
If you are using any other programming framework supported by Azure App Service, you may need to refer how this can be done in those frameworks.
Also, I recommend you refer to the following tutorial to get a better understanding.
Hope this helps