Hello Ville,
Thanks for the update.
Have you tried in a different or private browser?
->Do you have any other authentication methods like phone or SMS at this sign in window? If so, you can make these the default and remove the authenticator app.
Regarding removing all authenticators, this may need to disable security defaults in Entra admin portal. However, this is highly not recommended. Once disabled, you will only login with the password. To disable security defaults in your directory:
- Sign in to the Microsoft Entra admin center as at least a Security Administrator.
- Browse to Identity > Overview > Properties.
- Select Manage security defaults.
- Set Security defaults to Disabled (not recommended).
- Select Save.
Providing a default level of security in Microsoft Entra ID - Microsoft Entra | Microsoft Learn
Looking forward to your response.