Figured this out by accident, after finding only this unanswered question while Googling. These steps worked on my Galaxy S20+ running Android 12. YMMV on other devices and/or Android versions.
- Push and hold on the (Microsoft) Authenitcator App icon.
- Tap the info circle in the top right corner of the popup.
- Tap notifications.
- Tap Notification Categories.
- Tap All notificaitons (not on the slide to togge on/off, just tap on the text).
- Tap Sound and pick the notification sound you want.
At first I thought this changed the default noise for all notifications, but it appears to have only changed the sound for the Microsoft Authenticator app.