Hello !
In the new Outlook, you need to enable manually the Reminders window.
Open Outlook settings > General > Notifications.
Under "Calendar", notification style should be set to "Reminder", and "Automatically dismiss reminders for past events" should be disabled.
Please let me know if you have any other questions !