Changing the default open settings for files in the SharePoint Online is an available solution to stop pop-up window.
1.Site settings -> Site collection features -> Deactivate Open Documents in Client Applications by Default feature.
2.Library settings -> Advanced settings -> Select Open in the browser.
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.