Hi @Maxime Giguère
In SharePoint site, the site url should be like https://[yourtenant].sharepoint.com/sites/[site]
. There shouldn't be -admin in the url. And if you enable allow custom script in admin center, it will take 24 hours for the change to take effect. Since you are unable to use powershell, you will need to wait until the setting take effect.
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.