I believe I have managed to disable it:
First, I had created an AntiPhising policy myself on which I had forgotten to untick the "Show first contact safety tip". After doing this, I still got that annoying prefix on my mails, though.
Then I listed the policies using the ExchangeOnlineManagement PowerShell Module, as the actual functionalities of both the "Standard protection" and "Strict protection" (https://security.microsoft.com/presetSecurityPolicies) seems not to be something an Administrator should be informed of. This showed that the Standard Policy too had the EnableFirstContactSafetyTips set to true.
Unfortunately, this policy cannot be edited, but it can be disabled (see previous link). After disabling the Standard Preset the annoying banner was no longer applied to incoming mail.
I guess some Anti-phishing policy is good to have, so I suggest that you create one yourself. I have attached a picture of the Actions overview of my own policy.
Here are a list of commands to execute to get an overview of your Anti-Phishing policies through PowerShell:
In an elevated PS shell:
- Load the required module: Update-Module -Name ExchangeOnlineManagement
- Connect to the service: Connect-ExchangeOnline
- List your policies (by name): Get-AntiPhishPolicy | Format-Table Name
- Get the settings for some policy: Get-AntiPhishPolicy -Identity "<exact name from output from previous command>", e.g. Get-AntiPhishPolicy -Identity "Standard Preset Security Policy1648196500981"
- Look for setting EnableFirstContactSafetyTips (for me the second item listed). If this value is True, then the banner will be applied.
Hope this helps!
![actions](https://learn-attachment.microsoft.com/api/attachments/9b83cc9b-4bb2-4475-b13f-90afac1ac2c7?platform=QnA)