False Positives on Attack Simulation Training - And how to cancel the training assigned to the user as a result of the false positive?

Question

#1. Defender is reporting that users opened an attachment on an Attack Sumulation. Several users are claiming they did not open the attachment.

We've been using Defender for a little over 2 years, and we used another tool prior for 5 years prior to Defender. Not a single user has ever disputed failing a phishing simulation. Now, all at once, at least 17 users have reported that they didn't open the attachment, but Defender reports they did. Also, we rarely have more than just a couple users fail a simulation. I'm inclined to believe the users are correct and these are false positives.

Is there any way to adjust the test results, flag them as false positives, or otherwise fix them so that my reports are accurate? Or is my only option to exclude the simulation?

#2. Now these 17+ users are getting twice weekly emails to complete a training course that they shouldn't have to take. If they block the email address, they won't get training notifications for future failures. Is there a way to cancel the training requirement?

We do a lot of training, including several phishing courses. I really don't want to tell our users, including C-level execs, that they have to take another course because of false positives from our simulation tool.

Answer 1

Hi Jason,

Did these false positives just start occurring, or has this been happening for a while? I ask because I have seen some other users reporting similar behavior but I'm not sure if they are working with you or are separate users.

If you have attachment scanning or Data Loss Prevention (DLP) processing, it's possible that DLP is opening the attachment on send to check it and that is getting it flagged. You could try removing the DLP compliance policy (if applicable) to see if you face the same issue. If you want to exclude certain paths from DLP monitoring, DLP alerts, and DLP policy enforcement on your devices, you can also turn off those configuration settings by setting up file path exclusions. https://learn.microsoft.com/en-us/purview/dlp-configure-endpoint-settings

If you just want to remove the training emails though, you can remove or modify the end user notifications, or you can add exclusions to the training simulations themselves.

Otherwise I would recommend creating a support ticket to look into this and query additional logs from your tenant, since it will be harder to diagnose this without testing within the tenant itself.