Share via

Troubleshooting the red (Suspicious) Safety Tip for fraud detection checks

  • Article

Introduction

It has now been about 8 months since we released our antispoofing protection in Office 365, a feature that defends against Business Email Compromise, where the From and To domains are the same. You can read more about that feature at my other blog post Antispoofing in Office 365. To summarize, it defends against others spoofing your domain in the From: address - the one that appears in your email client - by figuring out if the sender is legitimate or malicious. It's similar to how DMARC works, the key difference is that while DMARC looks for a DNS record published at _dmarc.<domain>, the antispoofing protection interpolates if it does not exist. That is, if the domain in the From: address does not publish a DMARC record, what would it say if it did?

It has also been a few months since we first started rolling out Safety Tips to Office 365 customers (everyone will get them by mid-November). Since that time, the question that has arisen the most is this: Why am I getting a red Safety Tip in my email? This is a legitimate sender!

Note: the above is inserted into the HTML part of a message. If there is no HTML, the text is added as plain-text.

For the search engines, that's "This sender failed our fraud detection checks and may not be who they appear to be." .

Figuring out the base conditions of why it was stamped

This red safety tip gets stamped when a message fails a spoofing check. Domains that are provisioned with an organization are spoofed are known as self-to-self spoofing (or intra-org spoofing), and is applied to all customers within Office 365. Domains that are external to an organization (e.g., google.com) are known as cross-domain (or external domain) spoofing. This is only available to Advanced Threat Protection and E5 customers of Office 365.

There are 2 common reasons why a "legitimate" message gets a red Safety Tip about a message failing fraud detection checks when the From: domain and To: domain are the same, and 1 less common reason:
.

  1. The sender is sending emails from an unauthorized source that is sending as your domain (all customers), or an external domain (ATP customers) in the SMTP MAIL FROM, but not in the domain's SPF record, and is also not signing the emails with DKIM
    .
    This occurs when a department within your organization or the sending domain's organization spins up a mail server, or outsources to a Software-as-a-Service provider, who sends email as your domain but they are not in the SPF record. To determine this, open up the message headers and look at the Authentication-Results header. Suppose the service that is sending emails as you (or in the example below, contoso.com) is HRnotifications.com and they are sending from the IP 1.2.3.4:

     Authentication-Results: spf=fail (sender IP is 1.2.3.4)
  smtp.mailfrom=contoso.com; dkim=none (message not signed)
  header.d=none; dmarc=none action=none
  header.from=contoso.com; compauth=fail reason=001

    In this case, the SMTP MAIL FROM is contoso.com, and the From: domain is also contoso.com. Contoso.com also does not publish a DMARC record. A case like this will almost always get a red Safety Tip and mark as spam because even though a DMARC record is not published, it would have failed had it been.

    I say "almost always" because Office 365 does suppress the Safety Tip and mark as spam action if we have enough reputation data to determine that this is a valid message. It works most of the time, but not for small senders, nor for senders with insufficient reputation, or senders with poor reputation.

    The last tag in the Authentication-Results header is the compauth property. This stands for composite authentication and is Office 365's combined SPF, DKIM, DMARC, and all other internal results. If a message fails explict authentication (DMARC quarantine/reject), the reason code will be 000. If a message does not publish a DMARC record and Office 365 has no valid signals on the message, the reason code will be 001 (other codes are reserved for internal use, but a failed message always starts with 0xx). The compauth result is only stamped for ATP customers.

    Even if the domain does not have an SPF record, the antispoofing check will still apply.

     Authentication-Results: spf=none (sender IP is 1.2.3.4)
  smtp.mailfrom=contoso.com; dkim=none (message not signed)
  header.d=none; dmarc=none action=none
  header.from=example.com; compauth=fail reason=001

    .

  2. The sender is sending emails that pass SPF or DKIM, but neither the domain that passes SPF nor the domain the passes DKIM aligns with the From: domain
    .
    Just passing SPF or DKIM is insufficient to suppress the Safety Tip or mark as spam. For example:

     Authentication-Results: spf=pass (sender IP is 1.2.3.4)
  smtp.mailfrom=HRnotifications.com; dkim=pass (signature was valid)
  header.d=HRnotifications.com; dmarc=none action=none 
  header.from=example.com; compauth=fail reason=001

    Even though this passed SPF and DKIM, HRnotifications.com != example.com. One of the SPF-passing or DKIM-signing domains must align with example.com. If it doesn't, a Safety Tip will be inserted unless Office 365 determines that the sending IP or domain has enough reputation to suppress the Safety Tip.

    Once again, the compauth has failed with a reason code of 001 (implict auth failure). Self-to-self spoofing does not yet stamp this code, and neither does non-ATP customers in Office 365. However, you can still figure it out by looking at the Authentication-Results (since compauth=fail reason=001 only applies to ATP customers, cross-domain spoofing is not applied to non-ATP customers either).
    .

  3. [In process of being fixed] The sender publishes a DMARC record, but Office 365 had a DNS lookup failure and DMARC TempError'ed
    .
    A corner case we are working on fixing is when a message ought to have passed DMARC, but didn't because of a DNS lookup failure:

     Authentication-Results: spf=pass (sender IP is 1.2.3.4)
  smtp.mailfrom=HRnotifications.com; dkim=pass (signature was valid)
  header.d=notifications.example.com; dmarc=temperror action=none
  header.from=example.com

    In this case, DMARC should have passed by aligning on the DKIM domain with the From: domain, but it couldn't because of the DNS lookup failure. A case like this may fail the antispoofing check and insert the red Safety Tip.

    We are working on this to treat it as a DMARC BestGuessPass and supersede the TempError.
    .

Common scenarios when this occurs

When do the first two occur most often?

The IETF has published RFC 7960 - Interoperability Issues between DMARC and Indirection Email Flows. This document describes all the times when a valid message fails DMARC.

Since Office 365's Exact-Domain antispoofing check is similar to DMARC, it is subject to some of the same problems. The difference between regular DMARC and Office 365's antispoofing checks is that our checks will try to figure out automatically that the sender, though failing authentication, is legitimate so you don't have to do anything; whereas DMARC makes you explicitly configure things to make it work. Office 365 doesn't always figure it out automatically so the more you configure it, the lower the chances that legitimate email will get marked as spam.

RFC 7960 was written by some smart people, and have done a great job to inventory as many different scenarios that are possible. The ones that I see the most often are the following, along with their solutions (I discuss this more in https://aka.ms/AntispoofingInOffice365):
.

  1. A third party is sending email, on your domain's behalf, but is not in your SPF record or DKIM-signing as your domain
    .     This constitutes the majority of cases. Even we here at Microsoft see this all the time. Not a week goes by where someone doesn't ping me asking why they are getting the red Safety Tip. It's because Microsoft has clamped down on spoofing because the problem of exact-domain spear phishing is so prevalent.The fix for this, either:
    .
    a)  Add the sender's IPs to your SPF record, or add the include:<their domain> to your SPF record. See Set up SPF in Office 365.
    .
    OR
    .
    b)     Have the sender DKIM-sign as your domain. This requires them giving you to publish a public key at a selector you specify, e.g., myselector, and then publishing the key at myselector._domainkey.<yourdomain>.
    .
    OR

    c)  Add the sender's IPs to your IP Allow list, see Configure the Connection Filter Policy in Office 365.
    .
    OR
    .
    d)     Create an Exchange Transport Rule (ETR) allow rule:- Add the sender's IPs to an ETR Allow rule- Or, if you want to be more secure, add the sender's IPs + the sender domain to an ETR Allow rule
    - Or, if you want to impress your friends, create an ETR Allow rule that looks for "spf=pass" in the Authentication-Results header and the sending domain in the envelope is the domain sending on your behalf (e.g., HRnotifications.com). Alternatively, you look for "dkim=pass (signature was verified) header.d=<signing domain>" in the Authentication-Results header. You would use this ETR in the case that the sending domain passes SPF or DKIM, but is not the same as your domain (i.e., they are sending on your behalf and authenticating, but not aligning)Both option (c) and option (d) will skip filtering and suppress the Safety Tip. End user Safe Senders will skip filtering but will not suppress the Safety Tip.See Transport rules in Office 365.
    .
    OR
    .
    e)     Use the Set-PhishFilterPolicy to specify senders who are allowed to send to you on your behalf, or an external domain that is sending spoofed email that you think is legitimate. See Set-PhishFilterPolicy. This is only available to Advanced Threat Protection customers.
    .

  2. You are sending to a mailing list which is modifying and replaying the message back into your environment
    .     This occurs with the following routing path:

     Office 365 -> Mailing list (pass SPF, DKIM, DMARC... then modify the message) -> Office 365

    This is one of the hardest ones to solve. Section 4.1.3 of RFC 7960 talks about this. If you are in control of the mailing list, there are a couple of good options you can make to your mail server:

    a) Set it up to not modify the message (not the most popular option)

    b) Rewrite the From: address so that it doesn't fail alignment

    Suppose the message you send looks like this when sending out:

    From: Example User <example@contoso.com>
    To: Mailing List <list@mailingList.org>

    When it goes to the list and gets replayed, it could be rewritten like this:

    From: Example User via Mailing list <list@mailingList.org>
    To: Example User <example@contoso.com>
    Reply-To: Example User <example@contoso.com>

    Not everyone in the email industry likes this solution, but I think it's pretty good if the list server is under your control.

    I talk more about this here: https://blogs.msdn.microsoft.com/tzink/2017/03/22/a-tip-for-mailing-list-operators-to-interoperate-with-dmarc-to-avoid-failures/. It is strongly recommended that you pursue this path when possible.

    c) Submit it as a false positive to Office 365 and we'll see if we can add it to our list of "Do not enforce antispoofing checks from these senders" list. This is a manually maintained list and we only add mailing lists to it, and it takes a long time to update. But if the list is not under your control, this is an option.
    .

  3. You are forwarding email into and out of Office 365 and modifying the message. Sometimes on purpose, but sometimes not.
    If your email path goes like this:

     Internet -> Office 365 -> on-prem mail server -> Office 365

    If the on-prem mail server modifies the message, it could fail antispoofing. This occurs if you are running Exchange on-prem which modifies messages (see here for details), or you have another appliance that inserts footers into a message or does some other modification.

    If you are running Exchange, the solution for this is to set up connectors so that the headers we stamp on the first pass through Office 365 are respected when you relay it back into the service. If you are not running Exchange, you will have to set up a few ETRs to respect the original spam or non-spam verdicts and route the message accordingly.

Those are the ones I see the most for customers of Office 365.
.

Sometimes this shows a tip and other times it doesn't!

That's expected. Our backend systems make an attempt to figure out if the sending pattern is legitimate or not. These are dynamic and can change.

For best results, you should authenticate your email and ensure that the authenticated domain aligns with the From: address.

Can I create a Domain Allow (SFV:SKA) or can users create safe senders (SFV:SFE) to suppress the tip?

No, those won't work (or, they may work but not in every case). And you wouldn't want them to, either. Domain Allows are string matches, which means that any spammer can spoof that sending domain or address in the From: address. The message will bypass regular filtering, but it will still get the safety tip to indicate that the sender did not pass fraud detection checks.

If you want to suppress the tip, see the instructions above.

What about disabling Safety Tips?

As a last-ditch attempt to solve the problem, you may be tempted to disable Safety Tips altogether. My advice is this - Don't do it!

Safety Tips are like a seat belt in your car. If it's too tight, or sits too high, or scratches you, the solution is to adjust it or buy one of those thingies that adds padding to it. The solution is not to drive without it.

Safety Tips helps protect you against spoofing, but it also helps defend against phishing (another red safety tip), to tell you when a sender is trusted, when you've skipped filtering due to IP Allows or Safe Senders, and other types of impersonation (coming soon in 2017). So by turning off Safety Tips, you've disabled all that protection.

You are much better off working through the issues about by inventorying IPs, getting proper DKIM signing,  fixing mailing lists, and making sure connectors are set up properly. That ensures you have the best protection possible. It is a little inconvenient (it's inconvenient here at Microsoft, too) but it is worth the effort.

But if you must turn off Safety Tips, you can do that here: Enable or disable Safety Tips in Office 365.
.

Conclusion

Hopefully this article helps you figure out why a message has that red Safety Tip inserted above. We do our best to make sure we only insert it in cases where we think the email is fraudulent, but sometimes legitimate email gets the tip. Fortunately, you can fix it yourself by following the steps above.

If you have questions, just let me know in the comments.

Comments

  • Anonymous
    November 29, 2016
    "Just passing SPF or DKIM is insufficient to suppress the Safety Tip or mark as spam."I'm sorry, but either I don't understand the semantics or this is wildly off-base. Who is Microsoft to decide what is legitimate or not if our SPF record says it should pass? That's out of control! Part of putting an SPF Record in place is to tell recipient Domains what is legitimate and what is not. You're effectively nulling our SPF record based on that example.
    • Anonymous
      November 29, 2016
      Just passing SPF or DKIM is insufficient if the domain that passes does not align with the From: address. After all, you wouldn't want a phisher setting up his own domain, publishing SPF records for it, signing with DKIM for it, yet putting your domain in the From: address... and having this suppress antispoofing checks.The domain that passes SPF or DKIM must align with the domain in the From: address for it to suppress the Safety Tip (unless we have other sender reputation data or configuration settings such as IP Allow rules).
      • Anonymous
        December 13, 2016
        Thanks for the response.
  • Anonymous
    December 08, 2016
    "You are sending to a mailing list which is modifying and replaying the message back into your environment” (Office 365 -> Mailing list (pass SPF, DKIM, DMARC... then modify the message) -> Office 365).If the mailing list is an on premise Microsoft Exchange Server Distribution Group under your control; can the “From:” address be rewritten and a “Reply-To:” inserted? Microsoft Exchange doesn’t have this option as far as I can see.
  • Anonymous
    December 15, 2016
    Thank you for helpful article.I highly recommend to include warnings verbatim in article. In this case it is “This sender failed our fraud detection checks and may not be who they appear to be”.This article will be a much help for people who searched that words in Google/Bing.
  • Anonymous
    February 02, 2017
    Thank you tzink for this very useful article.Further to my comment above regarding an Office 365 -> 3rd Party Mailing list -> Office 365 scenario: Microsoft Exchange Server Distribution Groups cannot be configured to rewrite the From: address. Therefore Microsoft Exchange Server as a 3rd party mailing list can only be compatible with Office 365 if Exchange Server does not modify the message body, thereby passing the original Office 365 DKIM signature and alignment.However, email sent from custom domains hosted in Office 365 do not appear compatible with antispoofing protection either. By default Office 365 DKIM signs all outgoing messages with d=tenant.onmicrosoft.com (initial domain) and therefore fails DKIM alignment.Is there official documentation explaining what Office 365 administrators must do for custom domains to be compatible with antispoofing protection including details of the side effects of signing with d=tenant.onmicrosoft.com?
  • Anonymous
    March 20, 2017
    Tzink"Submit it as a false positive to Office 365 and we’ll see if we can add it to our list of “Do not enforce antispoofing checks from these senders” list. This is a manually maintained list and we only add mailing lists to it, and it takes a long time to update. But if the list is not under your control, this is an option."What is the process for getting a mailing list server on this list? Can we simply point admins of mailing list servers so they can request this themselves? In Higher Ed this is a big problem because of the wide use of mailing list servers and it's causing lots of complaints and I don't feel the burden should fall on us to submit these requests every time a faculty member complains about these safety tips on their mailing list emails.
    • Anonymous
      March 22, 2017
      Best way is to open a support ticket requesting it. We don't add every mailing list, though.
      • Anonymous
        May 24, 2017
        Has any progress been mae by Microsoft to improve thier identification of mailing lists so they don't get stamped by these messages? It's becoming a bigger and bigger problem, at least for us.
  • Anonymous
    May 02, 2017
    Thank you Terry, that is really interesting !When you say "Just passing SPF or DKIM is insufficient if the domain that passes does not align with the From: address."Are you talking about subdomain alignment or main domain alignement ?Case 1 - main domain alignment :Authentication-Results: spf=pass (sender IP is 1.2.3.4) smtp.mailfrom=ga.example.com; dkim=pass (signature was valid) header.d=b.example.com; dmarc=none action=none header.from=b.example.comCase 2 - subdomain alignment :Authentication-Results: spf=pass (sender IP is 1.2.3.4) smtp.mailfrom=ga.example.com; dkim=pass (signature was valid) header.d=ga.example.com; dmarc=none action=none header.from=ga.example.comBecause in both cases, main domain is align for SPF, DKIM and FROM... but not subdomains !
    • Anonymous
      May 12, 2017
      It has to be a relaxed alignment, that is, the organizational-domain of the header.from must match the organizational-domain that passed SPF or DKIM. Both of your examples are fine.
  • Anonymous
    May 30, 2017
    The comment has been removed
    • Anonymous
      June 08, 2017
      We use sender/recipient reputation when making checks and that can affect delivery and stamping the red safety tip.
  • Anonymous
    June 06, 2017
    Thanks for this article, it was more helpful than calling support. I ran into the temperror this past week and opened a case with support. Support recommended changing our spf records and adding the senders IP/domain to our allowed senders in EOP. Can you provide a status update on the “We are working on this to treat it as a DMARC BestGuessPass and supersede the TempError.”?Also, how do you go about submitting the false positive? When I contacted support they seemed to be unaware of this option.
    • Anonymous
      June 08, 2017
      a) I don't have an update on the TempError overriding, it's currently moved down the priority stack.b) Keep pushing on Support, but this blog post is your best resource for resolving this. Fraud checks are complicated, and Support doesn't necessarily have the background required to troubleshoot.
  • Anonymous
    June 13, 2017
    Hi Terry,Thank you so much for the sharing. This is our guide when we trouble shoot the fraud detection issue. I want to consultant one scenario. Is there any discussion that the incorrect clock on the sending email server is a factor in the fraud detection firing?Recently, we got a escalation from customer that their legitimate emails were added safety tip. Based on checking, the SPF=pass, DKIM=none, dmarc=none, but the P1 and P2 are different. In addition, It has only ever happened when the clock of said mail server was set in the future.so I am not sure it is the P1/P2 caused the safety tip?or the P1/P2 and incorrect clock in the server contribute caused it?
    • Anonymous
      June 14, 2017
      The incorrect clock makes no difference.P1/P2 being different aren't necessarily related, what makes the difference is if the P2 From: matches the recipient domain. P1/P2 being different only matters if it doesn't authenticate, or align with each other. See http://aka.ms/AntispoofingInOffice365
  • Anonymous
    September 20, 2017
    The comment has been removed
    • Anonymous
      December 16, 2017
      Raise for Escalation with 2 or more samples. (Attach headers & EOP trace)
  • Anonymous
    December 09, 2017
    This appears to have broken in the past few weeks.We have had numerous reports of businesses sending emails through a 3rd party, from and to their own domain name, with correct SPF records, and full alignment, and they are still seeing the message.
    • Anonymous
      December 16, 2017
      Like it mentioned above, passing only spf doesn’t resolve the issue.Make sure you have dmarc inplace and it can pass, also make sure From address is valid format.Try these 2 and if after fixes these 2 still failed, then either post more details or create support ticket. We will help you diagnose.