Microsoft Entra ID Protection Weekly Digest reports drastically higher number of risky sign-ins than Microsoft Entra ID Identity Protection

Question

The number of "New risky sign-ins detected (in real-time)" reported in the Microsoft Entra ID Protection Weekly Digest for my tenant has jumped from 78 last week to 8498 this week.

I was of course alarmed and tried to investigate where this exorbitant number might originate.

However in the Microsoft Entra admin center I find no trace of this drastic rise in risky sign-ins.

The Identity Protection > Report > Risky sign-ins page shows the usual peaceful trickle of <10 events per day, all false positives as usual.

Where would I look for the source of that alarming number in the weekly digest mail?

Answer 1

Hello @Tilman Schmidt ,

You can drop the investigation because this is an Expected behavior. The real-time data displayed in the portal is dynamically calculated, meaning the numbers in the weekly digest may not exactly match what you see in the portal at a later time. The real-time detection process analyzes signals directly from the request pipeline, while the weekly digest numbers are derived from raw signals rather than aggregated data.

Additionally, if any sign-ins were incorrectly flagged as risky (false positives), Microsoft's machine learning algorithms may have automatically remediated or dismissed them. As a result, these sign-ins would no longer appear in the portal.

That's why you'll observe such differences when compares with weekly email and Azure Portal data.

I hope this information is helpful. Please feel free to reach out if you have any further questions.

Answer 2

@Tilman Schmidt , you figured out the correct answer yourself in your reply on Apr 4, 2025, 2:41 a.m., but I'll post it here as a separate answer.

TL;DR: Some risky sign-in (real-time) events may have a Risk State = none, and the only way to view those is to deselect all of the Risk State filters.

All of the gaslighting answers from Microsoft employees are wrong; "risky sign-in (in real-time)" events that were immediately auto-remediated or that were dismissed shortly after detection are still visible in the Entra admin center Identity Protection "Risky sign-ins" blade when the filters are set correctly.

Explanation:

The events contributing to the “New risky sign-ins detected (in real-time)” count in the weekly digest are those whose Risk level (real-time) = Low / Medium / High. The values of Risk State could be anything (At risk, Confirmed compromised, Confirmed safe, Dismissed, Remediated) or nothing. The last word here is critical.

The natural assumption is that selecting all available Risk State values will display all risky sign-ins. However, this is not the case. Sign-ins whose Risk State is "None" are excluded whenever any (or all) Risk State filter value is selected. Misleadingly, there is no "None" option available in the filter. So, the only way to view sign-ins whose Risk State = none is to deselect all of the Risk State filters.

Once you do that that and also filter the Risk level (real-time) to include all values (Low, Medium, High) and you should see all the events the digest included in its count.

It may also be helpful to customize the columns and enable display of the “Risk level (real-time)” column. This doesn't affect filtering, but since you're filtering on that column, it's useful to actually see it column.

If anyone from Microsoft reads this, I'd like to request that they update the UI of the Risk State filter to include "None" as a selectable value. And it would also be nice to have the “Risk level (real-time)” column shown by default.