Discrepancy between Conditional Access legacy authentication impact report and Sign-in logs

Question

Discrepancy between Conditional Access legacy authentication impact report and Sign-in logs

Yash Junghare 0

Description:

We are in the process of implementing a Conditional Access policy to block legacy authentication protocols across our tenant.

To evaluate the impact before enforcement, we used Report-only mode and reviewed the “View report-only mode impact” under the Conditional Access policy targeting legacy authentication.

The Graph visualization in the Conditional Access policy impact report indicates that multiple users will be impacted by the policy. However, when we perform a detailed investigation through Sign-in logs, we see zero sign-ins using legacy authentication protocols.

Steps We Followed to Investigate:

Enabled Report-only mode on the Conditional Access policy blocking legacy authentication protocols.

Reviewed the impact report which lists several user accounts expected to be affected.

Navigated to Microsoft Entra Admin Center → Monitoring & health → Sign-in logs.

Filtered by:

User sign-ins (interactive)

  **Client App = Legacy authentication clients** (IMAP, POP, SMTP, MAPI over HTTP, Exchange ActiveSync, Autodiscover, etc.)
  
     **Status = All (Success and Failure)**
     
     No matching sign-ins were found during the same timeframe shown in the impact report.

What We Need Help With:

Clarification on how the Graph impact report determines legacy authentication usage.

Why is there a discrepancy between the impact report and the actual sign-in logs?

Is the impact report using outdated or cached data?

How can we be confident in the accuracy of the report before enforcing the block?

1 answer

Your answer

Answer 1

Kancharla Saiteja 4,195 Microsoft External Staff Moderator

Hi @Yash Junghare,

Based on your query, here is my understanding: You would like to block legacy authentication and check the logs of legacy authentication.

I believe you might have configured conditional access policy as required and can also be verified using the steps provided in the document: Block legacy authentication with Conditional Access.

To check the sign in logs for legacy authentications, please follow the steps in this document: Identify legacy authentication use.

If you would like to use workbook for legacy authentication you go through this document: Sign-ins using legacy authentication workbook.

If you are unable to view any impacted sign in logs, you need to make sure you follow the steps:

Sign in to the Microsoft Entra admin center as at least a Reports Reader.
Browse to Entra ID > Monitoring & health > Sign-in logs.
Add the Client App column if it isn't shown by clicking on Columns > Client App.
Select Add filters > Client App > choose all of the legacy authentication protocols and select Apply.
Also perform these steps on the User sign-ins (non-interactive) tab.

Make sure you change the timeframe is properly selected and make sure you check the non-interactive sign in as well.

For a better view you can also view reports in insights and reporting of conditional access policy. This will give you more information on the logs and provide you better understanding.

I hope this information is helpful. Please feel free to reach out if you have any further questions.

If the answer is helpful, please click "Accept Answer" and kindly "upvote it". If you have extra questions or if this answer does not help you, please click "Comment".

Kancharla Saiteja 4,195 Reputation points Microsoft External Staff Moderator

2025-05-08T08:44:45.5+00:00

Hi @Yash Junghare,

Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Kancharla Saiteja 4,195 Reputation points Microsoft External Staff Moderator

2025-05-09T09:52:36.3333333+00:00

Hi @Yash Junghare,

Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Yash Junghare 0 Reputation points

2025-05-15T09:42:43.6933333+00:00

Hi Team,

Thank you for the detailed explanation and the links to relevant documentation.

However, I’d like to clarify that the “Block legacy authentication” Conditional Access policy was not created by me — it was automatically applied by Microsoft in report-only mode as part of the Secure Future initiative. I have not made any manual changes to the policy.

Here’s the issue I’m encountering:

When I view the Microsoft-managed Conditional Access policy impact report in the Azure Portal, I can see that several sign-ins are marked as potentially impacted.

But when I check the Sign-in logs in the Microsoft Entra admin center (using all the filters you mentioned, including both interactive and non-interactive sign-ins, and selecting legacy client apps), the result is zero entries — no sign-ins using legacy authentication.

I’ve attached screenshots from both the Conditional Access policy impact report and the Entra sign-in logs for comparison.

This inconsistency is making it difficult to determine if there are actual legacy auth usages or if this is a false positive in the policy impact report.

Would appreciate your insights on why there’s a discrepancy between these two data sources and whether this might be due to how Microsoft is interpreting or simulating legacy auth in the report-only mode.

image(1).png

image.png

Thanks,

Yash
Kancharla Saiteja 4,195 Reputation points Microsoft External Staff Moderator

2025-05-15T11:46:56.0966667+00:00

Hi @Yash Junghare,

Based on the screenshots I see the conditional access policy has not applied for many of the user accounts. The policy impact graph says not applied for all 100% sign-in's. Here is the information on the same: Policy evaluation results.

This confirms that the policy is not applied and that's the reason you are not able to view any legacy authentications.

I hope this information is helpful. Please feel free to reach out if you have any further questions.

Share via

Discrepancy between Conditional Access legacy authentication impact report and Sign-in logs

1 answer

Your answer