Share via

Sign-in logs show user logins failing but access is still granted

Sam Davidoff 1 Reputation point
2024-01-27T14:12:36.82+00:00

We have a conditional access policy set up to require users logging into to our Azure DevOps application to to reauthenticate every day. This is done via a policy that targets the Azure DevOps Resource and sets the session sign-in frequency to one day.

The policy is set to "On" and in the Entra ID sign-in logs we can see that it is applied to users who try to access Azure DevOps. But the access is not actually blocked/reauthentication required on a daily basis. In other words, notwithstanding that the policy is applied, user sessions to Azure Devops persist for (far) longer than 24 hours.

What's also strange is that, in the sign-in logs, these accesses are shown as "Failed" with a message

The refresh token has expired or is invalid due to sign-in frequency checks by conditional access. The token was issued on {issueDate} and the maximum allowed lifetime for this request is {time}.

Which makes sense. Excepts that the logins DONT actually fail.

We have similar refresh policies set up for other apps and they appear to be working.

Any suggestions on what could be going on here?

Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
20,645 questions
0 comments

5 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Andy David - MVP 145.6K Reputation points MVP
    2024-01-27T16:14:26.09+00:00

    You really can't target DevOps alone.

    You have to target the entre Azure mgmt API suite:

    https://learn.microsoft.com/en-us/entra/identity/conditional-access/concept-conditional-access-cloud-apps#windows-azure-service-management-api

    When you target the Windows Azure Service Management API application, policy is enforced for tokens issued to a set of services closely bound to the portal. This grouping includes the application IDs of:

    • Azure Resource Manager
    • Azure portal, which also covers the Microsoft Entra admin center
    • Azure Data Lake
    • Application Insights API
    • Log Analytics API

    Because the policy is applied to the Azure management portal and API, services, or clients with an Azure API service dependency, can indirectly be impacted. For example:

    • Classic deployment model APIs
    • Azure PowerShell
    • Azure CLI
    • Azure DevOps
    • Azure Data Factory portal
    • Azure Event Hubs
    • Azure Service Bus
    • Azure SQL Database
    • SQL Managed Instance
    • Azure Synapse
    • Visual Studio subscriptions administrator portal
    • Microsoft IoT Central

    Note The Windows Azure Service Management API application applies to Azure PowerShell, which calls the Azure Resource Manager API. It does not apply to Microsoft Graph PowerShell, which calls the Microsoft Graph API Also: Session timeouts work best when targeting users, not apps per se ,but if you target the Azure Service API , I would think it will work for you , so I would test that out:

    https://learn.microsoft.com/en-us/entra/identity/conditional-access/howto-conditional-access-session-lifetime#user-sign-in-frequency

    0 comments
  2. Sam Davidoff 1 Reputation point
    2024-01-27T21:10:47.6433333+00:00

    Interesting. So I tried both things you recommended and neither block DevOps' persistence. I set a CAP for 1 day sign-in frequency targeting the Azure DevOps and Windows Azure Service Management API, and I set a separate policy that sets the same 1 day sign-in frequency but for a certain group of users regardless of app. The result is the same. In the sign-in logs, the sign-in is recorded as "Failed" with the same error message above, CAP shows that all both policies (app-specific and user-specific) were applied. And yet the DevOps session marches on undisturbed... Has anyway successfully used Conditional Access to increase the sign-in frequency for DevOps?

  3. Andy David - MVP 145.6K Reputation points MVP
    2024-01-28T22:39:42.98+00:00

    So I was curious about this and it appears there are scenarios where a CA policy is enforced with Dev Ops:https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/change-application-access-policies?view=azure-devops#cap-support-on-azure-devops As I mentioned before the block works from what I have seen using Azure Mgmt as the target app, but I wonder if there is something being missed here

  4. James Hamil 23,216 Reputation points Microsoft Employee
    2024-03-05T23:07:20.7633333+00:00

    Hi @Sam Davidoff , if you're still having issues, can you please send me an email at "azcommunity@microsoft.com" with subject "ATTN: James Hamil" and your subscription ID? I can open a free support ticket for you.

    Best,

    James

  5. Sam Davidoff 1 Reputation point
    2024-03-08T20:03:06.3966667+00:00

    It appears that ADO currently does not respect sign-in frequency, but that is expected to change soon:

    https://github.com/MicrosoftDocs/azure-devops-docs/issues/13983

    0 comments