"sign-in frequency" every time not working as expected and described.

Update after some more testing.

Recap:

We have the following setup in a test-tenant with standard settings and required p2 licenses.

• PIM managed groups with only eligible memberships
• the configuration requires "Authentication context"
• The authentication context is custom made for this use case and linked with, for now, 1 Conditional Access policy.
• The conditional access policy is configured to require phishing resistant MFA, and Sign-In Frequency is set to "Every time". The policy is scoped to "all users"

The goal is to require MFA every time a user activates membership to one of those PIM managed groups, as those provide access to administrative permissions.

The expected behaviour, based on the Microsoft description is that a user is required to provide MFA for the activation if there is more than 5 minutes between those activations.

Updated experience after more testing:

The actual experience is very unpredictable:

• Sometimes no MFA is requested, even with the first logon if the user activates their membership within 5min after logging on to the Azure portal,
• Sometimes re-authentication with if MFA is requested as expected,
• Most of the times MFA is requested the first time, but succedent activations don't require re-authentication. It can happen after 1 hour those do, or 20 min, or... And in addition is the behaviour very different from browser to browser.

The Authentication Context is only used for this purpose.

The tests were done in a tenant with no other active CA policies in place.

The device is not joined to the tenant.

The account which is logged on to the device is not the account used to logon to the tenant.

Due to the unpredictability, we can't provide steps for the reproduction of the issue, as it is also stil not clear in which circumstances the behaviour is not as expected or described. I also assume this behaviour we experience is also not "as designed".

Thanks for the advice and feedback!