Forum Discussion

LouisMastelinck's avatar
LouisMastelinck
Brass Contributor
Jun 11, 2021

Azure AD: sign-in risk calculation is wrong?

Hi all, 

I noticed these past weeks some weird logic in the way sign-in risk is calculated or handled. 
It especially impactful when MFA is enforced via an CA policy that is based on the users sign-in risk.

Some examples: 

Example 1:

User is suddenly active from Russia. 
No registered device
First time sign-in this month in Russia
no exclusions in the policy. 

On the interrupted sign-in there are no CA policies applied. 
The next sing-in Russia on office365 shell has has a sign-in risk of none

Why do I find this weird?
User activity moves from Paris to Russia.
The sign-in with the interrupt does not lower the risk as it has been interrupted.
So the next sign-in in Russia should still have a risk factor as there has not been a completed mfa request.

Example 2: 
user is suddenly in Italy. 
Again no previous history in Italy. 
No azure ad joined device. 
Sign-in risk was considered none


Why do I find this weird?
Again a user moved to a country where haven't seen any activity in the last month. 
There is no registered device in any sign-in log. 

Example 3:
User is suddenly active from Tunisia. 
Normal activity is France. 
The user first fails 2 times to sign-in as his primary authentication due to wrong password. 
3rd sign-in log he is interrupted as his device requires authentication. 
4rd sign-in log he FAILS on completing the MFA request
5th sign-in log the user signs-in using with none sign-in risk. 

Screenshot of the failed mfa request

 

Screenshot of the successful sign-in his risk

 

Why do I find this weird?

This is the one that blows my mind the most.
Users go's to new country.
Fails 2 times on his password.
Then gets prompted with mfa but fails to complete authentication. 
Logicaly thinking you would thing that this would raise his sign-in risk. 
2 failures and 1 mfa prompt not completed. 
Next sign-in = 0 sign-in risk. 

 

As documented Microsoft states the following about risk calculations: 

But I believe my 3 scenario's have enough evidence that the risk state of these sign-ins should not have the state of none. Especially when previous attempt are failed. 

Having insights on how this "risk" is calculated would bring us a step further. 


Anyone who can share an opinion/insights on this? 

Kind regards

Louis

 

 

  • update:

    Today I have another case of what is according to me wrong risk calculation or risk loss. 

     

     

    1) the user signs-in on apple internet accounts, is interrupted and needs to confirm mfa.
    Based on the logs the does not complete the mfa request.

    2) the next sign-in log to apple internet accounts is a success.
    Sign-in risk is suddenly "none".

     

    Details of sign-in log 1

     

    Details of sign-in log 2

     

     

    It does not make sense to me that the user loses his risk after a failed mfa request according to the logs.

     

     

Resources