Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community

Azure AD: sign-in risk calculation is wrong?

Brass Contributor

Hi all, 

I noticed these past weeks some weird logic in the way sign-in risk is calculated or handled. 
It especially impactful when MFA is enforced via an CA policy that is based on the users sign-in risk.

Some examples: 

Example 1:

User is suddenly active from Russia. 
No registered device
First time sign-in this month in Russia
no exclusions in the policy. 
2021-06-11 10_32_40.png

On the interrupted sign-in there are no CA policies applied. 
The next sing-in Russia on office365 shell has has a sign-in risk of none

Why do I find this weird?
User activity moves from Paris to Russia.
The sign-in with the interrupt does not lower the risk as it has been interrupted.
So the next sign-in in Russia should still have a risk factor as there has not been a completed mfa request.

Example 2: 
user is suddenly in Italy. 
Again no previous history in Italy. 
No azure ad joined device. 
Sign-in risk was considered none

2021-06-11 10_52_ Microsoft​ Edg.png


Why do I find this weird?
Again a user moved to a country where haven't seen any activity in the last month. 
There is no registered device in any sign-in log. 

Example 3:
User is suddenly active from Tunisia. 
Normal activity is France. 
The user first fails 2 times to sign-in as his primary authentication due to wrong password. 
3rd sign-in log he is interrupted as his device requires authentication. 
4rd sign-in log he FAILS on completing the MFA request
5th sign-in log the user signs-in using with none sign-in risk. 

2021-06-11 11_01_03png.png

Screenshot of the failed mfa request

2021-06-11 11_16_36- Microsoft​ Edge.png

 

Screenshot of the successful sign-in his risk2021-06-11 11_08_55-PMicrosoft​ Edge.png

 

Why do I find this weird?

This is the one that blows my mind the most.
Users go's to new country.
Fails 2 times on his password.
Then gets prompted with mfa but fails to complete authentication. 
Logicaly thinking you would thing that this would raise his sign-in risk. 
2 failures and 1 mfa prompt not completed. 
Next sign-in = 0 sign-in risk. 

 

As documented Microsoft states the following about risk calculations: 2021-06-11 11_23_48-What is risk_ Azure AD Identity Protection _ Microsoft Docs and 11 more pages - .png

But I believe my 3 scenario's have enough evidence that the risk state of these sign-ins should not have the state of none. Especially when previous attempt are failed. 

Having insights on how this "risk" is calculated would bring us a step further. 


Anyone who can share an opinion/insights on this? 

Kind regards

Louis

 

 

1 Reply

update:

Today I have another case of what is according to me wrong risk calculation or risk loss. 

 

2021-06-16 09_10_38-Microsoft​ Edge.png

 

1) the user signs-in on apple internet accounts, is interrupted and needs to confirm mfa.
Based on the logs the does not complete the mfa request.

2) the next sign-in log to apple internet accounts is a success.
Sign-in risk is suddenly "none".

 

Details of sign-in log 1
2021-06-16 09_12_36-details 1 Microsoft​ Edge.png

 

Details of sign-in log 2
2021-06-16 09_14_18-Policy details  - Microsoft​ Edge.png

 

 

It does not make sense to me that the user loses his risk after a failed mfa request according to the logs.