Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community
Users can now check their sign-in history for unusual activity
Published Oct 17 2019 09:00 AM 58.3K Views

Howdy folks,

 

I’m excited to announce the public preview of Azure AD My Sign-Ins—a new feature that allows enterprise users to review their sign-in history to check for any unusual activity. As we discussed in a previous blog post, our team defends against hundreds of millions of password-based attacks every day.

 

The My Sign-Ins page empowers users to see:

 

  • If anyone is trying to guess their password.
  • If an attacker successfully signed in to the account from a strange location.
  • What apps the attacker tried to access.

Robyn Hicock, who managed this feature, wrote a guest blog post where she dives into the details on this update. You’ll find her blog post below.

 

As always, we’d love to hear any feedback or suggestions you may have. Please let us know what you think in the comments below or on the Azure AD feedback forum.

 

Best regards,

Alex Simons (@Alex_A_Simons)

Corporate VP of Program Management

Microsoft Identity Division

 

___________________________________________________________________________________________

Hi everyone!

I’m super excited to share details about the new My Sign-Ins tile found on the users Overview blade:

 

Azure AD My Sign-Ins 1.png

Just click the My Sign-Ins tile to display the location details of how an account was accessed.

 

Here’s an example where I successfully signed in to Office 365 on Windows 10 from Washington:

Azure AD My Sign-Ins 2.png

Successful sign-in

 

Most users should recognize their activity as being normal. However, if a user notices a Successful sign-in from strange location, browser, or operating system, an attacker may have gained access to the account. In this case, the user should change their password immediately and then go to the Security info page to update their security settings.

 

There is a chance of a false positive since the approximate location and map is based on the IP Address (we call this “IP Address Geolocation”). Mobile networks are especially hard to geolocate since they sometimes route traffic through distant locations. For example, if a user signs in on their phone from Washington, the location might show the sign-in coming from California. This is why it helps to check more details about the sign-in, such as the operating system, browser, and app to confirm if it’s actually bad activity.

 

Unsuccessful sign-in

 

An Unsuccessful sign-in, which shows no session activity, means that primary authentication (username/password) failed. This could mean that the user mistyped their password or an attacker was trying to guess the password. If it’s because an attacker was trying to guess the password (but was unsuccessful), then there’s no need for the user to change their password. However, this is a great reason for the user to register for Azure Multi-Factor Authentication (MFA), so even if the hacker eventually guesses the password, it won’t be enough to access the account. Based on our studies, accounts protected by MFA are 99.9 percent less likely to be compromised.

 

Azure AD My Sign-Ins 3.png

An Unsuccessful sign-in, which shows Session activity of “Additional verification failed, invalid code,” means that primary authentication (username/password) succeeded, but MFA failed. If it was an attacker, they correctly guessed the password but were unable to pass the MFA challenge—such as round tripping a code to a phone number or by using the Microsoft Authenticator app. In this case, the user should still change their password (since the attacker got it right) and go to the Security info page to update their security settings.

Azure AD My Sign-Ins 4.png

Filtering sign-ins

 

You can use the Search bar at the top to filter sign-ins by state, country, browser, operating system, app, or account. For example, below I filtered sign-ins in to the My Groups app:

Azure AD My Sign-Ins 5.png

Looking ahead

 

In the future, we’ll add This wasn’t me and This was me buttons. We’ll also highlight unusual activities detected with Identity Protection. This user feedback will help improve the accuracy of our risk detection systems. We do all of this already with the Recent Activity page for consumer Microsoft Accounts.

 

We’d love to hear your feedback and suggestions on the My Sign-Ins Public Preview before it becomes generally available. Please let us know what you think in the comments below or on the Azure AD feedback forum.

 

Thanks!

Robyn Hicock (@RobynHicock)

Senior Program Manager

Microsoft Identity Security and Protection team

28 Comments
Copper Contributor
This is an awesome feature for Enterprise customers like me! Security is the most important consideration when we decide to partner with a new organization. Any estimate on when this reaches GA? Great job @robynhicock!
Copper Contributor

It would be nice if the approximate location was shown in the closed event.  Showing Just "US" or [country] tells me nothing about the login.  If I see that a login event happened in Arizona vs Ohio, there is something wrong with that login.   Knowing that the occasional mobile login may cross borders. 

 

Lastly, if the IP address is a trusted location for the organization, does it also make sense to flag it or color code it? Is this planned with the AIP tie in?

Copper Contributor

Great additions @robynhicock 

 

[Agree with the above regarding the IP]

 

- Would be great to have the time zone listed, the location helps but time wise PST, CST etc.. would provide further insight.

- In reference to the session activity will there be an addition of the authentication method use? SMS or Authenticator App?

- Will GA cover other devices, such as Mac?

- The additions of the "wasn't me", will that incorporate a trigger to the IT admins and the account will be locked out? Interested to determine if there are any particular plans to block the account and how it would reference to the risky sign-on in AIP and conditional access.

- In the section where it states the user should still change their password, will SSPR need to be enabled for the enterprise?

 

Microsoft

@CryptoLulluby - Thanks! No date yet for GA because it depends on the feedback we get in Public Preview :)

 

@JoeTech - Good idea, thanks! We'll consider doing that for collapsed rows. Yes in the future we want to flag and color code unusual activities so users don't have to scroll and hunt for them. We wouldn't flag trusted locations though. Is there a reason you'd want us to?

 

@SSandz_ - Thanks! Yea the time zone is a good idea, I'll add that to our list. Yep we also want to add which authentication method was used. Mac should already be covered right now actually, does it not work for you? Once we add "This wasn't me" it will trigger a compromise recovery flow. The end user will have to prove their identity, change their password, and review their security info. If they finish that flow then the risky sign-in would be dismissed in the admin's Identity Protection report. Yes, SSPR would be needed. Thanks for the questions and feedback :)

Brass Contributor

Any idea where the Administration documentation is to enable the "new profile experience" ?  So far everything I find simply tells me to contact myself to enable;)

Copper Contributor

@robynhicock Users tend to know the ISP  they use but rarely their IP. We’re a CSP for small businesses and this is definitely part of the education plan for them to start watching logins/IP.  I have a cell phone on X carrier.  If it shows up on Y carrier, that may be a red flag.  Typical users we find login in 3 places: work,cell,home.  Anything outside of that raises flags, maybe not a red, but at least a yellow. If we can alert them that these are your work IPs, It’s that much better.  

Copper Contributor

That's Great feature for users for their security prospective. This feature will become more use full if their is option to get email notification on failure notification and new device login. 

Copper Contributor

Nice feature!

 

I'd like to see some integration with Conditional Access if possible. So that you can see if your sign-in hit any CA policy, for example when and why MFA was required. Can be usefull for users to understand what's going on in the back.

 

Copper Contributor

Great feature, with some remarks:

  • the label "This is your current session." seems to be placed at the first record in the list, which becomes wrong, as soon as you use the filter.
  • How can I show only unsuccessful logins in the list? You mentioned, that you can filter by state, but I am not sure what to type in.

Suggestion:

I know, that geolocation is often inaccurate, but have you thought of adding a whitelist or blacklist for regions, you are expecting logins from, or not at all?

Copper Contributor

@UW_ScottThe new profile experience can be enabled with the following steps.

 

 

  • Sign into the Azure portal as a global administrator or user administrator.
  • Browse to Azure Active Directory > User settings > Manage settings for access panel preview features
  • Under Users you can use the preview features for registering and managing security info – refresh, you can choose to enable for a Selected group of users or for All users.

More info in this post.

 

Jelle

 

Copper Contributor

@robynhicock  thanks for the prompt responses!  Will check on the Mac pieces :)

Microsoft

@UW_Scott - We're fixing that soon actually, so that you don't need to be in a preview to see the new My Profile experience :) But for now, @Jelle Revyn is correct (thanks)! The link is https://portal.azure.com/#blade/Microsoft_AAD_IAM/UsersManagementMenuBlade/UserSettings

 

@JoeTech - Really good points, thanks! We'll consider adding that.

 

@Deepak_ITTechPro -Thanks :) Yep our long term plan is to give admins the option to set up alerts. That way end users will be directed to check their sign-ins if we detect anything unusual. We do this in MSA today by alerting via SMS, alternate email, and the Authenticator app. We just want to get My Sign-Ins to GA first before we do the notifications though.

 

@Jan Bakker - Thanks! We've considered that but were thinking CA might be too much info for the typical end user. However all those details are available in the admin's sign-in report. 

 

@MatJo75 - Good catch about the "current session". I'll open a bug for that. Right now we don't have a good way to filter by just the unsuccessful logins, but before GA we want to add that capability. The search bar should be able to filter by state and country if you type the exact words, like "Washington" or "US". Let me know if that doesn't work for you. IT Admins can use Conditional Access to set up sign-in policies based on location. That's not something end users can do for themselves, but it's an interesting feature idea. Thanks!

Iron Contributor

Hi Robyn (robynhicock),

 

Tested Search:

Operating System - Part search works
Browser - Part search doesn't work (Actually "Chrome" OK, not "Google")
Approximate Location - Part search works
IP - Search doesn't work
App - Part search works
Account - Part search works
sign-in - Search doesn't work
Date - Search doesn't work

 

Also please need a Search 'Not X'. Example not Approximate Location US

 

What will help Users the most
1. Unsuccessful sign-in
2. Not Approximate Location X

Copper Contributor

Are the plans to have client access from App’s displayed? My users would like to see not just successful web auth’s

Microsoft

@David Taig - Thanks for testing it out! Yep we have some known issues with the Search functionality and want to improve that before GA. 

 

@Arran Goffe - I believe it should already show that. Do you have a specific app in mind?

Copper Contributor

Great feature, however; we are using an Internet Proxy from a renowned CA based vendor.

We are an europeing based company, with the proxy located near by, at the vendor subsidiary.

 

It appears you are looking up IP Addresses through WhoIs.com who will only give you the owners geo location, and not the geo location of the End Point. 

Consequently all sign-in's, through the Proxy, are registered as originating from US CA. Sign-in's bypassing the Proxy, as originating from europe, as expected.

 

I'm suggesting you look up IP Addresses in a reliable geo location database instead.

 

Best regards

Copper Contributor

This feature is good, but will be better If when we receive a request for authentication in Authenticator App, we can see the source of request of authentication.

 

Is there something in development related to this?

 

Thanks!

Copper Contributor

@robynhicock 

clipboard_image_0.png

Alerts not only for admin but user for as well when there is login from new device or IP Address. Can you pleases brief me about you said "We do this in MSA today by alerting via SMS, alternate email, and the Authenticator app."

Copper Contributor

You need to fix the Geo Location problem, or this feature will be totally useless.

 

Even though I'm Europe based, all my sign-in's appears as originating from US California.

Iron Contributor

Hi Robyn (@robynhicock) - Do you have an update coming soon to resolve the known issues and any enhancements from the requests above. Thanks David :cool:

Copper Contributor

Hi,

 

Is it possible the system can generate the sign-ins report regularly (e.g. weekly)? Then send it to every O365 users. Thx.

 

Samuel.

Microsoft

@IvanRafn - Thanks for reporting that! Work on this feature got delayed a few times to work on higher priority items, but we have picked it back up again. I'll bring up that geo location issue to our engineering team.

 

@Bikebrother - Yes that's in our roadmap to show more context in the Authenticator App notification :smile:

 

 

Microsoft

@Deepak_ITTechPro - Yep we want to eventually send alerts to AAD end users too. I meant we do this in MSA for consumer accounts. For example, this is one of the end user notifications from the Authenticator App when a password is changed: 

 

robynhicock_0-1594954169193.png

 

Microsoft

@David Taig - Yep we're working on it now! We've fixed those Search issues that you mentioned, and we're aiming to get this feature to GA by the end of July :happyface:

Microsoft

@Samuel_Ho - Hmm we don't currently generate weekly reports from My Sign-Ins. In the future we do want to notify end users if we detect unusual activity though.

Also if you're an admin for a tenant, you can view a much more detailed sign-ins report at https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/SignIns and that allows you to download and export the data:

 

 

Brass Contributor

Truly, AdminDroid is the most efficient way to create a weekly failed login report.  It is cost effective and also provides in the login failure report cases where the username is invalid on the tenant. This is handy because you can see the password spray attempt.

 

My issue with this is that I have a case where a user saw an unsuccessful login attempt without session data but it also trigged an MFA challenge. In this case the app was the Office 365 Management app on an Apple iPhone running the very old iOS 12 as shown below.  

Kathie-my-signins-figure-1-rev1.png Needless to say we have a robust CA policy in place for foreign locations and unusual travel, so regardless of the employee affirming the phantom MFA request they never would have gotten in.  However, my question is, why did the failed password attempt successfully raise an MFA challenge? Shouldn't this only happen after a successful password attempt?  

Copper Contributor

I know this is an older thread, but since we just now rolling out communications to our users about this feature, I was curious why this interface does not show SSO logins, or app logins using the same credentials?

Does that not seem like a huge gap in the reporting?

Version history
Last update:
‎Aug 19 2021 04:21 PM
Updated by: