Forum Discussion
Dealing with high number of failed log on attempts from foreign countries utilizing Exchange Online
- Dec 21, 2018
Not sure if any one has seen this. There is a new tool for your basket. This has helped us greatly.
A couple of months ago Microsoft released to preview and then has pushed forward 'Authentication Policies'.
These authentication policies are processed prior to being passed to AAD or ADFS saving the failed login against the account
And yes this can be applied to individual or small groups to test first (just remember to wait to assure the policy is applied to the user in question before calling it good or not)
See "https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/disable-basic-authentication-in-exchange-online"
Basic outline
Assure you have modern authentication enabled for your organization
Create an authentication policy blocking basic auth for pop, imap and such (The biggest one we were seeing was imap)
If you have any user or service accounts that requires basic auth for any of the protocols you are disabling in the previous policy, create a second policy allowing the protocols
If you have any users that utilizing pop, imap or any other method you determine don't need basic authentication, get them migrated to some other form of client app or access
If there are any accounts that absolutely require basic auth (ie we have a ticketing system that utilizes imap with basic auth to connect to a specific mailbox), make note of them to exclude in your query for users to apply your restricted policy to
Query for and apply unrestricted policy to service account or user that requires the basic auth for the protocols disabled by the restricted policy
Query for and apply restricted policy to the majority of your users
Apply restricted policy as global default (for new users)
Either wait 24 hours for it to be applied or touch a user property on the user and wait approximately 30 minutes.
Note, the below worked for me. Make sure you research and adjust for your own needs. I take no responsibility for what you do to your environment. These are only examples
Exchange Powershell commands used
connect-exopssession -UserPrincipalName {exchangeonline admin}
New-AuthenticationPolicy -Name "Block_Basic_Auth_Selective”
{Blocks basic auth for imap, pop, smtp but allows for things like activesync}
(Adjust according to your needs)
Set-AuthenticationPolicy -Identity “Block_Basic_Auth_Selective” -AllowBasicAuthActiveSync -AllowBasicAuthAutodiscover -AllowBasicAuthImap:$false -AllowBasicAuthMapi -AllowBasicAuthOfflineAddressBook -AllowBasicAuthOutlookService:$false -AllowBasicAuthPop:$false -AllowBasicAuthReportingWebServices -AllowBasicAuthRest -AllowBasicAuthRpc -AllowBasicAuthSmtp:$false -AllowBasicAuthWebServices -AllowBasicAuthPowerShellNew-AuthenticationPolicy "Allow_Basic_Auth"
(Adjust according to your needs)
Set-AuthenticationPolicy -Identity “Allow_Basic_Auth” -AllowBasicAuthActiveSync:$true -AllowBasicAuthAutodiscover:$true -AllowBasicAuthImap:$true -AllowBasicAuthMapi:$true -AllowBasicAuthOfflineAddressBook:true -AllowBasicAuthOutlookService:$true -AllowBasicAuthPop:$true -AllowBasicAuthReportingWebServices -AllowBasicAuthRest -AllowBasicAuthRpc -AllowBasicAuthSmtp:$true -AllowBasicAuthWebServices:true -AllowBasicAuthPowerShellTo simplifiy things for my environment I manually set the users that required basic auth (I only had two)
set-user -Identity "User One" -AuthenticationPolicy "Allow_Basic_Auth"
To touch the user to make the policy get applied quicker
set-user -Identity "User One" -STSRefreshTokensValidFrom $([System.DateTime]::UtcNow)
For the rest of my users
$Users = Get-User -ResultSize unlimited | Where {$_.RecipientType -eq "UserMailbox" -and $_.AuthenticationPolicy -eq $null}
$users =$users.WindowsEmailAddress
$users | %{Set-User -Identity $_ -AuthenticationPolicy “Block_Basic_Auth_Selective”}
If you want to touch the users to apply policy quicker, since the query is already in memory
$users | %{Set-User -Identity $_ -STSRefreshTokensValidFrom $([System.DateTime]::UtcNow)}
Now the following command will apply the restricted policy as the global default. (Note, when I first implemented this, the unrestricted users did not have a policy applied and as such I thought they would have no policy applied, but once the default policy was applied to the global config, it affected the unrestricted unconfigured users.)Set-OrganizationConfig -DefaultAuthenticationPolicy “Block_Basic_Auth_Selective”
Remember, mileage will vary. Read everything you can find on Authentication Policy/iesFor us, for now, this has completely removed the issues we were having with illigitimate failed login attempts and account lockouts.
We ran into only the one issue mentioned above with the accounts that had no policy assigned and then the global policy being appliedRemember, it takes approximately 24 hours for the policy to be applied to a user unless one of the user's properties are modified
There is one thing I will mention, at this time, when this is applied, there is nothing logged for failed attempts that fall afoul of the blocked basic auth policy even in Azure Ad Sign-ins
-Gene
I know that this is a super old thread, but it's still on the front page of google when you're trying to figure out how to fix this issue, so I thought that I'd share what I've found after a ton of research and troubleshooting.
First off is Conditional Access policies, they require you to have an Azure AD Premium license, and they will not help with this. They only apply to modern authentication attempts, and this attack tends to leverage the old basic authentication method so that all the authentication attempts get tunnelled through the O365 servers before hitting your ADFS to prevent blocking them at the firewall. Additionally Conditional Access policies apply after the authentication attempt and lets you know if the password was correct or not, so the attacker finds out if they are successful and your accounts still get locked. If you can turn off basic authentication in your environment, then this will help prevent access to the apps, but it doesn't solve the root account lockout issue.
Next is extranet lockout, it can be useful, but as has been mentioned previously it's being bypassed for the most part by attackers by them using rate limiting. There's already some links to some good articles on here about it so that's all I'll say.
Then there is Custom Claims Rules on your ADFS servers. This looks promising, but outside of some guides for some very specific scenarios like locking down all authentication attempts external to your network, it was very hard to find documentation on how to do this or the claims language in general. I ended up leaving this option alone when I found Client Access Rules.
Client Access Rules, these are the best option that I've found so far. These are essentially O365 side firewall rules that apply on initial connection to O365 before the authentication is sent over to your ADFS server, and are based off of the external connecting IP of the attacker if you have an IP filter in place. They are only configurable via PowerShell at this time, and they are a fairly new feature, so they do have their limitations. For example some of the conditions don't work on all of the products yet, and certain condition and exception combinations you would expect to be able to use aren't available. The big issue I've run into with these is that there is no way to specify regions, only IPs and IP ranges, so I created a PowerShell script to automate grabbing the attacker's IPs from the ADFS logs and update the Blacklist Client Access Rule I have.
The script will parse your local ADFS logs, pick out the originating IPs for all the attacks, enumerate how many bad attempts you are getting per IP, then add any IPs that exceed a specified number of bad password attempts to a specified Client Access Rule to be blocked if it's not in the exclusion list in the script or the Client Access Rule. It also backs up the existing rule to XML before and after making changes, outputs a parsed version of the ADFS logs to CSV, records everything it's doing to a log file, and emails the log file to a specified email address if any errors occur in the script, or an IP gets added to the blacklist rule. Personally I have it setup to run every hour via a scheduled task on my ADFS server, and the number of attacks have dropped significantly since I implemented it. I keep meaning to make a blog to post this thing, but since I haven't gotten around to in in months I figure this would be a good place to share it.
You can find the script on GitHub here:
https://github.com/Khelbun/PowerShellHybridExchange/blob/master/BlacklistBruteForce.ps1
I've tried to explain how it all works in the code comments at the beginning and to put everything you would need to set in variables at the top of the script.
Here's the link to Microsoft's documentation on the Client Access Rules:
https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/client-access-rules/client-access-rules
- Patrick RooneyNov 08, 2018Copper Contributor
Hi,
i just wanted to ask a quick question , maybe a stupid one mind you , but i noticed this is looking for event id's 411, but when i view the events in event viewer the ip information is contained in event id 516 & 512 , the 411 is looks like below , will the script still work ok or do i need to amend it ?
i agree with you guys this is becoming a total pain to manage as we have a cpl of accounts that get hit throughout the day, sometimes locking them up out of hours etc...and as mentioned before it seem to be run through alphabetically and even with some random names that aren't within the company, i'm hoping this helps with the problem, but i do think MS should be doing more
Event ID 411:
Token validation failed. See inner exception for more details.
Additional Data
Activity ID: 00000000-0000-0000-0000-000000000000
Token Type:
http://schemas.microsoft.com/ws/2006/05/identitymodel/tokens/UserName
Error message:
Removed@goodreason.com-The user name or password is incorrect
Exception details:
System.IdentityModel.Tokens.SecurityTokenValidationException: s.voigt@ikmconsulting.co.uk ---> System.ComponentModel.Win32Exception: The user name or password is incorrect
at Microsoft.IdentityServer.Service.Tokens.LsaLogonUserHelper.GetLsaLogonUserHandle(SafeHGlobalHandle pLogonInfo, Int32 logonInfoSize, SafeCloseHandle& tokenHandle, SafeLsaReturnBufferHandle& profileHandle)
at Microsoft.IdentityServer.Service.Tokens.LsaLogonUserHelper.GetLsaLogonUserInfo(SafeHGlobalHandle pLogonInfo, Int32 logonInfoSize, DateTime& nextPasswordChange, DateTime& lastPasswordChange, String authenticationType, String issuerName)
at Microsoft.IdentityServer.Service.Tokens.LsaLogonUserHelper.GetLsaLogonUser(UserNameSecurityToken token, DateTime& nextPasswordChange, DateTime& lastPasswordChange, String issuerName)
at Microsoft.IdentityServer.Service.Tokens.MSISWindowsUserNameSecurityTokenHandler.ValidateTokenInternal(SecurityToken token)
--- End of inner exception stack trace ---
at Microsoft.IdentityServer.Service.Tokens.MSISWindowsUserNameSecurityTokenHandler.ValidateTokenInternal(SecurityToken token)
at Microsoft.IdentityServer.Service.Tokens.MSISWindowsUserNameSecurityTokenHandler.ValidateToken(SecurityToken token)
System.ComponentModel.Win32Exception (0x80004005): The user name or password is incorrect
at Microsoft.IdentityServer.Service.Tokens.LsaLogonUserHelper.GetLsaLogonUserHandle(SafeHGlobalHandle pLogonInfo, Int32 logonInfoSize, SafeCloseHandle& tokenHandle, SafeLsaReturnBufferHandle& profileHandle)
at Microsoft.IdentityServer.Service.Tokens.LsaLogonUserHelper.GetLsaLogonUserInfo(SafeHGlobalHandle pLogonInfo, Int32 logonInfoSize, DateTime& nextPasswordChange, DateTime& lastPasswordChange, String authenticationType, String issuerName)
at Microsoft.IdentityServer.Service.Tokens.LsaLogonUserHelper.GetLsaLogonUser(UserNameSecurityToken token, DateTime& nextPasswordChange, DateTime& lastPasswordChange, String issuerName)
at Microsoft.IdentityServer.Service.Tokens.MSISWindowsUserNameSecurityTokenHandler.ValidateTokenInternal(SecurityToken token)
Event ID 516:
The following user account has been locked out due to too many bad password attempts.
Additional Data
Activity ID: 00000000-0000-0000-0000-000000000000
User:
removed@forgoodreason.co.uk
Client IP:
212.38.173.4,52.97.135.253
nBad Password Count:
8
nLast Bad Password Attempt:
08/11/2018
Event ID 512:
The account for the following user is locked out. A login attempt is being allowed due to the system configuration.
Additional Data
Activity ID: 00000000-0000-0000-0000-000000000000
User:
removed@goodreason.co.uk
Client IP:
212.38.173.4,40.101.96.109
Bad Password Count:
8
nLast Bad Password Attempt:
08/11/2018great write up by the way eric ...
paddi - Dale PuumalaOct 25, 2018Copper Contributor
I downloaded your script and modified it to block the entire /24 for each IP address (bigger hammer). It works great. So great that...in less than 24 hours I reached a limit on the AnyOfClientIPAddressesOrRanges Property. I receive the error now: The length of the property is too long. The maximum length is 40960 and the length of the value provided is 44620.
Have you ran across that yet? Account lockouts are getting really old.
It would be wonderful if MS would just put in true geo-blocking, not Conditional Access, which only works after successful authentication.
- Daniel S. Gurrola IIOct 29, 2018Copper ContributorI am working on this very same thing and it has been nasty! I got to the exact same place as Eric albeit just a TINY bit differently, but exact same idea. Since most of these attacks are from countries we have no business with or executives traveling to we block out the IP's whole registered subnet.
I am working on adding the ability to get that data from http://who.is or elsewhere (I am curious about that Linux command Eric uses for GeoIP data). Adding this to the script will most definitely save me having to go look it up. I have 4 /8's and 2 /16 subnets in my lists now. so for now we are OK with the limit you ran into.
We are going to have to pony up for some form of token access or service like DUO which we already use for Remote Desktop Gateway MFA. I just need a way to authenticate people that SHOULD be authenticated not just usernames that CAN be. I'm looking for a way to allow our Execs to just not be bothered with this at ALL since it's only a matter of time until someone goes to China and then we are back to the drawing board! - Eric SobkowiczOct 26, 2018Copper Contributor
I haven't run into that issue actually. In fact when I was setting this all up and working with MS support I asked them if there was any limits to what can be put in the AnyOfClientIPAddressesOrRanges property of the Client Access Rule as I was worried about this happening, and they said that there were not. I've been running my script hourly since April and I so far have only 63 entries and it's still going strong. How many entries do you have in the rule now, and how are you putting in your /24 ranges? If you can send me your adjusted script and the values of your AnyOfClientIPAddressesOrRanges property on your rule I can take a look and see if I can figure out what's going on there.
- Dale PuumalaOct 26, 2018Copper Contributor
We set our threshold much lower than yours. We looked through our logs and even at 4 hits on an individual IP address they were still coming from other countries. So we set our threshold at 4 knowing that we increased the risk of a false positive and we would have to come back through and remove it and whitelist it after the fact. With that being said, in a matter of 18 hours we reached 1267 entries in our AnyOfClientIPAddressesOrRanges property. So the script worked well! Too well...
I'll send you a message on here if the attach file thing doesn't work with the full text of the modified script and you can see the section that I modified to convert to IPv4. I commented in it.
- Andy DavidOct 25, 2018MVP
Client Access Rules, Conditional Access, Block Basic Auth in Exo with this new powershell command. It's all getting too much frankly. We just need a solution in one place that can be easily managed and audited
- Eric SobkowiczOct 26, 2018Copper Contributor
Andy, I 100% agree that it should be something we already have as a simple GUI option for all O365 tenants, but unfortunately it doesn't appear to be something that they are interested in doing as people have been asking for it since they first put out O365 and they haven't yet implemented it.
I'm actually planning on seeing if I can make a Powershell script to create and maintain region blocking via a Client Access Rule as even with all the above protections in place we're still getting hammered with brute force attempts, and I can't find anything out there that does it currently. Hopefully I can find some spare time in the next month or so to make that.