Azure ATP Service Account getting locked out

[Robren]

Hello.

For the past few weeks the service account we have configured in the Azure ATP portal keeps getting locked out by Domain Controllers. I am not sure why this would happen since the agent services on the DCs run under Local System/Service. I am assuming it has to do with some Powershell script running in the background, but I cannot determine the cause.

I've validated the password in the portal, and it is correct. I have restarted the agent services on each DC, and they all start up fine. Yet over night the account got locked out again.

What is using the service account to do work on DCs? How can this be troubleshot?

Thanks in advance,

Robert

[#2]

The sensor is using those credentials for various scenarios for authentication, for LDAP, for name resolution, for lateral movement mapping...
The thing is that if one of the sensors was using a wrong password,  it should have failed starting...

Are you using just a single set of credentials?

Idea:

create new set of credentials for AATP, and replace in the portal.

make sure not to disclose the credentials to anyone else.

After all sensors get synced with the new credentials, unlock the old account and see if it still locks out.

If it does, there is something other than AATP that is trying (and fails) to use this account, and you might want to trace who is it by increasing auditing in the DC.

[#3]

@Eli Ofek thanks for the idea. I tried it, and it didn't work. I created a brand new account, put is in the portal, and a few hours later the new account locked out. I will also note that the old account, which is no longer associated with the ATP console, did NOT lockout.

is there a way to figure out which Azure ATP agent install is the cause?

Thank you,

Robert

[#4]

@Eli Ofek i uninstalled the agent on each DC and then reinstalled it. The account got locked out again using the new account. i checked the error log on the offending agent, and this is what it showed:

2019-10-03 17:55:08.1794 Error DomainNetworkCredentialsManager GetInternal failed [domainName=med]

our domain name in the Azure ATP portal on the Directory Services tab is not "med". it is "domainname.med".

[#5]

@Robren , well, that eliminates any 3rd party action here...

First time I see this kind of outcome.

Are you aware of any special / non standard lockout policy in the forest?

It's weird, because if  this is the only credentials you provided to AATP and did not put them anywhere else, then the sensor used them without problems if you see them all running.

If the password would fail, the sensor would not be able to start...

so I am guessing it is getting locked out because of a specific action it does (which is not a wrong password).

Can you share your workspace id (in text format) in a private message? I will try to see if I can find any clues in telemetry from this deployment.

My best suggestion at this point is to check for any special lockout policy besides failed logon attempts.

Also - If you search the new account in AATP portal and go to it's logical activities page, do you see any alerts on this account? any significant logical activities that look odd (besides the lockout which should also appear there).

Just to make sure - once the account is locked out - the sensors fail, correct?

[#6]

@Eli Ofek if i leave the account locked out, the sensors will all start to fail over time, yes.

i will try and find the info you requested. 

thank you

[#7]

@Robren , if this error was produced after the account lockout it is expected I guess.

Do you have only one domain ? or is it a forest where med is the parent domain?

AATP will try to traverse all the domains in the forest, not just the domain of the AATP account you provided.

[#8]

@Eli Ofek just one domain. 

[#9]

Eli, thanks for your help. I am just going to open a case in Azure portal.

if they figure it out, i'll post back here.

[#10]

@Robren Can you send a screenshot of the Directory services tab from the config screen ( in a private message) ?

[#11]

@Eli Ofek A year on from this last post...which was left hanging, we are seeing something similar. Random domain controllers, in a multi domain, single forest, locking up the ATP svc-account. Was a solution ever found for this, and never posted back ?

For us it has been happening for a >2 weeks, plus we are also seeing random "all domain controllers are unreachable by sensor"...and in the alert, it just talks about one DC !  Sensor upgrade issues ?

[#12]

@StuartH .  I don't remember the outcome of this specific case, maybe @Robren remembers how the issue concluded ?