ATP and group managed service account not working on RODC

[19873306]

We have ATP sensors set up on our domain controllers. A group managed service account (gMSA) is being used.  There are a few read only domain controllers that can't seem to read the password, even though the servers are in the group that can read the GMSA user password. The GMSA account is set with permissions for 'log in as service'.  Any suggestions on what to look for? 

Errors from the sensor log:

2020-09-14 22:02:11.7896 Debug DirectoryServicesClient SetState Creating
2020-09-14 22:02:11.8346 Info ImpersonationManager CreateImpersonatorAsync started [UserName=<MSA-ACCOUNT> IsGroupManagedServiceAccount=True]
2020-09-14 22:02:11.8846 Info ImpersonationManager CreateImpersonatorAsync finished [UserName=<MSA-ACCOUNT> IsSuccess=False]
2020-09-14 22:02:11.8846 Warn DirectoryServicesClient CreateLdapConnectionAsync failed to retrieve group managed service account password. [DomainControllerDnsName=servername.domain.corp Domain=domain.corp UserName=<MSA-ACCOUUNT> ]
2020-09-14 22:02:12.0846 Error DirectoryServicesClient+<CreateLdapConnectionAsync>d__38 Microsoft.Tri.Infrastructure.ExtendedException: CreateLdapConnectionAsync failed [DomainControllerDnsName=<serverwhereATPfailing.domain.corp]
at async Task<LdapConnection> Microsoft.Tri.Sensor.DirectoryServicesClient.CreateLdapConnectionAsync(DomainControllerConnectionData domainControllerConnectionData, bool isGlobalCatalog, bool isTraversing)
at async Task<bool> Microsoft.Tri.Sensor.DirectoryServicesClient.TryCreateLdapConnectionAsync(DomainControllerConnectionData domainControllerConnectionData, bool isGlobalCatalog, bool isTraversing)
2020-09-14 22:02:12.0946 Error DirectoryServicesClient Microsoft.Tri.Infrastructure.ExtendedException: Failed to communicate with configured domain controllers
at new Microsoft.Tri.Sensor.DirectoryServicesClient(IConfigurationManager configurationManager, IDomainNetworkCredentialsManager domainNetworkCredentialsManager, IImpersonationManager impersonationManager, IMetricManager metricManager, IWorkspaceApplicationSensorApiJsonProxy workspaceApplicationSensorApiJsonProxy)
at object lambda_method(Closure, object[])
at object Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
at void Microsoft.Tri.Infrastructure.ModuleManager.AddModules(Type[] moduleTypes)
at new Microsoft.Tri.Sensor.SensorModuleManager()
at ModuleManager Microsoft.Tri.Sensor.SensorService.CreateModuleManager()
at async Task Microsoft.Tri.Infrastructure.Service.OnStartAsync()
at void Microsoft.Tri.Infrastructure.TaskExtension.Await(Task task)
at void Microsoft.Tri.Infrastructure.Service.OnStart(string[] args)

[#2]

@19873306 what OS version are those RODCs running?

[#3]

@EliOfek Server 2019.  I saw the patch for 2012, but it doesn't apply here.

[#4]

@19873306 Then I suggest opening a support case for this one so they can go over with you on the settings to make sure nothing was missed. in some environments this  can be a bit tricky to set up.

[#5]

@EliOfek Thank you. Should this support request be opened through Azure portal? 

[#6]

Yes

[#7]

@EliOfek Case 120091525000664  created.  The technician is indicating that there are issues with npcap or winpcap.  We have other servers in our environment which are running ATP sensor without either NPCAP or WINPCAP.Are either required for AZURE ATP sensor? I do not see them listed in the pre-requisites here https://docs.microsoft.com/en-us/azure-advanced-threat-protection/atp-prerequisites

The server once had Wireshark installed. I have uninstalled Wireshark, and Winpcap.  

The problem I am having is not with installation, it is with the sensor not running. 

[#8]

@19873306 , if you don't have npcap installed before you install the sensor,

the sensor deployment will auto install a "local" winpcap install. you won't see it in "add\remove" programs, but you can see the driver service running with this command:
sc qc npf

[#9]

sc qc npf does not return anything.

I also tried powershell get-service:

Get-Service npf
Get-Service : Cannot find any service with service name 'npf'.
At line:1 char:1
+ Get-Service npf
+ ~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (npf:String) [Get-Service], ServiceCommandException
+ FullyQualifiedErrorId : NoServiceFoundForGivenName,Microsoft.PowerShell.Commands.GetServiceCommand

[#10]

@19873306 Make sure you are running elevated when running this.

If you still can't find npf driver, check also

sc qc npcap

If you don't have this one as well, then you have no capturing driver installed, which means the sensor cannot work.

I would try to uninstall and reinstall. if the same problem returns, it means you have something that is somehow blocking or reversing the driver installation.

Most likely a 3rd party security software, so try to disable during before deployment to see if it makes things work.

[#11]

@EliOfek I uninstalled the sensor, rebooted, then reinstalled.

I now have 

C:\WINDOWS\system32>sc qc npf
[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: npf
TYPE : 1 KERNEL_DRIVER
START_TYPE : 2 AUTO_START
ERROR_CONTROL : 1 NORMAL
BINARY_PATH_NAME : \??\C:\WINDOWS\system32\drivers\npf.sys
LOAD_ORDER_GROUP :
TAG : 0
DISPLAY_NAME : NetGroup Packet Filter Driver
DEPENDENCIES :
SERVICE_START_NAME :

C:\WINDOWS\system32>

However, the sensor still will not start

Partial error message:

2020-09-18 22:55:35.0283 Warn DirectoryServicesClient CreateLdapConnectionAsync failed to retrieve group managed service account password.

The referenced group managed service account is installed on the server, ands tests true from powershell. 

Partial event log message in directory services indicate the password is fetched successfully:

A caller successfully fetched the password of a group managed service account.

Group Managed Service Account Object:
CN=Microsoft Azure ATP Sensor,OU=ATP,OU=Azure

[#12]

@19873306 
So it seems you are over the initial issue.

As for the Gmsa issue, it's a bit more tricky.

Check errors and warnings in both the sensor logs and the updater logs around this time span to see if you get new insights about what went wrong, or else I suggest opening a support case  as it might be tricky.

[#13]

Is any patch need to install before start ATP installation?

For my case, we have tried to install ATP after installed OS and promoted as RODC, but the service is not getting start. After installed all latest patches, it's automatically started. No change made.

[#14]

Only for 2019. what was the error in the logs when it failed to start before patching?

[#15]

We were the same. I just recently built\added some new DC's and installed ATP. Couldn't get the service to start and saw the same error in the Logs. Made sure the new machines had access to the GMSA and log on as a service enabled... still no go...
Saw your message here and noted the server had a Defender definition update to do... so I installed that and bounced the box and voila! 🙂
I repeated this process on all my other new DC's and it fixed every one of them.
Thanks!

[#16]

I have a similar issue, I have already opened a case, but 24 hours still no reply from Microsoft support.