ATP and group managed service account not working on RODC

%3CLINGO-SUB%20id%3D%22lingo-sub-1666698%22%20slang%3D%22en-US%22%3EATP%20and%20group%20managed%20service%20account%20not%20working%20on%20RODC%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1666698%22%20slang%3D%22en-US%22%3E%3CP%3EWe%20have%20ATP%20sensors%20set%20up%20on%20our%20domain%20controllers.%20A%20group%20managed%20service%20account%20(gMSA)%20is%20being%20used.%26nbsp%3B%20There%20are%20a%20few%20read%20only%20domain%20controllers%20that%20can't%20seem%20to%20read%20the%20password%2C%20even%20though%20the%20servers%20are%20in%20the%20group%20that%20can%20read%20the%20GMSA%20user%20password.%20The%20GMSA%20account%20is%20set%20with%20permissions%20for%20'log%20in%20as%20service'.%26nbsp%3B%20Any%20suggestions%20on%20what%20to%20look%20for%3F%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EErrors%20from%20the%20sensor%20log%3A%3C%2FP%3E%3CP%3E%3CEM%3E2020-09-14%2022%3A02%3A11.7896%20Debug%20DirectoryServicesClient%20SetState%20Creating%3CBR%20%2F%3E2020-09-14%2022%3A02%3A11.8346%20Info%20ImpersonationManager%20CreateImpersonatorAsync%20started%20%5BUserName%3D%3CMSA-ACCOUNT%3E%20IsGroupManagedServiceAccount%3DTrue%5D%3CBR%20%2F%3E2020-09-14%2022%3A02%3A11.8846%20Info%20ImpersonationManager%20CreateImpersonatorAsync%20finished%20%5BUserName%3D%3CMSA-ACCOUNT%3E%20IsSuccess%3DFalse%5D%3CBR%20%2F%3E2020-09-14%2022%3A02%3A11.8846%20Warn%20DirectoryServicesClient%20CreateLdapConnectionAsync%20failed%20to%20retrieve%20group%20managed%20service%20account%20password.%20%5BDomainControllerDnsName%3Dservername.domain.corp%20Domain%3Ddomain.corp%20UserName%3D%3CMSA-ACCOUUNT%3E%20%5D%3CBR%20%2F%3E2020-09-14%2022%3A02%3A12.0846%20Error%20DirectoryServicesClient%2B%3CCREATELDAPCONNECTIONASYNC%3Ed__38%20Microsoft.Tri.Infrastructure.ExtendedException%3A%20CreateLdapConnectionAsync%20failed%20%5BDomainControllerDnsName%3D%3CSERVERWHEREATPFAILING.DOMAIN.CORP%3E%3C%2FSERVERWHEREATPFAILING.DOMAIN.CORP%3Eat%20async%20Task%3CLDAPCONNECTION%3E%20Microsoft.Tri.Sensor.DirectoryServicesClient.CreateLdapConnectionAsync(DomainControllerConnectionData%20domainControllerConnectionData%2C%20bool%20isGlobalCatalog%2C%20bool%20isTraversing)%3CBR%20%2F%3Eat%20async%20Task%3CBOOL%3E%20Microsoft.Tri.Sensor.DirectoryServicesClient.TryCreateLdapConnectionAsync(DomainControllerConnectionData%20domainControllerConnectionData%2C%20bool%20isGlobalCatalog%2C%20bool%20isTraversing)%3CBR%20%2F%3E2020-09-14%2022%3A02%3A12.0946%20Error%20DirectoryServicesClient%20Microsoft.Tri.Infrastructure.ExtendedException%3A%20Failed%20to%20communicate%20with%20configured%20domain%20controllers%3CBR%20%2F%3Eat%20new%20Microsoft.Tri.Sensor.DirectoryServicesClient(IConfigurationManager%20configurationManager%2C%20IDomainNetworkCredentialsManager%20domainNetworkCredentialsManager%2C%20IImpersonationManager%20impersonationManager%2C%20IMetricManager%20metricManager%2C%20IWorkspaceApplicationSensorApiJsonProxy%20workspaceApplicationSensorApiJsonProxy)%3CBR%20%2F%3Eat%20object%20lambda_method(Closure%2C%20object%5B%5D)%3CBR%20%2F%3Eat%20object%20Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()%3CBR%20%2F%3Eat%20void%20Microsoft.Tri.Infrastructure.ModuleManager.AddModules(Type%5B%5D%20moduleTypes)%3CBR%20%2F%3Eat%20new%20Microsoft.Tri.Sensor.SensorModuleManager()%3CBR%20%2F%3Eat%20ModuleManager%20Microsoft.Tri.Sensor.SensorService.CreateModuleManager()%3CBR%20%2F%3Eat%20async%20Task%20Microsoft.Tri.Infrastructure.Service.OnStartAsync()%3CBR%20%2F%3Eat%20void%20Microsoft.Tri.Infrastructure.TaskExtension.Await(Task%20task)%3CBR%20%2F%3Eat%20void%20Microsoft.Tri.Infrastructure.Service.OnStart(string%5B%5D%20args)%3C%2FBOOL%3E%3C%2FLDAPCONNECTION%3E%3C%2FCREATELDAPCONNECTIONASYNC%3E%3C%2FMSA-ACCOUUNT%3E%3C%2FMSA-ACCOUNT%3E%3C%2FMSA-ACCOUNT%3E%3C%2FEM%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1666698%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EATP%20Sensor%20GMSA%20can't%20read%20passsword%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1666718%22%20slang%3D%22en-US%22%3ERe%3A%20ATP%20and%20group%20managed%20service%20account%20not%20working%20on%20RODC%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1666718%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F616179%22%20target%3D%22_blank%22%3E%4019873306%3C%2FA%3E%26nbsp%3Bwhat%20OS%20version%20are%20those%20RODCs%20running%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1666743%22%20slang%3D%22en-US%22%3ERe%3A%20ATP%20and%20group%20managed%20service%20account%20not%20working%20on%20RODC%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1666743%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F106935%22%20target%3D%22_blank%22%3E%40Eli%20Ofek%3C%2FA%3E%26nbsp%3BServer%202019.%26nbsp%3B%20I%20saw%20the%20patch%20for%202012%2C%20but%20it%20doesn't%20apply%20here.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1667495%22%20slang%3D%22en-US%22%3ERe%3A%20ATP%20and%20group%20managed%20service%20account%20not%20working%20on%20RODC%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1667495%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F616179%22%20target%3D%22_blank%22%3E%4019873306%3C%2FA%3E%26nbsp%3BThen%20I%20suggest%20opening%20a%20support%20case%20for%20this%20one%20so%20they%20can%20go%20over%20with%20you%20on%20the%20settings%20to%20make%20sure%20nothing%20was%20missed.%20in%20some%20environments%20this%26nbsp%3B%20can%20be%20a%20bit%20tricky%20to%20set%20up.%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
Occasional Contributor

We have ATP sensors set up on our domain controllers. A group managed service account (gMSA) is being used.  There are a few read only domain controllers that can't seem to read the password, even though the servers are in the group that can read the GMSA user password. The GMSA account is set with permissions for 'log in as service'.  Any suggestions on what to look for? 

 

Errors from the sensor log:

2020-09-14 22:02:11.7896 Debug DirectoryServicesClient SetState Creating
2020-09-14 22:02:11.8346 Info ImpersonationManager CreateImpersonatorAsync started [UserName=<MSA-ACCOUNT> IsGroupManagedServiceAccount=True]
2020-09-14 22:02:11.8846 Info ImpersonationManager CreateImpersonatorAsync finished [UserName=<MSA-ACCOUNT> IsSuccess=False]
2020-09-14 22:02:11.8846 Warn DirectoryServicesClient CreateLdapConnectionAsync failed to retrieve group managed service account password. [DomainControllerDnsName=servername.domain.corp Domain=domain.corp UserName=<MSA-ACCOUUNT> ]
2020-09-14 22:02:12.0846 Error DirectoryServicesClient+<CreateLdapConnectionAsync>d__38 Microsoft.Tri.Infrastructure.ExtendedException: CreateLdapConnectionAsync failed [DomainControllerDnsName=<serverwhereATPfailing.domain.corp]
at async Task<LdapConnection> Microsoft.Tri.Sensor.DirectoryServicesClient.CreateLdapConnectionAsync(DomainControllerConnectionData domainControllerConnectionData, bool isGlobalCatalog, bool isTraversing)
at async Task<bool> Microsoft.Tri.Sensor.DirectoryServicesClient.TryCreateLdapConnectionAsync(DomainControllerConnectionData domainControllerConnectionData, bool isGlobalCatalog, bool isTraversing)
2020-09-14 22:02:12.0946 Error DirectoryServicesClient Microsoft.Tri.Infrastructure.ExtendedException: Failed to communicate with configured domain controllers
at new Microsoft.Tri.Sensor.DirectoryServicesClient(IConfigurationManager configurationManager, IDomainNetworkCredentialsManager domainNetworkCredentialsManager, IImpersonationManager impersonationManager, IMetricManager metricManager, IWorkspaceApplicationSensorApiJsonProxy workspaceApplicationSensorApiJsonProxy)
at object lambda_method(Closure, object[])
at object Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
at void Microsoft.Tri.Infrastructure.ModuleManager.AddModules(Type[] moduleTypes)
at new Microsoft.Tri.Sensor.SensorModuleManager()
at ModuleManager Microsoft.Tri.Sensor.SensorService.CreateModuleManager()
at async Task Microsoft.Tri.Infrastructure.Service.OnStartAsync()
at void Microsoft.Tri.Infrastructure.TaskExtension.Await(Task task)
at void Microsoft.Tri.Infrastructure.Service.OnStart(string[] args)

 

 

11 Replies
Highlighted

@19873306 what OS version are those RODCs running?

Highlighted

@Eli Ofek Server 2019.  I saw the patch for 2012, but it doesn't apply here.

 

Highlighted

@19873306 Then I suggest opening a support case for this one so they can go over with you on the settings to make sure nothing was missed. in some environments this  can be a bit tricky to set up.

Highlighted

@Eli Ofek Thank you. Should this support request be opened through Azure portal? 

Highlighted
Highlighted

@Eli Ofek Case 120091525000664  created.  The technician is indicating that there are issues with npcap or winpcap.  We have other servers in our environment which are running ATP sensor without either NPCAP or WINPCAP.Are either required for AZURE ATP sensor? I do not see them listed in the pre-requisites here https://docs.microsoft.com/en-us/azure-advanced-threat-protection/atp-prerequisites

 
The server once had Wireshark installed. I have uninstalled Wireshark, and Winpcap.  

 

The problem I am having is not with installation, it is with the sensor not running. 
Highlighted

@19873306 , if you don't have npcap installed before you install the sensor,

the sensor deployment will auto install a "local" winpcap install. you won't see it in "add\remove" programs, but you can see the driver service running with this command:
sc qc npf

 

Highlighted
sc qc npf does not return anything.

I also tried powershell get-service:

Get-Service npf
Get-Service : Cannot find any service with service name 'npf'.
At line:1 char:1
+ Get-Service npf
+ ~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (npf:String) [Get-Service], ServiceCommandException
+ FullyQualifiedErrorId : NoServiceFoundForGivenName,Microsoft.PowerShell.Commands.GetServiceCommand
Highlighted

@19873306 Make sure you are running elevated when running this.

If you still can't find npf driver, check also

sc qc npcap

 

If you don't have this one as well, then you have no capturing driver installed, which means the sensor cannot work.

I would try to uninstall and reinstall. if the same problem returns, it means you have something that is somehow blocking or reversing the driver installation.

Most likely a 3rd party security software, so try to disable during before deployment to see if it makes things work.

Highlighted

@Eli Ofek I uninstalled the sensor, rebooted, then reinstalled.

 

I now have 

C:\WINDOWS\system32>sc qc npf
[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: npf
TYPE : 1 KERNEL_DRIVER
START_TYPE : 2 AUTO_START
ERROR_CONTROL : 1 NORMAL
BINARY_PATH_NAME : \??\C:\WINDOWS\system32\drivers\npf.sys
LOAD_ORDER_GROUP :
TAG : 0
DISPLAY_NAME : NetGroup Packet Filter Driver
DEPENDENCIES :
SERVICE_START_NAME :

C:\WINDOWS\system32>

 

 

However, the sensor still will not start

 

Partial error message:

2020-09-18 22:55:35.0283 Warn DirectoryServicesClient CreateLdapConnectionAsync failed to retrieve group managed service account password.

 

The referenced group managed service account is installed on the server, ands tests true from powershell. 

 

Partial event log message in directory services indicate the password is fetched successfully:

 

A caller successfully fetched the password of a group managed service account.

Group Managed Service Account Object:
CN=Microsoft Azure ATP Sensor,OU=ATP,OU=Azure

Highlighted

@19873306 
So it seems you are over the initial issue.

As for the Gmsa issue, it's a bit more tricky.

Check errors and warnings in both the sensor logs and the updater logs around this time span to see if you get new insights about what went wrong, or else I suggest opening a support case  as it might be tricky.