Azure ATP Sensor install failing (Updater Service do not start)

[PhilippFoeckeler]

Hello All!

We try to install the Azure ATP Sensor on a DC, setup wizard is running until this point

...then do some retries for about 3 minutes, during this time the service "Azure Advanced Threat Protection Sensor Updater" is several times on state "starting" und back to not started.

Then setup fails with 0x80070643 and do a rollback.

In the "Microsoft.Tri.Sensor.Updater-Errors" log, we find this error every 10 seconds during the setup:

2019-12-23 11:27:37.8384 Error CommunicationWebClient+<SendWithRetryAsync>d__8`1 Microsoft.Tri.Infrastructure.ExtendedException: Sanitized exception: [Type=System.Net.Http.HttpRequestExceptionMessage=7INzM3PVZQKggOiiHcWjqw==StackTrace= at async Task<HttpResponseMessage> System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task<HttpResponseMessage> sendTask, HttpRequestMessage request, CancellationTokenSource cts, bool disposeCts)
at async Task<TResponse> Microsoft.Tri.Common.CommunicationWebClient.SendAsync<TResponse>(byte[] requestBytes, int offset, int count)
at async Task<TResponse> Microsoft.Tri.Common.CommunicationWebClient.SendWithRetryAsync<TResponse>(byte[] requestBytes, int offset, int count)InnerException=Microsoft.Tri.Infrastructure.ExtendedException: Sanitized exception: [Type=System.Net.WebExceptionMessage=5iiWw0iPCPzCGdZStU4OxA==StackTrace= at Stream System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, out TransportContext context)
at void System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)InnerException=]]
at async Task<TResponse> Microsoft.Tri.Common.CommunicationWebClient.SendWithRetryAsync<TResponse>(byte[] requestBytes, int offset, int count)
at async Task<TResponse> Microsoft.Tri.Common.CommunicationWebClient.SendAsync<TResponse>(IRequestWithResponse<TResponse> request)
at async Task<TResponse> Microsoft.Tri.Sensor.Common.ServiceProxy<TWebClientConfiguration>.SendAsync<TResponse>(IRequestWithResponse<TResponse> request)
at async Task Microsoft.Tri.Sensor.Updater.SensorUpdaterConfigurationUpdater.UpdateConfigurationAsync(bool isStarted)
at void Microsoft.Tri.Infrastructure.TaskExtension.Await(Task task)
at new Microsoft.Tri.Sensor.Updater.SensorUpdaterConfigurationUpdater(IConfigurationManager configurationManager, IMetricManager metricManager, ISecretManager secretManager, IWorkspaceApplicationSensorApiJsonProxy workspaceApplicationSensorApiJsonProxy)
at object lambda_method(Closure, object[])
at object Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
at void Microsoft.Tri.Infrastructure.ModuleManager.AddModules(Type[] moduleTypes)
at ModuleManager Microsoft.Tri.Sensor.Updater.SensorUpdaterService.CreateModuleManager()
at async Task Microsoft.Tri.Infrastructure.Service.OnStartAsync()
at void Microsoft.Tri.Infrastructure.TaskExtension.Await(Task task)
at void Microsoft.Tri.Infrastructure.Service.OnStart(string[] args)

A proxy is used which allows access to *.atp.azure.com without auth. In proxy logs, we see no block for this server, only successful requests from this DC. There is no indication that 443 would be blocked somewhere else...

The AD account which is configured in the ATP portal was checked, domain is given in FQDN there and the password is correct.

Any ideas someone?

[#2]

@PhilippFoeckeler Effectively this error means it was blocked.

Is your proxy doing SSL inspection?

[#3]

No - there is no SSL inspection on the proxy... and in the proxy logs  no blocks for this server. Very strange. 

Local Firewall is switched off. So for this SSL connection to localhost on port 444, i cannot see any reasons that this should be not possible.

[#4]

@PhilippFoeckeler , Any chance you can temporary bypass this proxy just to see if it resolves the issue?
At least for the error sample you published, the problem is going to the azure backend, not to localhost.

[#5]

@EliOfek , First of all Thank you so much for the help so far!!

Unfortunately, we are still stuck at the same point - the system is sensitive and Risk and Security Team do not allow to connect directly to the internet, even if it's temporary..

What we tried in the meantime:

We enabled routing to another Proxy which is used by other Domain Controllers (where the ATP sensor could be installed without any problems).... Proxy can be used in the browser, proxy was set as system proxy.... same issue - no blocks whatsoever are visible at the proxy.

We disabled local endpoint protection for this server (Cisco AMP) during the install....same issue.

I think i will open a Premier Support Ticket so that a MS Engineer can have a look in a remote session on this server.... 

[#6]

@PhilippFoeckeler 

I am getting exactly the same error at my client's site. Identical configuration used each time, majority of DCs installation works fine, but on those that don't I see exactly the same issue as you describe.

While the install is proceeding (or stuck mid point as you are seeing), the sensor appears in the console in a stopped state (it is even possible to configure the update settings), but when the client install eventually times out with the error, the sensor gets cleared out of the console as well. The only other symptom I have noticed is the multiple 'unexpected restart' entries in the system log and the ATP Sensor updater service usually stuck in the 'starting state'. I have not found any issues with the WMI performance adaptor either, something I checked because it is a dependency of the Sensor updater service.

[#7]

@Richard Adams , a bit confused, are you getting the same error during deployment or service  start?

[#8]

@EliOfek 

Hi,

Same as the poster's original screenshot. The installation gets stuck (about midway through according to the GUI). After a long wait it removes the sensor from the console, backs out the client install and displays the installation failure error (0x80070643) which suggests proxy issues, but if this was the case how would the client register in Security Center in the first place?  I am using exactly the same installation process on all my Client's DCs, so far I have had 15 successful and another 5 or so fail.

Richard

[#9]

@Richard Adams The screenshot alone does not mean it's the same issue.

And you are right, if you managed to see it registers and removed, then it's most likely something else.

You will need to take a look at the logs to know why.

see

https://docs.microsoft.com/en-us/azure-advanced-threat-protection/troubleshooting-atp-using-logs#azu...

[#10]

@PhilippFoeckeler So the issue has been resolved now 🙂 

[#11]

@PhilippFoeckeler how did you solve this issue? We are facing exactly the same. 1DC cannot connect to endpoint, all others can. Authentication and TLS inpection are already disabled on proxy. 

[#12]

1. @ricklahaye Please try silent install and use proxy URL in the command used. Also, select bypass proxy server for local server option in LAN settings of the browser. 

Something like below:-

"Azure ATP sensor Setup.exe" /quiet ProxyUrl="http://abc.com:port number of proxy" NetFrameworkCommandLineArguments="/q" AccessKey=""

Let me know if that helps. 

[#13]

@Vishal_Sharma_4224 HI Vishal 

We are having same issue while instilling ATP sensor .

Tried to install silently but same error code it is getting.  0x80070643

Our DC is hosted on private LB.

we are using proxy settings to connect the internet 

[#14]

@Pritam1560 Please paste the main error logs here..

[#15]

It would be great if can send the error ligs in a private message.

[#16]

Please let me know what exactly do you see in deployment\MSI logs.. Have you used command below while doing silent install? "Azure ATP sensor Setup.exe" /quiet ProxyUrl="http://abc.com:port number of proxy" NetFrameworkCommandLineArguments="/q" AccessKey="" Also, please select option Bypass proxy server for local addresses in browser LAN settings if you are using browser specific proxy.

[#17]

@Vishal_Sharma_4224 

Hi Vishal we have tried with silent instillation with bypass proxy enable but got same error.

in logs we can see error returning code 

[20EC:2318][2020-04-16T20:56:04]i007: Exit code: 0x80070643, restarting: No

1.is there any dependencies on .net frameowrk 

2.this dc is configured in standard load balencer .is that something to do with that

In other DC which is  hosted in Europe region configured in Basic LB there ATP is installing properly same is not happening in another DC

[#18]

@Pritam1560 

Yes, this is a dependency .NET Framework 4.7 must be installed as a pre-requisite..

For all the pre-requisites you may refer link below:-

https://docs.microsoft.com/en-us/azure-advanced-threat-protection/atp-prerequisites .

[#19]

Hi @Pritam1560 

We are having same issue ,  are you solve this issue?

[#20]

@minah 

Could you please let us know what error are you witnessing in the Deployer logs?