Forum Discussion
Solved
MDI sensor service will not start on ADFS server
I've exhausted my ability to troubleshoot why my ADFS sensor installs just will not start, so hoping someone can provide some guidance on how to get this working 🙂
Info:
- Windows Server 2022 Datacenter
- Public IP, no proxy
- Using gMSA
- Sensor version: 2.203.16523.48348
- Successful installation /w gMSA on DCs
Troubleshooting:
- Verified that ADFS auditing was set to verbose
- Verified that gMSA could access database
- Verified that gMSA is allowed to logon as a service under the DCs
- Is this need on the ADFS servers as well?
- Verified that the sensor config was given a FQDN DC.
- Verified DisableRenegoONserver is set to 0
- Verified DisableRenegoONclient is set to 0
- The dns name for our sensor endpoint is resolving correctly.
Observations:
- Microsoft.Tri.Sensor.Updater is not listening on 444, but system is
- There is an ATP certificate in the machine personal store from the installation, despite the logs saying one isn't found/used.
Log entry:
Microsoft.Tri.Sensor.log
2023-05-31 17:58:00.5355 Error ExceptionHandler Microsoft.Tri.Infrastructure.ExtendedException: RestrictCpuAsync failed, exiting ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at int System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
at int System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at int System.Net.Security._SslStream.EndRead(IAsyncResult asyncResult)
at int System.Net.TlsStream.EndRead(IAsyncResult asyncResult)
at void System.Net.Connection.ReadCallback(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at Stream System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, out TransportContext context)
at void System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at async Task<HttpResponseMessage> System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task<HttpResponseMessage> sendTask, HttpRequestMessage request, CancellationTokenSource cts, bool disposeCts)
at async Task<TResponse> Microsoft.Tri.Common.CommunicationWebClient.SendAsync<TResponse>(byte[] requestBytes, int offset, int count)
at async Task<TResponse> Microsoft.Tri.Common.CommunicationWebClient.SendWithRetryAsync<TResponse>(byte[] requestBytes, int offset, int count)
at async Task Microsoft.Tri.Common.CommunicationWebClient.SendAsync(IVoidRequest request)
at async Task Microsoft.Tri.Sensor.Common.ServiceProxy<TWebClientConfiguration>.SendAsync(IVoidRequest request)
at async Task Microsoft.Tri.Sensor.SensorResourceManager.RestrictCpuAsync()
--- End of inner exception stack trace --
Microsoft.Tri.Sensor-Errors
2023-05-31 17:58:00.5355 Error ExceptionHandler Microsoft.Tri.Infrastructure.ExtendedException: RestrictCpuAsync failed, exiting ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at int System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
at int System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at int System.Net.Security._SslStream.EndRead(IAsyncResult asyncResult)
at int System.Net.TlsStream.EndRead(IAsyncResult asyncResult)
at void System.Net.Connection.ReadCallback(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at Stream System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, out TransportContext context)
at void System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at async Task<HttpResponseMessage> System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task<HttpResponseMessage> sendTask, HttpRequestMessage request, CancellationTokenSource cts, bool disposeCts)
at async Task<TResponse> Microsoft.Tri.Common.CommunicationWebClient.SendAsync<TResponse>(byte[] requestBytes, int offset, int count)
at async Task<TResponse> Microsoft.Tri.Common.CommunicationWebClient.SendWithRetryAsync<TResponse>(byte[] requestBytes, int offset, int count)
at async Task Microsoft.Tri.Common.CommunicationWebClient.SendAsync(IVoidRequest request)
at async Task Microsoft.Tri.Sensor.Common.ServiceProxy<TWebClientConfiguration>.SendAsync(IVoidRequest request)
at async Task Microsoft.Tri.Sensor.SensorResourceManager.RestrictCpuAsync()
--- End of inner exception stack trace ---Microsoft.Tri.Sensor.Updater
2023-05-31 17:58:00.2690 Warn ClientCertificateAuthenticationMiddleware+ClientCertificateAuthenticationHandler AuthenticateCoreAsync missing client certificate
2023-05-31 17:58:00.2690 Warn AppBuilderExtension UseExceptionHandler IOException ignored [Details=InnerExceptionType=HttpListenerException ErrorCode=1229]
2023-05-31 17:58:00.2811 Warn ClientCertificateAuthenticationMiddleware+ClientCertificateAuthenticationHandler AuthenticateCoreAsync missing client certificate
2023-05-31 17:58:00.2811 Warn AppBuilderExtension UseExceptionHandler IOException ignored [Details=InnerExceptionType=HttpListenerException ErrorCode=1229]
2023-05-31 17:58:00.3003 Warn ClientCertificateAuthenticationMiddleware+ClientCertificateAuthenticationHandler AuthenticateCoreAsync missing client certificate
2023-05-31 17:58:00.3003 Warn AppBuilderExtension UseExceptionHandler IOException ignored [Details=InnerExceptionType=HttpListenerException ErrorCode=1229]
2023-05-31 17:58:00.3316 Warn ClientCertificateAuthenticationMiddleware+ClientCertificateAuthenticationHandler AuthenticateCoreAsync missing client certificate
2023-05-31 17:58:00.3316 Warn AppBuilderExtension UseExceptionHandler IOException ignored [Details=InnerExceptionType=HttpListenerException ErrorCode=1229]
2023-05-31 17:58:15.1918 Warn ClientCertificateAuthenticationMiddleware+ClientCertificateAuthenticationHandler AuthenticateCoreAsync missing client certificate
- The rep from Microsoft was able to resolve the startup issue!
If a server has HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\SendTrustedIssuerList set to 1, that will cause the service to fail.
According to https://learn.microsoft.com/en-us/windows-server/security/tls/what-s-new-in-tls-ssl-schannel-ssp-overview 0 is the default as of 2012R2
Since we did in-place upgrades, it makes sense that the value would be set from those previous OS's.
7 Replies
- EliOfek
Microsoft
Check acls on the cert. Make sure both local system and local service has read access to the cert.- EliOfekOK, I took a deeper look in the code that outputs this message.
The problem is that the sensor is sending a TLS request to the updater on localhost TCP 444.
It authenticates using a client certificate in the request,
But when the updater gets the request and tries to authenticate via the certificate, it is missing from the request.
So either something is blocking the sensor from putting it there, or we have some kind of "Man in the Middle" in the machine that "scrubs" the certificate out.
Are you familiar with anything installed on this machine that might temper with this request?
Some things I found related to similar cases from the past:
Having ADFS proxy installed on the same machine might cause it.
Some customers reported the issue fixed after setting those registry values to 0:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL]
"DisableRenegoOnServer"=dword:00000001
"DisableRenegoOnClient"=dword:00000001
If all fails, the next step is openings a support request, but I have to be honest with you:
The last case that did not resolved by all of the above required so much research time to check what got broken in the machine that eventually it would have been easier to just rebuild it.
