SOLVED

Azure ATP connection closed errors

%3CLINGO-SUB%20id%3D%22lingo-sub-308528%22%20slang%3D%22en-US%22%3EAzure%20ATP%20connection%20closed%20errors%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-308528%22%20slang%3D%22en-US%22%3E%3CP%3EHello%2C%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20just%20deployed%20Azure%20ATP%20in%20a%20fresh%20demo%20environment.%20No%20errors%20during%20installation%2C%20also%20when%20I%20search%20for%20computers%20or%20users%20I%20do%20get%20details%2C%20however%20when%20I%20try%20to%20trigger%20an%20alert%20by%20running%20nslookup%20-ls%20-d%20or%20run%20mimikatz%2C%20I%20don't%20get%20any%20alerts%20in%20Azure%20ATP.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20Sensor%20is%20installed%20on%20a%20domain%20controller%20server%202016%20standard%20%2C%20sensor%20agent%20version%20is%3C%2FP%3E%3CP%3E%3CFONT%3E2.59.6040.997%3C%2FFONT%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CFONT%3EThe%20domain%20controller%20runs%20in%20hyper-v%20and%20has%20two%20nics%2C%20one%20uses%20the%20default%20switch%20and%20the%20other%20is%20a%20private%20network%20for%20the%20lab.%26nbsp%3B%3C%2FFONT%3E%3C%2FP%3E%3CP%3E%3CFONT%3E%26nbsp%3B%3C%2FFONT%3E%3C%2FP%3E%3CP%3EThe%26nbsp%3B%3CFONT%3EMicrosoft.Tri.Sensor-Errors%3C%2FFONT%3E%20log%20file%20contains%20the%20following%20errors.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CFONT%3E2019-01-03%2017%3A38%3A50.3349%20Error%20ExceptionDispatchInfo%20System.Net.Http.HttpRequestException%3A%20An%20error%20occurred%20while%20sending%20the%20request.%20---%26gt%3B%20System.Net.WebException%3A%20The%20underlying%20connection%20was%20closed%3A%20A%20connection%20that%20was%20expected%20to%20be%20kept%20alive%20was%20closed%20by%20the%20server.%20---%26gt%3B%20System.IO.IOException%3A%20Unable%20to%20read%20data%20from%20the%20transport%20connection%3A%20A%20connection%20attempt%20failed%20because%20the%20connected%20party%20did%20not%20properly%20respond%20after%20a%20period%20of%20time%2C%20or%20established%20connection%20failed%20because%20connected%20host%20has%20failed%20to%20respond.%20---%26gt%3B%20System.Net.Sockets.SocketException%3A%20A%20connection%20attempt%20failed%20because%20the%20connected%20party%20did%20not%20properly%20respond%20after%20a%20period%20of%20time%2C%20or%20established%20connection%20failed%20because%20connected%20host%20has%20failed%20to%20respond%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20int%20System.Net.Sockets.Socket.EndReceive(IAsyncResult%20asyncResult)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20int%20System.Net.Sockets.NetworkStream.EndRead(IAsyncResult%20asyncResult)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20---%20End%20of%20inner%20exception%20stack%20trace%20---%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20int%20System.Net.Security._SslStream.EndRead(IAsyncResult%20asyncResult)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20int%20System.Net.TlsStream.EndRead(IAsyncResult%20asyncResult)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20void%20System.Net.Connection.ReadCallback(IAsyncResult%20asyncResult)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20---%20End%20of%20inner%20exception%20stack%20trace%20---%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20WebResponse%20System.Net.HttpWebRequest.EndGetResponse(IAsyncResult%20asyncResult)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20void%20System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult%20ar)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20---%20End%20of%20inner%20exception%20stack%20trace%20---%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20async%20Task%3CHTTPRESPONSEMESSAGE%3E%20System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task%3CHTTPRESPONSEMESSAGE%3E%20sendTask%2C%20HttpRequestMessage%20request%2C%20CancellationTokenSource%20cts%2C%20bool%20disposeCts)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20async%20Task%3CTRESPONSE%3E%20Microsoft.Tri.Common.CommunicationWebClient.SendAsync%3CTRESPONSE%3E(byte%5B%5D%20requestBytes%2C%20int%20offset%2C%20int%20count)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20async%20Task%3CTRESPONSE%3E%20Microsoft.Tri.Common.CommunicationWebClient.SendAsync%3CTRESPONSE%3E(IRequestWithResponse%3CTRESPONSE%3E%20request)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20async%20Task%3CTRESPONSE%3E%20Microsoft.Tri.Sensor.Common.ServiceProxy%3CTWEBCLIENTCONFIGURATION%3E.SendAsync%3CTRESPONSE%3E(IRequestWithResponse%3CTRESPONSE%3E%20request)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20async%20Task%20Microsoft.Tri.Sensor.EntitySender.SendEntityBatchAsync(EntityBatch%20entityBatch%2C%20EntityBatch%20postponedEntityBatch)%3CBR%20%2F%3E%26nbsp%3B%26nbsp%3B%20at%20async%20Task%20Microsoft.Tri.Sensor.EntitySender.SendEntityBatchesAsync()%3C%2FTRESPONSE%3E%3C%2FTRESPONSE%3E%3C%2FTWEBCLIENTCONFIGURATION%3E%3C%2FTRESPONSE%3E%3C%2FTRESPONSE%3E%3C%2FTRESPONSE%3E%3C%2FTRESPONSE%3E%3C%2FTRESPONSE%3E%3C%2FTRESPONSE%3E%3C%2FHTTPRESPONSEMESSAGE%3E%3C%2FHTTPRESPONSEMESSAGE%3E%3C%2FFONT%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CFONT%3EAny%20hints%20where%20to%20look%20wuld%20be%20appreciated.%26nbsp%3B%3C%2FFONT%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-309694%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20ATP%20connection%20closed%20errors%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-309694%22%20slang%3D%22en-US%22%3E%3CP%3Egreat%20news%2C%20and%20it%20makes%20sense%20because%20the%20log%20has%20many%20errors%20about%20connectivity%20with%20the%20AATP.%3C%2FP%3E%0A%3CP%3ENow%20you%20can%20start%20with%20attack%20and%20simulate%20scenario%20on%20your%20DC's.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-309687%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20ATP%20connection%20closed%20errors%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-309687%22%20slang%3D%22en-US%22%3E%3CP%3EHello%20Eli%20%26amp%3B%20Eli%2C%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%20for%20both%20your%20responses%2C%20I%20ended%20u%20disabling%20the%20firewall%20on%20my%20UPC%20ISP%20router%20and%20suddenly%20packets%20where%20floating%20as%20expected.%20Strange%2C%20as%20this%20used%20to%20work%20previously%2C%20but%20at%20least%20I%20found%20the%20issue.%20Thanks%20for%20your%20feedback.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAlex%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-309454%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20ATP%20connection%20closed%20errors%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-309454%22%20slang%3D%22en-US%22%3E%3CP%3Eyou%20can%20work%20with%20a%20demo%20environment%20without%20any%20issues.%3C%2FP%3E%0A%3CP%3Eback%20to%20the%20log%2C%20you've%20some%20error%20with%3A%20t%3CSPAN%3Ehe%20underlying%20connection%20was%20closed...%20and%20that%20means%20you've%20some%20issue%20with%20a%20connection%20from%20your%20local%20DC's%20to%20the%20AATP%2C%20it%20could%20be%20a%20connectivity%20issue%2C%20some%20SSL%20inspection%20or%20even%20firewall%20port.%3CBR%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-309411%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20ATP%20connection%20closed%20errors%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-309411%22%20slang%3D%22en-US%22%3E%3CP%3EHi%2C%20there's%20no%20proxy%20in%20place%2C%20this%20is%20a%20very%20simple%20setup%2C%20the%20DC%20runs%20in%20a%20VM%20on%20my%20notebook%20that%20has%20direct%20internet%20connection%20via%20my%20home%20ISP%20connection.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EBut%20meanwhile%20I%20found%20another%20article%20regarding%20the%20use%20of%20ATP%20for%20tenants%20created%20via%20demos.microsoft.com%20maybe%20that%20could%20be%20the%20issue%3F%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-309408%22%20slang%3D%22en-US%22%3ERE%3A%20Azure%20ATP%20connection%20closed%20errors%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-309408%22%20slang%3D%22en-US%22%3Emeanwhile%20I%20found%20this%20threat%2C%20which%20might%20be%20the%20reason%20for%20my%20issue%2C%20however%20I%20was%20able%20to%20create%20an%20instance.%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzure-Advanced-Threat-Protection%2FFull-Azure-ATP-Trial-for-testing-amp-demo-purposes%2Ftd-p%2F203104%22%20target%3D%22_blank%22%3Ehttps%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzure-Advanced-Threat-Protection%2FFull-Azure-ATP-Trial-for-testing-amp-demo-purposes%2Ftd-p%2F203104%3C%2FA%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-309407%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20ATP%20connection%20closed%20errors%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-309407%22%20slang%3D%22en-US%22%3E%3CP%3E503%20is%20normal%20when%20using%20a%20browser.%3C%2FP%3E%0A%3CP%3ESomething%20is%20blocking%20the%20connection.%3C%2FP%3E%0A%3CP%3ENotice%20that%20the%20services%20are%20running%20under%20local%20service%20%26amp%3B%20local%20system%2C%3C%2FP%3E%0A%3CP%3Eso%20they%20might%20be%20getting%20a%20different%20policy%20compared%20to%20your%20logged%20in%20user...%3C%2FP%3E%0A%3CP%3EAny%20chance%20they%20are%20getting%20some%20proxy%20settings%20via%20policy%20that%20keeps%20blocking%20it%20%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-309405%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20ATP%20connection%20closed%20errors%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-309405%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20and%20thanks%20for%20your%20reply%2C%20I%20have%20checked%20all%20prerequisites.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E1.%20server%20is%20helathy%3C%2FP%3E%3CP%3E2.%20No%20Sensor%20errors%20in%20ATP%20portal%3C%2FP%3E%3CP%3E3.%20Sensor%20is%20listed%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20deleted%20the%20ATP%20instance%20and%20created%20one%20from%20scratch.%26nbsp%3B%20however%20when%20I%20try%20to%20connect%3A%26nbsp%3B%3CFONT%3E%3CA%20href%3D%22https%3A%2F%2Fm365x727487sensorapi.atp.azure.com%2F%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fm365x727487sensorapi.atp.azure.com%2F%3C%2FA%3E%3C%2FFONT%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ENote%3A%20m365x727487%20is%20the%20name%20of%20my%20demo%20tenannt%2C%20i%20get%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%3EHTTP%20Error%20503.%20The%20service%20is%20unavailable.%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAfter%20I%20have%20setup%20the%20instance%2C%20I%20can%20browser%20domain%20specific%20objects%2C%20so%20it%20looks%20like%20some%20information%20is%20flowing%20up%20correctly.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-309353%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20ATP%20connection%20closed%20errors%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-309353%22%20slang%3D%22en-US%22%3E%3CP%3Efrom%20the%20tri%20logs%2C%20it%20looks%20like%20the%20sensor%20isn't%20stable%20and%20restarting%2C%20below%20a%20few%20points%20to%20check%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EMake%20sure%20Azure%20ATP%20Prerequisites%20is%3C%2FP%3E%0A%3CP%3EDid%20you%20saw%20outbound%20connection%20limit%20or%20network%20issues%20from%20DC's%20to%20AATP%3C%2FP%3E%0A%3CP%3EMake%20sure%20your%20DC's%20are%20healthy%20and%20connected%20(on%20AATP%20console)%3C%2FP%3E%0A%3CP%3EDo%20you've%20some%20SSL%20inspection%20or%20proxy%20to%20the%20internet%3F%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ETo%20simulate%20an%20attack%20scenario%20with%20AATP%2C%20it%20will%20be%20better%20with%20two%20Windows%20client%20(one%20victim%20and%20one%20attacker).%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
MVP

Hello, 

 

I just deployed Azure ATP in a fresh demo environment. No errors during installation, also when I search for computers or users I do get details, however when I try to trigger an alert by running nslookup -ls -d or run mimikatz, I don't get any alerts in Azure ATP. 

 

The Sensor is installed on a domain controller server 2016 standard , sensor agent version is

2.59.6040.997

 

The domain controller runs in hyper-v and has two nics, one uses the default switch and the other is a private network for the lab. 

 

The Microsoft.Tri.Sensor-Errors log file contains the following errors. 

 

2019-01-03 17:38:50.3349 Error ExceptionDispatchInfo System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server. ---> System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
   at int System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
   at int System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
   --- End of inner exception stack trace ---
   at int System.Net.Security._SslStream.EndRead(IAsyncResult asyncResult)
   at int System.Net.TlsStream.EndRead(IAsyncResult asyncResult)
   at void System.Net.Connection.ReadCallback(IAsyncResult asyncResult)
   --- End of inner exception stack trace ---
   at WebResponse System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at void System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
   --- End of inner exception stack trace ---
   at async Task<HttpResponseMessage> System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task<HttpResponseMessage> sendTask, HttpRequestMessage request, CancellationTokenSource cts, bool disposeCts)
   at async Task<TResponse> Microsoft.Tri.Common.CommunicationWebClient.SendAsync<TResponse>(byte[] requestBytes, int offset, int count)
   at async Task<TResponse> Microsoft.Tri.Common.CommunicationWebClient.SendAsync<TResponse>(IRequestWithResponse<TResponse> request)
   at async Task<TResponse> Microsoft.Tri.Sensor.Common.ServiceProxy<TWebClientConfiguration>.SendAsync<TResponse>(IRequestWithResponse<TResponse> request)
   at async Task Microsoft.Tri.Sensor.EntitySender.SendEntityBatchAsync(EntityBatch entityBatch, EntityBatch postponedEntityBatch)
   at async Task Microsoft.Tri.Sensor.EntitySender.SendEntityBatchesAsync()

 

Any hints where to look wuld be appreciated. 

8 Replies
Highlighted

from the tri logs, it looks like the sensor isn't stable and restarting, below a few points to check:

 

Make sure Azure ATP Prerequisites is

Did you saw outbound connection limit or network issues from DC's to AATP

Make sure your DC's are healthy and connected (on AATP console)

Do you've some SSL inspection or proxy to the internet?

 

To simulate an attack scenario with AATP, it will be better with two Windows client (one victim and one attacker).

Highlighted

Hi and thanks for your reply, I have checked all prerequisites. 

 

1. server is helathy

2. No Sensor errors in ATP portal

3. Sensor is listed

 

I deleted the ATP instance and created one from scratch.  however when I try to connect: https://m365x727487sensorapi.atp.azure.com/

 

Note: m365x727487 is the name of my demo tenannt, i get 

HTTP Error 503. The service is unavailable.

 

After I have setup the instance, I can browser domain specific objects, so it looks like some information is flowing up correctly. 

 

 

 

 

 

 

Highlighted

503 is normal when using a browser.

Something is blocking the connection.

Notice that the services are running under local service & local system,

so they might be getting a different policy compared to your logged in user...

Any chance they are getting some proxy settings via policy that keeps blocking it ?

Highlighted
meanwhile I found this threat, which might be the reason for my issue, however I was able to create an instance. https://techcommunity.microsoft.com/t5/Azure-Advanced-Threat-Protection/Full-Azure-ATP-Trial-for-tes...
Highlighted

Hi, there's no proxy in place, this is a very simple setup, the DC runs in a VM on my notebook that has direct internet connection via my home ISP connection. 

 

But meanwhile I found another article regarding the use of ATP for tenants created via demos.microsoft.com maybe that could be the issue? 

 

Highlighted
Best Response confirmed by Alex Verboon (MVP)
Solution

you can work with a demo environment without any issues.

back to the log, you've some error with: the underlying connection was closed... and that means you've some issue with a connection from your local DC's to the AATP, it could be a connectivity issue, some SSL inspection or even firewall port.

 

 

Highlighted

Hello Eli & Eli, 

 

Thanks for both your responses, I ended u disabling the firewall on my UPC ISP router and suddenly packets where floating as expected. Strange, as this used to work previously, but at least I found the issue. Thanks for your feedback. 

 

Alex

Highlighted

great news, and it makes sense because the log has many errors about connectivity with the AATP.

Now you can start with attack and simulate scenario on your DC's.