Suspicious events

Copper Contributor

Exchange 2016 fully patched.

Saw a few errors in Application log.

 

Source: MSExchange Front End HTTP Proxy

[Owa] An internal server error occurred. The unhandled exception was: System.ArgumentException: Invalid input value
Parameter name: input
at Microsoft.Exchange.Data.ApplicationLogic.Cafe.BackEndServer.FromString(String input)
at Microsoft.Exchange.HttpProxy.OwaResourceProxyRequestHandler.ResolveAnchorMailbox()
at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.InternalBeginCalculateTargetBackEnd(AnchorMailbox& anchorMailbox)
at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.<BeginCalculateTargetBackEnd>b__280_0()
at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate)

 

Source: ASP.NET 4.0.30319.0

Event code: 3005
Event message: An unhandled exception has occurred.
Event time: 3/8/2021 5:33:57 AM
Event time (UTC): 3/8/2021 1:33:57 PM
Event ID: 049c535e9be849829a634bccfc74e4ea
Event sequence: 5
Event occurrence: 4
Event detail code: 0

Application information:
Application domain: /LM/W3SVC/1/ROOT/owa-1-132593003067932026
Trust level: Full
Application Virtual Path: /owa
Application Path: C:\Program Files\Microsoft\Exchange Server\V15\FrontEnd\HttpProxy\owa\
Machine name: EXCH

Process information:
Process ID: 12956
Process name: w3wp.exe
Account name: NT AUTHORITY\SYSTEM

Exception information:
Exception type: ArgumentException
Exception message: Invalid input value
Parameter name: input
at Microsoft.Exchange.Data.ApplicationLogic.Cafe.BackEndServer.FromString(String input)
at Microsoft.Exchange.HttpProxy.OwaResourceProxyRequestHandler.ResolveAnchorMailbox()
at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.InternalBeginCalculateTargetBackEnd(AnchorMailbox& anchorMailbox)
at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.<BeginCalculateTargetBackEnd>b__280_0()
at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate)
at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.CallThreadEntranceMethod(Action method)



Request information:
Request URL: https://public_ip:443/owa/auth/x.js
Request path: /owa/auth/x.js
User host address: 35.244.82.13
User:
Is authenticated: False
Authentication Type:
Thread account name: NT AUTHORITY\SYSTEM

Thread information:
Thread ID: 7
Thread account name: NT AUTHORITY\SYSTEM
Is impersonating: False
Stack trace: at Microsoft.Exchange.Data.ApplicationLogic.Cafe.BackEndServer.FromString(String input)
at Microsoft.Exchange.HttpProxy.OwaResourceProxyRequestHandler.ResolveAnchorMailbox()
at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.InternalBeginCalculateTargetBackEnd(AnchorMailbox& anchorMailbox)
at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.<BeginCalculateTargetBackEnd>b__280_0()
at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate)
at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.CallThreadEntranceMethod(Action method)


Custom event details:

 

Source: C:\Program Files\Microsoft\Exchange Server\V15\Logging\HttpProxy\Owa\HttpProxy_2021030813-1.LOG

2021-03-08T13:33:57.477Z,8b72ab0b-1b16-46cf-b84e-48d6cbfa7b45,15,1,2176,9,,Owa,public_ip,/owa/auth/x.js,,FBA,false,,,,Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/88.0.4324.182 Safari/537.36 Edg/88.0.705.81,35.244.82.13,EXCH,302,,,GET,,,,,X-AnonResource-Backend-Cookie,,,,0,,,,0,,,0,,0,,0,0,,0,106,0,,,,,,,,,0,104,2,,106,,106,106,,,,BeginRequest=2021-03-08T13:33:57.371Z;CorrelationID=<empty>;ProxyState-Run=None;ProxyState-Complete=CalculateBackEnd;SharedCacheGuard=0;EndRequest=2021-03-08T13:33:57.477Z;,UnexpectedException=System.ArgumentException: Invalid input value Parameter name: input at Microsoft.Exchange.Data.ApplicationLogic.Cafe.BackEndServer.FromString(String input) at Microsoft.Exchange.HttpProxy.OwaResourceProxyRequestHandler.ResolveAnchorMailbox() at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.InternalBeginCalculateTargetBackEnd(AnchorMailbox& anchorMailbox) at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.<BeginCalculateTargetBackEnd>b__280_0() at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate Func`2 filterDelegate Action`1 catchDelegate) at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.CallThreadEntranceMethod(Action method);,,,,,

 

Also

2021-03-06T15:31:05.660Z,16a4dee4-37b2-430f-8df4-3bc228d55faf,15,1,2176,9,,Owa,mail.example.com,/owa/auth/x.js,,FBA,false,,,,Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html),104.225.219.16,EXCH,302,,,GET,,,,,X-AnonResource-Backend-Cookie,,,,0...: Invalid input value Parameter name: input at Microsoft.Exchange.Data.ApplicationLogic.Cafe.BackEndServer.FromString(String input) at Microsoft.Exchange.HttpProxy.OwaResourceProxyRequestHandler.ResolveAnchorMailbox() at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.InternalBeginCalculateTargetBackEnd(AnchorMailbox& anchorMailbox) at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.<BeginCalculateTargetBackEnd>b__280_0() at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate Func`2 filterDelegate Action`1 catchDelegate) at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.CallThreadEntranceMethod(Action method);,,,,,

 

Is this some kind of new exploit?

17 Replies

I have also started seeing these on my Exchange 2016 server that is fully patched with the latest CU. The errors are identical to yours. I have not been able to determine what these are and I've opened a support request with Microsoft to see if they can help figure out what is going on. This started on Saturday the 6th for me.

@Jason284 , i have the same event showing on several Exchange servers, all patched with the latest CUs and patches since 3.3.2021...

Hi, I have the same event in our Exchange Server 2016. "/ecp/default.flt","X-BEResource-Cookie", "/owa/auth/x.js","X-AnonResource-Backend-Cookie", /ecp/y.js","X-BEResource-Cookie"
Since most of our users are on mobile device I start to block external access to OWA with IP Address and Domain Restriction. ECP is already on secondary IP address without internet.
I am seeing the same exact error on our Exchange 2013 server. This happened the evening after we applied the zero day patch (KB5000871). Please let us know what you find out from Microsoft.

Seeing this message as Event ID 1003, Source MSExchange Front End HTTP Proxy, AND as Event ID 1309, Source ASP.NET 4.0.30319.0. With a

@RrrrowsdowerThis also appears to have started in less than 24 hours after installing the Exchange zero day patch for me as well. 

@Jason284 - Did you get any update from Microsoft on your case?

@DhruvaKudvaUnfortunately no, not a word yet.  Although I have noticed that there are a lot of people getting these exact errors and it seems to correlate with installing the Exchange zero day patch last week from what I can tell.  It's frustrating there hasn't been any response from Microsoft on this.

From what I understand, this could be due to Mailbox Anchoring. When an External connection is made (OWA) Exchange looks for the users mailbox on the same node as the Transport node.

If the users mailbox is not the same node as the transport node, it throws this error.
We get this error as well and each time it occurs, the users mailbox is NOT on the same node as the transport. Doesnt impact anything as the DAG handles this but it does seem to throw this error each time.

@MS_Tech_user1875 Do you got any feedback from MS? I see this events on fully patched exchange servers 2013 & 2016, at all of the servers the Test-ProxyLogon.ps1 found entries and webshells like discovery.aspx. Every server was cleaned and daily checked with ps scripts, msert scan. All the bad IPs are blocked  Before we blocked the bad IPs, we get the same events ASP.NET 4.... Web Event with owa an x.js Files. 

 

what is unclear to me, is this still a problem which indicates a compromise or active hacker access or are these prevented access attempts and the server is safe for now?

We just received an email from our ISP that they have detected activity suggesting our OWA was compromised. Not 100% sure but this may be evidence of exploitation. Investigating currently

I have a case open with Microsoft about this but I still have not heard anything from them. As of this morning I have completely disabled external access to OWA until we can get some answers as to what is really going on.
is there any udpate regarding this case?
Unfortunately no. I never heard back from Microsoft at all after opening a case, which isn't the greatest feeling. From what I can tell, and have researched myself, these error logs do appear to be related to the Exchange exploit, but whether or not it means you have been breached I don't know. I have ran all of Microsoft's scripts to search for any indication of compromise, which all came back clean on my server, so even though I was seeing these errors it doesn't appear that I have been compromised. I did restrict OWA to only my internal subnet temporarily until there is more information from Microsoft.
Same here, our ISP told us they detected activity suggesting compromise yet we have been testing at least weekly with defender scans and have nothing. All mitigations applied and no evidence has been found of exploitation on our end.
ok.. pls update if you get any additional info, thanks
good that you are'nt finding any threats. hope you've ran the testproxylogon.ps1 script released by microsoft.