OWA/ECP stop working after update ((

Copper Contributor

Hello.

I have one Exchange server under my control. After installing update KB5019758, the admin console stopped working.

I get message

 

 

ASSERT: HMACProvider.GetCertificates:protectionCertificates.Length<1 

 

 

 

[ExAssertException: ASSERT: HMACProvider.GetCertificates:protectionCertificates.Length<1]
   Microsoft.Exchange.Diagnostics.ExAssert.AssertInternal(String formatString, Object[] parameters) +232
   Microsoft.Exchange.Clients.Common.HmacProvider.GetCertificates() +472
   Microsoft.Exchange.Clients.Common.HmacProvider.GetHmacProvider() +143
   Microsoft.Exchange.Clients.Common.HmacProvider.ComputeHmac(Byte[][] messageArrays) +16
   Microsoft.Exchange.HttpProxy.FbaModule.SetCadataCookies(HttpApplication httpApplication) +811
   Microsoft.Exchange.HttpProxy.FbaFormPostProxyRequestHandler.HandleFbaFormPost(BackEndServer backEndServer) +2727
   Microsoft.Exchange.HttpProxy.FbaFormPostProxyRequestHandler.ShouldContinueProxy() +20
   Microsoft.Exchange.HttpProxy.ProxyRequestHandler.BeginProxyRequestOrRecalculate() +229
   Microsoft.Exchange.HttpProxy.ProxyRequestHandler.InternalOnCalculateTargetBackEndCompleted(TargetCalculationCallbackBeacon beacon) +1367
   Microsoft.Exchange.HttpProxy.<>c__DisplayClass3f.<OnCalculateTargetBackEndCompleted>b__3e() +311
   Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(TryDelegate tryDelegate, FilterDelegate filterDelegate, CatchDelegate catchDelegate) +35
   Microsoft.Exchange.HttpProxy.Diagnostics.SendWatsonReportOnUnhandledException(MethodDelegate methodDelegate, LastChanceExceptionHandler exceptionHandler) +120
   Microsoft.Exchange.HttpProxy.ProxyRequestHandler.CallThreadEntranceMethod(MethodDelegate method) +69

[AggregateException: One or more errors have occurred.]
   Microsoft.Exchange.HttpProxy.ProxyRequestHandler.EndProcessRequest(IAsyncResult result) +409
   System.Web.CallHandlerExecutionStep.InvokeEndHandler(IAsyncResult ar) +212
   System.Web.CallHandlerExecutionStep.OnAsyncHandlerCompletion(IAsyncResult ar) +166

 

Сould you please help me to get ECP and OWA working again? What additional information can I provide?

9 Replies

Hi @KosmosKami!

 

OWA/ECP errors after an Exchange Security update is something quite usual.

These errors occur if the security update was manually installed on a server that has User Account Control (UAC) enabled, but without using elevated permissions.

 

Use elevated permissions to reinstall the security update on the server.

-Select Start, and then type cmd.
-Right-click Command Prompt from the search results, and then select Run as administrator.
-If the User Account Control window appears, select the option to open an elevated Command Prompt window, and then select Continue. If the UAC window doesn’t appear, continue to the next step.
-Type the full path of the .msp file for the security update, and then press Enter.
-After the update installs, restart the server.

 

If that doesn't fix your issue, you'll probably need to check the ECP Virtual directory. You can find the detailed instructions here: OWA or ECP stops working after you install a security update - Exchange | Microsoft Learn

 

Hope this helps and please let us know if you finally fix the issue. If not, we'll need to perform further checks. 

 

Good luck :) 

@FcoManigrasso Many thanks for the help. Of course I will try to reinstall the update in the way you indicated. I am interested in figuring out for myself what is the difference between the two methods? In the case of installation by normal startup, a request for privilege escalation appears. Aren't these similar methods?

Hi @KosmosKami,

 

That's a very good question. And unfortunately my answer will not be as clear as desired. 

In many security updates Microsoft suggest to install them through an elevated CMD. 

Why? Below my personal point of view, ( again, it's my personal interpretation and not confirmed by MS ).

Launching the update through the setup file you'll get a prompt for admin privileges. That prompt "interrupt" the native process asking for the permissions to go ahead. During the whole process privileges are required, ( ad, schema, exchange... ), and I think that those privileges aren't inherited correctly from that mentioned first prompt. 

Launching the update from an elevated CMD will not interrupt the process and during the whole time it will identify an admin with the correct roles to install all the required paths. This is why this method causes less issues. 

Again, this is my personal point of view got after many years working with Exchange and installing such updates. 

Maybe @Vasil Michev could give you more detailed info about this topic, or tell if I'm wrong with my statement. ( He's one of the best Exchange engineer that I know ). 

Anyway give it a try... I solved many problems like your one following that MS suggestion.

Many thanks for helping and sharing your knowledge @FcoManigrasso . Right now I have half of the problem ESP is working, but OWA is unavailable. I try navigate to C:\Program Files\Microsoft\Exchange Server\V15\FrontEnd\HttpProxy and take a copy of the SharedWebConfig.config file.  Then Paste a copy of that file into the C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess directory. And then restart the IIS Services (iisreset). Unfortunately, it didn't help I'm also checked the certificate used for https binding, (in IIS.) is the same for the Exchange Front End, and the Exchange Back End web sites.

Hi @KosmosKami,

Happy to hear that ECP is working now.

Regarding OWA, I'll need more info... Which error do you get?

Do you get any log in EV? Which ones?

Please check also that the certificate is still valid. You can check it running:

(Get-AuthConfig).CurrentCertificateThumbprint | Get-ExchangeCertificate | Format-List

@FcoManigrasso On OWA page I am get an uninformative message after authorization

:-(
Something went wrong
Unfortunately, we cannot obtain this information right now. Please reply later. If you encounter problems, please contact support

And I still can't find the detailed log file that is responsible for OWA. I have seen posts online about what to look for in the IIS logs. Where is it on the right path?

Regarding the certificate, I can say that I checked the state of health with a script HealthChecker.ps1 and it warned that the validity of some certificates was coming to an end. So I used an another script MonitorExchangeAuthCertificate.ps1 to renew the certificates and then point them to IIS.

 

Hi @KosmosKami,

You need to check the Event Viewer for the errors ID's when you try to access OWA. 

Please provide also the output of the cmdlt posted in my previous reply, ( you can send me that in a private message ). It's possible that the certificate update failed and that could be the reason of your error. But without more details it's really hard to know. In EV you should be able to see more detailed error pointing to the right root cause.

@FcoManigrasso 

It's quite interesting. I tried to find events using the error or warning filter. It turned out that the event I was interested in was with the information level. Event code 1309. The source is ASP.NET 4.0.30319.0

Event code: 3005
Event message: An unhandled exception occurred.
Event time: 07.03.2023 21:20:45
Event time (UTC): 07.03.2023 18:20:45
Event ID: c51f5e3a06fc4aa8b569417c2d2cbc90
Event sequence: 2
Event occurrence: 1
Event detail code: 0

Application information:
Application domain: /LM/W3SVC/2/ROOT/owa-4603-133226868366278403
Trust level: Full
Application Virtual Path: /owa
Application Path: C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\owa\
Machine name: EMAIL

Process information:
Process ID: 20032
Process name: w3wp.exe
Account name: NT AUTHORITY\system

Exception information:
Exception type: ArgumentException
Exception message: An element with the same key has already been added.
in System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
in System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at Microsoft.Exchange.Clients.Owa.Core.OwaSettingsLoaderBase.InitializeLocalVersionFolders()
in Microsoft.Exchange.Clients.Owa.Core.OwaSettingsLoaderBase.Load()
at Microsoft.Exchange.Clients.Owa.Core.OwaSettingsLoader.Load()
in Microsoft.Exchange.Clients.Owa.Core.OwaApplicationBase.ExecuteApplicationStart(Object sender, EventArgs e)
in Microsoft.Exchange.Clients.Owa.Core.OwaModule.Init(HttpApplication context)
in System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr AppContext, HttpContext context, MethodInfo[] handlers)
in System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr AppContext, HttpContext context)
in System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr AppContext, HttpContext context)
in System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext)



Request information:
Request URL: https://email.contoso.com:444/owa
Request path: /owa
User host address: my-ip-address
User:
Is authenticated: False
Authentication Type:
Thread account name: NT AUTHORITY\system

Thread information:
Thread ID: 63
Thread account name: NT AUTHORITY\system
Is impersonating: False
Stack trace: in System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
in System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at Microsoft.Exchange.Clients.Owa.Core.OwaSettingsLoaderBase.InitializeLocalVersionFolders()
in Microsoft.Exchange.Clients.Owa.Core.OwaSettingsLoaderBase.Load()
in Microsoft.Exchange.Clients.Owa.Core.OwaSettingsLoader.Load()
in Microsoft.Exchange.Clients.Owa.Core.OwaApplicationBase.ExecuteApplicationStart(Object sender, EventArgs e)
at Microsoft.Exchange.Clients.Owa.Core.OwaModule.Init(HttpApplication context)
in System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr AppContext, HttpContext context, MethodInfo[] handlers)
in System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr AppContext, HttpContext context)
in System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr AppContext, HttpContext context)
in System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext)


Custom event details:

 

 

Hi @KosmosKami,

From the EV log, it still seems that something is wrong with the SharedWebConfig.config file.

Maybe that copy paste wasn't the best way. Let me suggest the following steps:

- Navigate to C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess and if your pasted SharedWebConfig.config file is still there, move it to another location as backup.

- Run cd %ExchangeInstallPath%\bin to change the current directory to the bin folder that's under the Exchange installation path.

Use the DependentAssemblyGenerator.exe tool to generate the file:

 

DependentAssemblyGenerator.exe -exchangePath "%ExchangeInstallPath%\bin" -exchangePath "%ExchangeInstallPath%\ClientAccess" -configFile "%ExchangeInstallPath%\ClientAccess\SharedWebConfig.config"

 

- Restart the Server.

 

( Source: Event ID 1309 and you can't access OWA and ECP after you install Exchange Server 2016 or Exchange Se... ).

 

Hope this helps.