EX2019-CU10 OWA/ECP not working after July Security Update

Emil Gustafsson 266

Hello,

After installing the July Security update access to ECP and OWA is broken.
Mail Flow works, but accessing OWA or ECP returns the following error:

ASSERT: HMACProvider.GetCertificates:protectionCertificates.Length<1

It also logs error 1003 to the Event Logs.

As many others have suggested, we have tried replacing the OATH Certificate according to this: https://learn.microsoft.com/en-us/exchange/troubleshoot/administration/cannot-access-owa-or-ecp-if-oauth-expired several times, we also waited >60 minutes after doing it - but the error persists. Even after full server reboot.

Please advice on what to do next.

Full Stack Trace Here:

Server Error in '/owa' Application.  
ASSERT: HMACProvider.GetCertificates:protectionCertificates.Length<1  
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.  
  
Exception Details: Microsoft.Exchange.Diagnostics.ExAssertException: ASSERT: HMACProvider.GetCertificates:protectionCertificates.Length<1  
  
Source Error:  
  
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.  
  
Stack Trace:  
  
  
[ExAssertException: ASSERT: HMACProvider.GetCertificates:protectionCertificates.Length<1]  
   Microsoft.Exchange.Diagnostics.ExAssert.AssertInternal(String formatString, Object[] parameters) +241  
   Microsoft.Exchange.Diagnostics.ExAssert.RetailAssert(Boolean condition, String formatString, T1 parameter1, T2 parameter2) +2694334  
   Microsoft.Exchange.Clients.Common.HmacProvider.GetCertificates() +363  
   Microsoft.Exchange.Clients.Common.HmacProvider.GetHmacProvider() +140  
   Microsoft.Exchange.Clients.Common.HmacProvider.ComputeHmac(Byte[][] messageArrays) +14  
   Microsoft.Exchange.HttpProxy.FbaModule.SetCadataCookies(HttpApplication httpApplication) +1032  
   Microsoft.Exchange.HttpProxy.FbaFormPostProxyRequestHandler.HandleFbaFormPost(BackEndServer backEndServer) +3581  
   Microsoft.Exchange.HttpProxy.FbaFormPostProxyRequestHandler.ShouldContinueProxy() +20  
   Microsoft.Exchange.HttpProxy.ProxyRequestHandler.BeginProxyRequestOrRecalculate() +257  
   Microsoft.Exchange.HttpProxy.ProxyRequestHandler.InternalOnCalculateTargetBackEndCompleted(TargetCalculationCallbackBeacon beacon) +1528  
   Microsoft.Exchange.HttpProxy.<>c__DisplayClass280_0.<OnCalculateTargetBackEndCompleted>b__0() +303  
   Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate) +35  
   Microsoft.Exchange.HttpProxy.ProxyRequestHandler.CallThreadEntranceMethod(Action method) +59  
  
[AggregateException: One or more errors occurred.]  
   Microsoft.Exchange.HttpProxy.ProxyRequestHandler.EndProcessRequest(IAsyncResult result) +414  
   System.Web.CallHandlerExecutionStep.InvokeEndHandler(IAsyncResult ar) +231  
   System.Web.CallHandlerExecutionStep.OnAsyncHandlerCompletion(IAsyncResult ar) +172  
  
  
Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.8.4330.0

Willem Hendrik Berkhof 107 Reputation points

2021-07-14T19:46:21.55+00:00

Same problem here since Exchange security update last night. Recreating and assigning a new certificate will not resolve the problem.
Outlook works fine!, but OWA and ECP only work to signin page, after typing credentials it gives the message.

Hope someone can help to fix this!
Eric Yin-MSFT 4,386 Reputation points

2021-07-15T02:23:49.36+00:00

Hi,
Are you accessing ECP/OWA via a load balanced URL? If so, try directly accessed from the server URL.
TomBlue201 1 Reputation point

2021-07-27T18:47:16.05+00:00

I experience the same error. Currently 2 servers, on one server the workaround to re-create the OAUTH certificate worked well. On the other one, it worked at first glance, but after the next reboot the error appeared again. After removing the Security Update KB5007480 everything's ok again.

As it is designated to be a "hybrid" server by next week, I of course do not want to operate this server without that update.

Does anybody experience the same issue where the re-creation of the OAUTH cert is not a permanent solution?
Has anybody solved that issue?

Cheers, Thomas
Chet Wolosek 1 Reputation point

2021-08-24T16:16:37.33+00:00

If you are running a local exchange server in Hybrid mode just run the latest version of the Hybrid Configuration Wizard and follow the steps to reconfigure the connection. Gave it 30 minutes after going through the configuration and now everything is working again.
Chet Wolosek 1 Reputation point

2021-08-24T16:16:48.607+00:00

If you are running a local exchange server in Hybrid mode just run the latest version of the Hybrid Configuration Wizard and follow the steps to reconfigure the connection. Gave it 30 minutes after going through the configuration and now everything is working again.

Accepted answer

Willem Hendrik Berkhof 107 Reputation points

2021-07-14T21:08:15.277+00:00

Followed this instructions has solved the problem:
https://learn.microsoft.com/en-us/exchange/troubleshoot/administration/cannot-access-owa-or-ecp-if-oauth-expired

But is does take time (more then one hour) before it works.
Please sign in to rate this answer.

4 people found this answer helpful.
Emil Gustafsson 266 Reputation points

2021-07-15T04:14:59.113+00:00

Can confirm this.

I renew the cert for like the fifth time and then went to bed.

Woke up and saw this comment and checked, and mine works now too.

Romeo S 16 Reputation points

2021-07-15T04:18:21.08+00:00

Same here, never had so much anxiety after waking up but at least I'm awake now :-))

Emil Gustafsson 266 Reputation points

2021-07-15T04:19:56.227+00:00

Yeah really got the system going and jump out of bed didn't it :D

Trank0™ ®☺ 1 Reputation point

2021-07-15T19:40:33.137+00:00

Yeah, you are right, my cert gonna expire on year 26 but at event viewer it don´´t stop to alert me since I did yesterday the "security Exchange 2016 update" and this link to renew and apply the new cert only to the auth, and wait a couple of hours, did it only in one server and it fix every OWAs and ECPs

thanks a lot, so fast to report this Microsoft fail, in my case I updated test enviroment, on saturday I´m gonna do on producction, we´ll see but I hope I only will have to do this on first room servers.

thanks a lot Willem

Best regards and keep save

Anne Gast 1 Reputation point

2021-07-15T22:55:48.96+00:00

Thank you so much for this link. We have a simple configuration, so going through all the steps all the way down through restarting IIS fixed our issue immediately. Now I can go back to being on vacation.

Adrian Marin 1 Reputation point

2021-07-17T14:21:40.58+00:00

You can try to copy "Microsoft Exchange Server Auth Certificate" to the "Trusted Root Certification Authorities" on all servers in order to restore the ecp and owa functionality through hardware balancer.
For me these worked: if you go to specific server with http://server.domain/owa worked OK but for the https://vip.domain/owa the page prompted continously for credentials.

Jesse Flintoff 211 Reputation points

2021-07-18T07:02:21.083+00:00

Thank youu

Polaski, Kenneth 1 Reputation point

2021-07-22T14:44:58.783+00:00

Works great. Thanks for the link. No waiting time. The problem was fixed after the last command.

Joshua Thompson 156 Reputation points

2021-08-03T17:46:54.347+00:00

Late installer to the game. Same issue here. The only certificate that I see that is expired is my Exchange Federation Delegation cert. is this the same one that you are referring to as expired?

Will following the link to fix the cert issues impact mail flow at all. We run hybrid.

What is this oAuth certificate used for? Could it be listed under a different name in my exchange certificates?

Anton Yundin 1 Reputation point

2021-11-11T13:29:18.177+00:00

It worked for me after 5 hours of waiting, EXCH2016CU21
Sign in to comment

10 additional answers

Marcel De Oliveira Lima 1 Reputation point

2021-11-06T01:52:15.66+00:00

Just out of curiosity: I had this problem with a 2019 Exchange Server in Hybrid mode and another one but a fresh install (AD & exchange - lab demo). Both servers took hours to get it resolved but the last one as it is hosted in a Demo Environment I shutdown the whole network and on the next day the Exchange server was back on.
It seems like the waiting hours are not related with leaving the server on but something regarding the certificate publishing/date/time. Does anybody know what happens under the hood and could give an better explanation, rather than "In some environments, it may take an hour for the OAuth certificate to be published." from https://learn.microsoft.com/en-us/exchange/troubleshoot/administration/cannot-access-owa-or-ecp-if-oauth-expired ?
Please sign in to rate this answer.

0 comments No comments
Sign in to comment
Anton Yundin 1 Reputation point

2021-11-11T13:28:23.963+00:00

It worked for me after 5 hours of waiting, EXCH2016CU21
Please sign in to rate this answer.

0 comments No comments
Sign in to comment