Released: March 2024 Exchange Server Security Updates

[The_Exchange_Team]

Update 4/23/2024: We have now released April 2024 Hotfix Updates which address known issues in March 2024 SU updates.

Microsoft has released Security Updates (SUs) for vulnerabilities found in:

• Exchange Server 2019
• Exchange Server 2016

SUs are available for the following specific versions of Exchange Server:

• Exchange Server 2019 CU13 and CU14
• Exchange Server 2016 CU23

The March 2024 SUs address vulnerabilities responsibly reported to Microsoft by security partners and found through Microsoft’s internal processes. Although we are not aware of any active exploits in the wild, our recommendation is to immediately install these updates to protect your environment.

These vulnerabilities affect Exchange Server. Exchange Online customers are already protected from the vulnerabilities addressed by these SUs and do not need to take any action other than updating any Exchange servers or Exchange Management tools workstations in their environment.

More details about specific CVEs can be found in the Security Update Guide (filter on Exchange Server under Product Family).

Security Advisory ADV24199947 information

After you install this security update, Exchange Server no longer uses Oracle Outside In Technology (also known as OutsideInModule or OIT). OIT performs text extraction operations when processing email messages that have attachments in Exchange Transport Rule (ETR) and Data Loss Prevention (DLP) scenarios.

For more information, see The OutsideInModule module is disabled after installing the March 2024 SU.

Update installation

The following update paths are available:

• Install the latest CU. Use the Exchange Update Wizard to choose your current CU and your target CU to get directions.
• Inventory your Exchange Servers to determine which updates are needed using the Exchange Server Health Checker script. Running this script will tell you if any of your Exchange Servers are behind on updates (CUs, SUs, or manual actions).
• Re-run the Health Checker after you install an SU to see if any further actions are needed.
• If you encounter errors during or after installation of Exchange Server, run the SetupAssist script. If something does not work properly after updates, see Repair failed installations of Exchange Cumulative and Security updates.

Known issues with this release (please see April 2024 Hotfix Update for resolution):

• Download domains not working after installing the March 2024 SU
• OwaDeepTestProbe and EacBackEndLogonProbe fail after installing March 2024 SU
• Search error in Outlook cached mode after installing March 2024 SU
• After installation of the March 2024 SU, newly received messages might show yellow envelope icon in Outlook client, no matter the read status of the message. This issue is cosmetic.
• Certain add-ins (such as My Templates) might not work properly after March 2024 SU is installed.
• Published calendars might not work with a "This calendar isn't available" error.

FAQs

Our organization is in Hybrid mode with Exchange Online. Do we need to do anything?
Exchange Online is already protected, but this SU needs to be installed on your Exchange servers, even if they are used only for management purposes. If you change the auth certificate after installing an SU, you should re-run the Hybrid Configuration Wizard.

The last SU we installed is a few months old. Do we need to install all SUs in order to install the latest one?
SUs are cumulative. If you are running a CU supported by the SU, you do not need to install all SUs in sequential order; simply install the latest SU. Please see this blog post for more information.

Do we need to install SUs on all Exchange Servers within our organization? What about ‘Management Tools only’ machines?
Our recommendation is to install SUs on all Exchange Servers and all servers and workstations running the Exchange Management Tools to ensure compatibility between management tools clients and servers. If you are trying to update the Exchange Management Tools in the environment with no running Exchange servers, please see this.

Documentation may not be fully available at the time this post is published.

Blog post updates:

• 4/23/2024: Added a banner and links to April 2024 Hotfix Updates which address known issues with March 2024 SU
• 4/5/2024: Added a known issue with published calendars.
• 3/28/2024: Added a known issue where some add-ins are not working properly.
• 3/22/2024: Added a 'yellow envelope icon' issue to known issues.
• 3/14/2024: Added a workaround for the Outlook Search issue that some environments (not all) can experience.

The Exchange Server Team

[Bhalchandra_Atre-MSFT]

@Martin_Aigner Thanks for reaching out, we have enough data at the moment for diagnosis.

[Martin_Aigner]

@PankajNTT  Mail from EXO to EXO doesn't flow via onPrem in our environment so i don't expect any side effect here - the experience may be different for other setups for sure. Seems something is going on in the transport here.

[Nasir Ali]

For the search issue with outlook 

Please see the KB for the workaround
Search error in Outlook cached mode after installing March 2024 SU - Microsoft Support

[Sam_T]

@The_Exchange_Team 

We are aware that "OwaDeepTestProbe and EacBackEndLogonProbe fail after installing the March 2024 SU" however after application of this SU on an Exchange 2016 server, "HealthManagerWorkItemQuarantineMonitor" is also coming up as "Unhealthy".  It was in a Healthy state immediately prior to installation of the March SU.  Is this behavior expected ? 

[Zaher124]

Hi,

After patching, all users outlook keeps asking for username and password, I could use OWA just fine and all services are ok.

Exchange 2016, outlook 2016 as well.

I could see a lot of 401 on the load balancer log, however when connecting directly without load balancer outlook works ok.

Load balancer is pulse secure vtm 21.4

Please help.

Zaher

[nikitinns]

Good evening.

Any news regarding fix for this excellent update?

Has anyone uninstalled this update?

[Zaher124]

@nikitinns I uninstalled it and still have the issues...

[edisontts]

Hi @The_Exchange_Team 

3 bugs after the Febuarry and March security updates on the microsoft exchange 2019.

1. ) Email recall functionality broken due to Extended Protection enabled (February 2024 Security Patch).
2. ) Outlook clients unable to search from the server (March 2024 Security Patch).
3.) Unread envelope icon affected for internal users' received emails (March 2024 Security Patch).

Kindly advise.!!

[Bhalchandra_Atre-MSFT]

@Zaher124 Please check if the extended protection is enabled, more details here

[Matthias Nguyen]

After the update we have massive problems with SMTP anonymous connector. Our printer can't send mails...

[Bhalchandra_Atre-MSFT]

@Matthias Nguyen No such known issues with anonymous connector, I suggest to open support ticket for further debugging of your issue.

[Matthias Nguyen]

@Bhalchandra_Atre-MSFT Thanks for your reply.

I'm not the only one with this issue. I have read in other forums about that issue too. 

[Bhalchandra_Atre-MSFT]

@Matthias Nguyen The anonymous connector works fine in my test lab with SU installed. Can you message the settings of your connector directly to me? 

[PankajNTT]

@Bhalchandra_Atre-MSFT @The_Exchange_Team We have all users hosted on ExO  but we have SMTP relay running on server which sends email through Printers, application to O365 from there to Internet.

Please let me know if you have seen any issue in emails from Exchange on-premise (SMTP Relay) to O365 or any connector issue. I have seen one user mentioned his SMTP connector is not working after installing update.

I dont see any clear communication from Microsoft what issues occurred after Installing March update.  can you please comment what is impacted or not impacted.

[Martin_Aigner]

We see also issues in message transport and EWS getting unresponsive in between:

Agent 'Mailbox Rules Agent' caused an unhandled exception 'StorageTransientException: Query/Update out of office history operation.' while handling event 'OnDeliveredMessage'

What we see in the queues: STOREDRV.Deliver.Exception:StorageTransientException.MapiExceptionLockViolation; Failed to process message due to a transient exception with message Query/Update out of office history operation.

Opening up a Support case now

[Bhalchandra_Atre-MSFT]

@PankajNTT The blog is regularly updated with the known issues, so far there are no issues found with SMTP relay.

[pazzoide76]

Same problem of Unread envelope icon affected for internal users' received emails with two 2016 hybrid exchanges and with two other 2016 non-hybrid exchanges

[muhammetsukruozdemir]

Office online server stopped working.

Exchange inetpub log
2024-03-18 11:37:012001:xxxx:xxxx::3005:e6dd:669e:1d8c:5fde GET /owa/auth/errorFE.aspx httpCode=500&CorrelationID=<empty>;&cafeReqId=5a78e230-b060-487a-bc51-c928a2bd53f1;&encoding=; 443 
2024-03-18 11:37:012001:xxxx:xxxx::3005:e6dd:669e:1d8c:5fde GET /owa/email address removed for privacy reasons/wopi/files/@/owaatt owaatt=LVMtMS01LTIxLTM3NDI4NTAxMzMtMjkwNzUyOTA2MC
2024-03-18 11:37:422001:xxxx:xxxx::3005:e6dd:669e:1d8c:5fde GET /owa/auth/errorFE.aspx httpCode=500&CorrelationID=<empty>;&cafeReqId=c4578f14-5348-4ca2-8c98-976247a8f7d0;&encoding=; 443
2024-03-18 11:37:422001:xxxx:xxxx::3005:e6dd:669e:1d8c:5fde GET /owa/email address removed for privacy reasons/wopi/files/@/owaatt owaatt=LVMtMS01LTIxLTM3NDI4NTAxMzMtMjkwNzUyOTA2MC
2024-03-18 11:37:482001:xxxx:xxxx::3005:e6dd:669e:1d8c:5fde GET /owa/auth/errorFE.aspx httpCode=500&CorrelationID=<empty>;&cafeReqId=ee8be3fc-21b5-4804-a6a5-813d02103c01;&encoding=; 443
2024-03-18 11:37:482001:xxxx:xxxx::3005:e6dd:669e:1d8c:5fde GET /owa/email address removed for privacy reasons/wopi/files/@/owaatt owaatt=LVMtMS01LTIxLTM3NDI4NTAxMzMtMjkwNzUyOTA2MC



Office Online Server Log
2024-03-18 11:37:39.74	w3wp.exe#WebWordViewer (0x1514)	0x542C	Hosted Office Online	WAC Hosting Interaction	buqbb	Monitorable	{"WacRequestId":"6f6921d3-def1-491c-be42-f3780de9f72d","request-id":"c4578f14-5348-4ca2-8c98-976247a8f7d0","X-FEServer":"EX19-01","httpStatusCode":"InternalServerError","logPrefix":"Wopi,CheckFile,WACSERVER","requestDurationMs":"79","bytesRead":"0","bytesWritten":"0"}	8b624c18-5b1c-4e22-9993-ebc1d28831d1
2024-03-18 11:37:39.74	w3wp.exe#WebWordViewer (0x1514)	0x542C	Hosted Office Online	WAC Hosting Interaction	buqa8	Unexpected	Wopi,CheckFile,WACSERVER ServerError [url:UREDACTED_(3AyqGr3v459TFiFE+E46KM/aSp6aBN6Ge0qGPQANrqI=)]	8b624c18-5b1c-4e22-9993-ebc1d28831d1
2024-03-18 11:37:39.74	w3wp.exe#WebWordViewer (0x1514)	0x542C	Hosted Office Online	WAC Hosting Interaction	adhsk	Unexpected	WOPI CheckFile: Catch-All Failure [exception:Microsoft.Office.Web.Common.EnvironmentAdapters.UnexpectedErrorException: WOPI    at Microsoft.Office.Web.Apps.Common.WopiTalky.AddHostResponseDataAndThrow(Exception exception, HttpRequestAsyncResult result)    at Microsoft.Office.Web.Apps.Common.WopiTalky.LogAndThrowWireException(HttpRequestAsyncResult result, HttpRequestAsyncException delayedException)    at Microsoft.Office.Web.Apps.Common.WopiDocument.LogAndThrowWireException(HttpRequestAsyncResult result, HttpRequestAsyncException delayedException)    at Microsoft.Office.Web.Common.HttpRequestAsync.End()    at Microsoft.Office.Web.Apps.Common.WopiDocument.CheckWopiFile()]	8b624c18-5b1c-4e22-9993-ebc1d28831d1

[DanySupernova]

@muhammetsukruozdemir 

Same problem here... Office Online Server integration doesn't work anymore in our Exchange 2019 CU14 after this patch

[Jaroslaw2335]

In my case, migrating to another database helps.

[stefandechert]

Hi guys, not really related to the Exchange SU, but as here are some guys reading who also have quite big installations and might also face the following issue:

Since installation of the march updates (Exchange as well as regular Windows Server updates) most of our DCs show constantly increasing lsass memory usage (until they die).

Feels like the same lsass memory leak issue like in november 2022.

We can see that in many different environments no matter if it´s a domain with or without Exchange and no matter which Server version (2016, 2019, 2022).

It´s a quite moderate increase. In some smaller environments we only noticed by checking the monitoring deeply. In some really big environments the increase per hour is way bigger.

Appreciate your responses

[Markus_Meister]

Hi @stefandechert Have you seen the increased memory usage for lsass see only applying the march patches?

[ceantuco]

@stefandechert 

I just checked a patched and un-patched domain controller and you are correct. The patched DC lsaas memory utilization is currently at 685K meanwhile my secondary DC which has not been patched is at 141K. I wonder if it it will continue to increase as days go by. 

I posted it on Reddit's patch Tuesday mega thread.

[stefandechert]

Hi @Markus_Meister, it occurs since applying the march updates.

[Markus_Meister]

When is for the mentioned problem witch search and unread flag an update expected?

2. ) Outlook clients unable to search from the server (March 2024 Security Patch).
3.) Unread envelope icon affected for internal users' received emails (March 2024 Security Patch).

We have our patching of OS for PROD Systems planned for next saturday, but with this problem I wait with the exchange server patches another week and hope it is fixed soon.

[Zaher124]

Hi @Bhalchandra Atre 

Extended Protection is disabled, outlook continues to ask for username and password.

All other services are running perfectly fine even OWA. 

Any tips?

Zaher

[ceantuco]

@Markus_Meister I am also holding off installing the Mar SU until MS releases a fix. 

[Bhalchandra_Atre-MSFT]

@Zaher124 check for any known issues reported by Exchange health checker, if that doesn't help then open a support ticket please.

[a2hawks]

So I am seeing the yellow envelope issue at a customer. If they use a contact I have set up on their exchange server, I get the envelope. If they enter in my email address I don't get the envelope. They have EX2016 on prem and I am on Exchange Online. They are on Outlook 2016 and I am on 365.

If I reply to the email, the yellow envelope goes away.

[boogieshafer]

@The_Exchange_Team 

regarding the workaround for the search problem, i see there is a setting in the Office 2016 admx files for "Disable Hybrid Searching"

i'm curious if this GPO template based option is preferable?/equivalent? to the 2 explicit registry entries already mentioned

[Zaher124]

The outlook issue is fixed, for some reason the patch removed windows auth from mapi in the backend.

[adixro]

Hi @Nino Bilic would this count towards MS announced throttling for Exch 2016 as of March 24th if we defer this somewhat patch (well...30 days monitoring)?

[stefandechert]

Hi guys, news about the lsass memory leak thing:

We deinstalled the Windows Server March 2024 update from an affected domain controller and the issue was gone instantly!

[SevenOfNine]

@stefandechert

Yes, we have the same isssue. Memory usage of DCs constantly increasing after the march upates.

[stefandechert]

We´ve just opened a support case for the lsass issue.

Curious about the result...

[ceantuco]

Hi @stefandechert @SevenOfNine since the lsaas memory consumption is not crashing my DC yet, i will not remove the march update. However, I am going to hold off installing the update on my second DC for now. I rebooted the server and lsaas has gone from 700K to 55K. 

Please let us know what MS says about this issue. 

Thank you! 

[SteveTH]

@stefandechert which patch number did you uninstall exactly on the DCs?

[DanySupernova]

Guys, I read that even after uninstalling the update the problems persist. Question: but what if I reapplied the CU14 to my Exchange 2019 CU14? Do you think it would work?

[stefandechert]

@ceantuco ok, I´ll keep you current. Saw multiple other blogs/forums/threads with people encounting the exact same issue.

@SteveTH it was the March Security Update KB5035857 on Server 2022. We will now uninstall from all DCs (except two or three to have some examples for our MS support case).

[Stefan_Just]

It seems that the public calendar sharing feature (using owa calendar urls) is not working anymore since we installed the march 2024 cu. Can anyone else confirm this issue?

[Terrymmd91]

Anyone else having issues with message recall since the update? Someone attempted one today and it did not process, the "recall" message just sat in the inbox. Might have something to do with the unread message envelope issue.

[embert2165]

I just tried the message recall to an outside email address, and it worked no issue on my side using exchange 2019 on Premise with outlook LYSC 2021

[a2hawks]

@Terrymmd91I have had one user recalling an internal email that has reported this issue on EX2016 latest CU and patch. Someone else in this thread also reported an issue with Recall. The issue was as you described, the recall message sat in the inbox.

[Terrymmd91]

@embert2165 I think the key there is the external address, which makes sense if it is tied to the read envelop issue since it's an internal-only issue. Thanks for checking.

@a2hawksThanks for the info,  we are EX2019 on the latest CU. 

[amstuartc]

I have reports of the Envelope issue within Outlook in Cached mode on an Exchange 2016 CU23 Site - Any news from Microsoft as to a Fix for this issue or will it be advised to remove the Security Patch ?

[Bhalchandra_Atre-MSFT]

@a2hawks Message recall is issue with specific to Outlook update, please check this article https://support.microsoft.com/en-us/office/recall-message-in-outlook-desktop-stops-working-after-feb...

[a2hawks]

@Bhalchandra_Atre-MSFTIs there a time frame for the Office 2016 patch? Or has it already been deployed?

[edisontts]

@Bhalchandra_Atre-MSFT 

Message recall is issue with specific to Outlook update, please check this article https://support.microsoft.com/en-us/office/recall-message-in-outlook-desktop-stops-working-after-feb...

Has this update been applied to Office 2021 and 2019? Kindly advise. 

[Nino Bilic]

@adixro We did not start throttling / blocking based on Security Update versions just yet, that I am aware of.

[Bhalchandra_Atre-MSFT]

@edisontts @a2hawks The message recall issue is fixed with March updates for Outlook, do check the same article https://support.microsoft.com/en-us/office/recall-message-in-outlook-desktop-stops-working-after-feb...