Customer Guidance for Reported Zero-day Vulnerabilities in Microsoft Exchange Server

[The_Exchange_Team]

Update 11/8/2022: We have now released November 2022 Security Updates for Exchange Server. Please install those (or newer) updates to address vulnerabilities mentioned in this post. Mitigations are no longer recommended.

We have gotten several questions from our customers related to recent reports of new zero-day vulnerabilities impacting Exchange Server.

The MSRC team has released a blog post on this subject, with more information and mitigations that you can apply to your Exchange servers right away. We will keep updating these posts as the situation develops and we continue working on this:

MSRC - Customer Guidance for Reported Zero-day Vulnerabilities in Microsoft Exchange Server

Updates:

• 10/7 - Further improvement to the URL Rewrite rules on October 7. The EEMS service is receiving new rule automatically. EOMTv2 script has been updated (script auto-updates on Internet connected machines and the updated version is 22.10.07.2029).Updated manual URL Rewrite steps on the MSRC blog.
• 10/6 - Additional improvement to EOMTv2 rule (space was removed from a filter; the rule still worked but was changed for consistency sake). EOMTv2 version increased to 22.10.06.0840
• 10/5 - Further improvement to the URL Rewrite rules on October 5. The EEMS service is receiving new rule automatically. EOMTv2 script has been updated (script auto-updates on Internet connected machines and the updated version is 22.10.05.2304). Updated manual URL Rewrite steps on the MSRC blog.
• 10/4 - We have updated the URL Rewrite rules on October 4. The EEMS service is receiving new rule automatically. EOMTv2 script has been updated (script auto-updates on Internet connected machines and the updated version is 22.10.03.1829). We also updated the manual URL Rewrite steps on the MSRC blog.
• 10/2 - Addition to the Mitigations section: we strongly recommend Exchange Server customers disable remote PowerShell access for non-admin users in your organization. More details in the MSRC blog post.
• 9/30 - For customers who have the Exchange Emergency Mitigation Service (EEMS) enabled, Microsoft released the URL Rewrite mitigation for Exchange Server 2016 and Exchange Server 2019. The mitigation will be enabled automatically. Please see this blog post for more information on this service and how to check active mitigations.
• 9/30 - Microsoft Security Threat Intelligence team released a blog Analyzing attacks using the Exchange vulnerabilities CVE-2022-41040 and CVE-2022-41082
• 9/30 - We have now released a PowerShell script to help apply the mitigation to servers automatically.

The Exchange Server Team

[4ppl3c0r3]

Agreed, @Nino Bilic,

I truly do not believe that trying to make something perfect before publishing is the right thing to do as it never will be perfect.

Don't let perfect be the enemy of good.  

Initially, the RegEx pattern provided likely prevented some initial access attempts and only later was discovered to be too specific, so it changed.  

Initially, the "How to block" action was defaulted to 403 Forbidden which definitely prevented access, changing it to "Abort Request" is probably the better option, but BOTH equally prevented access.

Both pieces of initial guidance were still beneficial AND I would much prefer to learn about them days sooner than the corrected guidance.  This necessarily means there will be changes which promulgated the now-implemented change log.

Re: Remote PowerShell via 443.  Thank you for that clarification.  Port blocking is NOT an effective mitigation.  Remote PowerShell access should be disabled for all users unless they have a specific need.

Thank you.

[JMittes]

@The_Exchange_Team  @Nino Bilic   will Microsoft investigate also on this???

TIA. br Johannes 

There's a bypass to the fix to the bypass to the mitigation for ProxyNotShell / CVE-2022-41040. 🤦‍:male_sign:
twitter.com/honoki/status/…
Because characters can be encoded, looking for ".*autodiscover.json.*Powershell.*" in {REQUEST_URI} isn't sufficient.
Use {UrlDecode:{REQUEST_URI}} instead!

https://twitter.com/wdormann/status/1577667670048120833?s=46&t=_gz6YU4Z2nVgak3_AirJvg

[Nino Bilic]

@JMittes I can confirm that we are aware and are investigating.

[JMittes]

@Nino Bilic  Thx for quick answer:folded_hands:

[4ppl3c0r3]

UPDATE: Microsoft (in a later post) advises against blindly disabling all accounts since this will include all of your administrator accounts.  I was able to re-enable my own and other admin accounts with the previous instructions, all in the same Exchange Management Shell (PowerShell with Exchange Add-Ins) locally on the Exchange Server; but this is not guaranteed.  Worst case, by disabling Remote PowerShell for yourself, you may remove the ability to re-enable it for yourself.  Again, it worked for me, but it is NOT recommended.

Instead, I propose changing your command to exclude yourself from the list of users to disable by filtering with

-and samaccountname -ne "{your admin username}"

Therefore, the first EMS PowerShell command below and the sentence immediately prior have been revised:

@Nino Bilic, I've just completed disabling Remote PowerShell for ALL users, except for my (your) user, with this:

Get-User -ResultSize Unlimited -Filter 'RemotePowerShellEnabled -eq $true -and samaccountname -ne "{your admin username}"' | Set-User -RemotePowerShellEnabled $false

Then I re-enabled Remote PowerShell for individuals who actually need it with this:

Get-User -filter 'samaccountname -eq "{username of user to re-enable}"' | Set-User -RemotePowerShellEnabled $true

For a single user, I receive an access denied error when trying to disable Remote PowerShell, and that user is "MSOL_{randomstring}." 

This user was "created by Microsoft Azure Active Directory Connect with installation identifier {randomstring} running on computer {...} configured to synchronize to tenant {domain}. This account must have directory replication permissions in the local Active Directory and write permission on certain attributes to enable Hybrid Deployment," as indicated in the user's Description.

Do I need to be concerned about this user, necessary for Azure AD Connect?  OR may I disregard?  Please note, this user doesn't have a mailbox, and in fact, doesn't even have a logon name (but, does have a "pre-Windows 2000" logon name).  Also, it's likely this user was created with a strong password during creation/installation of Azure AD Connect.

Therefore, my assumption is that I don't need to be concerned about this user, but again, I'd like to hear it from Microsoft.

Thank you.

[osozu]

@Nino Bilic  Can you clarify if for some default accounts like DiscoverySearchMailbox, guest, DefaultAccount, krbtgt, etc can be remote powershell disabled without harm something or what about DAG clusters, is there some accounts which must be remote PowerShell enabled to make DAG cluster operable. In this article Control remote PowerShell access to Exchange servers | Microsoft Learn is nothing about this specific cases.

EDIT: and what about external url for powershell virtual directory should it be disabled (cleared) on some exchange server we have it enabled?

Thanks 

[BB9193]

@Nino Bilic we have also started the process of disabling remote powershell but need guidance from MS as to what the dependencies are.  I've not found any documentation to address the question of which accounts, other than admin, needs this access for proper functionality.

[ceantuco]

I am also in the process of removing remote shell access... basically all users and mailbox users are able to remotely connect to power shell.... 

Should I remove remote power shell access for all users until MS releases a patch? Including administrator? I really do not have a need since we have a GUI. 

I saw a few users with $ after their names.. I am assuming these are system users as well.

[tzh18]

When I disabled RemotePowerShell per the guidance, I found that it nulled the AuthenticationPolicy of the user. In fact every time you do "Set-User "Therese Lindqvist" -RemotePowerShellEnabled $false" command, it will null out the AuthenticationPolicy. I am not sure if this is specific to just our environment, but it did the same in our lab environment.

We use Hybrid Modern Authentication and the AuthenticationPolicy setting is how we disable legacy authentication for the users. Luckily, I had a dump of what the AuthenticationPolicy is for each user and was able to set it back fairly quickly.

[toddnelson-work]

Is there any reason related to the "Disable remote PowerShell access for non-admins" recommended mitigation that could cause primary SMTP addresses to change?
I'm seeing several accounts in our environment where the primary SMTP address was changed after disabling Exchange remote PowerShell.

[4ppl3c0r3]

NOTABLY: if you add any Users to your Active Directory or Exchange, the default seems to be to allow remote PowerShell access (likely why all users seem to have this access).

Therefore, if you add a user to Active Directory or Exchange, you should probably also immediately disable their remote PowerShell access even after you've performed both mitigations.  Currently, it seems, each new user is a new remote PowerShell attack vector.

Get-User -filter 'samaccountname -eq "{username of new user}"' | Set-User -RemotePowerShellEnabled $false

@The_Exchange_Team (for whatever reason, can't @mention Nino anymore) is there any way to change the RemotePowerShellEnabled default to False?

[ConanChiles]

hi,

posted a script to assist with identifying which accounts to disable RemotePowerShell in Exchange .

check and test for yourself, don't brick your Exchange.

https://github.com/microsoft/CSS-Exchange/issues/1261

[andydavid1]

FYI: If you shoot yourself in the foot and disable remote powershell for Exchange "admins", re-enable with the snap-in on the Exchange Server:
Add-PSSnapIn Microsoft.Exchange.Management.PowerShell.Snapin
set-user <Admin user> -RemotePowerShellEnabled $true

[Nino Bilic]

@4ppl3c0r3 I have to say, this is a bit risky. This will also disable your admin accounts; you should be careful to not lock out your admin accounts! Might be obvious to others but I just wanted to say. I would not worry about accounts that do not log in, like DiscoveryMailbox and stuff. Also - nope, I do not know of a way to create users with RemotePS set to disabled.

@ceantuco Please be careful blocking admin users; do not want you to be in a situation where you are locked out. EAC runs CMDlets, for example, I am not totally confident if there is a dependency on user's (Admin's) PS context for the UI also (but that would make sense because RBAC).

[JO376]

Good morning,

Thx @The_Exchange_Team @Nino Bilic for quick reaction and adapting Rewrite rule again - much appreciated to see that the new version with {UrlDecode:{REQUEST_URI}} was already "installed" when I came to the office this morning.

BUT: What about that Get-ChildItem... PowerShell to quickly check the logs for attacks? Will that still work with the known pattern? How to identify attacks that used kinda "encoded" patterns?

Thx for a short advice.

[nk_87]

Recent HealthChecker Script has a mistake: "$validConditionsInput = $rule.conditions.add.input -eq "{UrlDecode: {REQUEST_URI}}""

It should be "$validConditionsInput = $rule.conditions.add.input -eq "{UrlDecode:{REQUEST_URI}}"" (no space between Decode: and {REQUEST_) 

Exchange Health Checker version 22.10.05.2305

[Todd J Vanscoter]

I dont understand how all accounts have access to run remote powershell on Exchange.  Reading this:

Control remote PowerShell access to Exchange servers | Microsoft Learn

Under "What do you need to know before you begin?"  third bullet states:

"By default, all user accounts have access to remote PowerShell. However, to actually use remote PowerShell to connect to an Exchange server, the user needs to be a member of a management role group, or be directly assigned a management role that enables the user to run Exchange cmdlets"

I don't understand how users have access to run remote powershell commands on Exchange with the above information.  Can this be explained?

[Bhalchandra_Atre-MSFT]

@nk_87 the issue has been fixed and updated health checker released

[ceantuco]

@ninobilic thanks for your advise. I will only disable power shell for non admin users. 

@JO376 Thanks for pointing out the change... I probably would have not noticed it. 

[4ppl3c0r3]

@Nino_Bilic, re:

@4ppl3c0r3 I have to say, this is a bit risky. This will also disable your admin accounts; you should be careful to not lock out your admin accounts! Might be obvious to others but I just wanted to say. I would not worry about accounts that do not log in, like DiscoveryMailbox and stuff. Also - nope, I do not know of a way to create users with RemotePS set to disabled.

Thank you for the clarification.  I have updated my prior posting with a warning and an adjustment to the PowerShell filter.

[JO376]

@ConanChiles 

hi,

posted a script to assist with identifying which accounts to disable RemotePowerShell in Exchange .

check and test for yourself, don't brick your Exchange.

https://github.com/microsoft/CSS-Exchange/issues/1261

Thx, just had a look at it - as far as I can see I can run the script to just check what users would be affected. From what I see, line 306 is commented out and your script will not break anything unless I remove that comment #?

Do I miss something?

Advice for german users: check lines 175 ff as the script uses english group names ("Administrators" vs "Administratoren" and so on).

[Nino Bilic]

@Todd J Vanscoter Please see the CVE FAQ. This is not a problem of permissions, rather of access because of vulnerability.

[Nino Bilic]

@JO376 The guidance that we have published is in two blogs: Customer Guidance for Reported Zero-day Vulnerabilities in Microsoft Exchange Server
and Analyzing attacks using the Exchange vulnerabilities CVE-2022-41040 and CVE-2022-41082.

[Dennisdsmolinski]

When I run the EOMTv2.ps1 script I see "VERBOSE: Running latest EOMTv2 version 22.10.06.0840 downloaded" which is not the same as what is at the top of the blog for today's latest update: "EOMTv2 version increased to 22.10.06.2304".

What is the correct version number?

[JO376]

@ConanChiles 

I did some testing with your script, results look reasonable. 

Maybe you can change it to use well know SIDs instead of group names?

-519 => Enterprise Admins (Organisations-Admins in German)

-522 => Domain Admins (Domänen-Admins in German)

-544 => Administrators (Administratoren in German)

Would make your script "multi language" with just a small adjustment to it. ;)

[Nino Bilic]

@Dennisdsmolinski Sigh, that was totally my fault; you are right, it is 22.10.06.0840.

[Dennisdsmolinski]

Thank you Nino. 

[JShomper]

@Nino Bilic re: PowerShell access - could you also restrict virtual directory access to /PowerShell to 127.0.0.1 and/or Exchange Server IPs to reduce the attack surface? In theory, the average user shouldn't be able to RDP/logon locally to those servers.

[Nino Bilic]

@JShomper It's an idea but we have not explored this. Issues could be - of course - that your Admins might also not be able to reach the destination and management tools would break (because EAC is based on PowerShell calls).

[ceantuco]

I removed and re-added the rule with the latest recommendations from MS.

Since port 443 is used for the attack, should I remove the Windows Firewall rule I created blocking ports 5985-5986?

Please advise

Thank you!

[David_Richard]

@ceantucoYes, remove it. But did you disable remote PowerShell access for nonadmin users/service accounts?

An excellent way is to create a security group and add the admin accounts/service accounts to that group and run the DisableRemotePS.ps1, which you can find here:

https://www.alitajran.com/0-day-vulnerability-microsoft-exchange/

[Nino Bilic]

@ceantuco Well, truth is - they will not do any damage; I am not immediately aware of anything (Exchange related) that would use those ports, so you could leave them blocked. The main thing is to realize that this was not an effective mitigation for CVE-2022-41082.

[ceantuco]

@David_Richard I am working on removing PS access for non admin accounts now. Thanks for the suggestion. 

@ninobilic I added the firewall rule on 09/30 and I have not experienced any Exchange related issues. Yes, I understand blocking those ports do not prevent exploitation. Thank you.

[Tonibert]

Hello , i have linked users. Do i have to disable RemotePowerShell on the original users of the mailboxes or the linked users.

Thanks for response 

[kekslicht0815]

Could we with Exchange 2019 also use ClientAccessRules to block access to RemotePowerShell except for our internal subnets to mitigate CVE-2022-41082?

E.g.

New-ClientAccessRule -Name "Block PowerShell" -Action DenyAccess -AnyOfProtocols RemotePowerShell -ExceptAnyOfClientIPAddressesOrRanges 192.168.10.1/24

[JO376]

@The_Exchange_Team @Nino Bilic 

Ok, so disabling Remote PowerShell is bit more tricky than expected. But it's just something that can and has to be done.

QUESTION:

How will MS handle this in future?
I mean, every new user will have Remote PowerShell enabled per default - so disabling Remote PowerShell is NOT! a one-time-job but has to be monitored/done over and over again. How to handle that?

[andydavid1]

@JO376 The recommendation to disable remote PS is most likely a temporary one until a permanent Exchange fix is made available. 

[ceantuco]

@The_Exchange_Team @nino bilic

I have 4 regular users that AD does not allow me to disable remote PS. I get the error message below. I am running the command as 'administrator':

Active Directory operation failed on DOMAINCONTROLLER. This error is not retriable. Additional information:
Insufficient access rights to perform the operation.
Active directory response: 00002098: SecErr: DSID-031514A0, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0
+ CategoryInfo : NotSpecified: (:) [Set-User], ADOperationException
+ FullyQualifiedErrorId : [Server=EXCHANGE2019,RequestId=2bb81a25-528d-4462-a452-fc274747e3d8,TimeStamp=10/7/2022 2:53
:39 PM] [FailureCategory=Cmdlet-ADOperationException] 61731BA3,Microsoft.Exchange.Management.RecipientTasks.SetUser
+ PSComputerName : EXCHANGE2019

How can I resolve this issue?

[Bhalchandra_Atre-MSFT]

@kekslicht0815 

That's a good approach but won't protect if internal machine is compromised. So, it's better to disable the RPS on the user.

@Tonibert You'll need to just run the Set-User command from Exchange Management Shell, it will take care of disabling

[LotharMueller]

@ceantuco 

If these are regular users.

Could it be an inheritance problem of the user object?

This could be checked on the security tab in ADUC.

In many times I saw an disabled inheritance when accounts are old/or copied.

[ceantuco]

@LotharMueller 

Thank you! Yes, once I enabled inheritance for those users, I was able to remove remote PS access. 

[Dennisdsmolinski]

Hi @Nino Bilic can either you or someone from your team confirm that no IISRESET nor server reboot is necessary after the mitigation script is applied to the server?  I need that as a Microsoft statement for our security team.  Thank you in advance.

[Nino Bilic]

@Dennisdsmolinski No restart or IIS reset is needed, no.

[Dennisdsmolinski]

Thank you very much Nino.

[karil222]

@The_Exchange_Team , @Nino Bilic  You have updated mitigation rule instruction, the step 6 has string "(?=.*autodiscover\.json)(?=.*powershell)", but pictures and EEMS says it differently "(?=.*autodiscover)(?=.*powershell)".

Which one is correct this time??

[Duncan1528]

Did you something via EEMS which makes Windows Defender go mad?

Since 04:14 GMT+1 tonight the Defender Service (Server 2K16) crashes nearly every minute and the restarts "Exception Processing Message 0xc000007b - Unexpected parameters" :(

WTH is going on?

[Duncan1528]

"Exception Processing Message 0xc000007b - Unexpected parameters" seems to be caused by Defender Update KB2267602 (Version 1.375.1727.0).

After installing KB2267602 (Version 1.375.1748.0) the problem seems to be resolved.

Did you ever test your updates before release?

[Sreejith]

Hello @The_Exchange_Team @Nino Bilic 

I tried to run the below command but still I can see RemotePowerShellEnabled status as $True on user accounts. What could be the reason any clue....?

Get-User -ResultSize Unlimited -Filter 'RemotePowerShellEnabled -eq $true -and samaccountname -ne "{your admin username}"' | Set-User -RemotePowerShellEnabled $false

Regards,

[Nino Bilic]

@karil222 Scripts are correct; manual steps had an issue with screenshot (has been corrected).

[Johan Parlevliet]

With the new rewrite rule: Version 22.10.07.2029: $pattern = '(?=.*autodiscover)(?=.*powershell)'
What would be the new command to find occurences in the logs: 

Get-ChildItem -Recurse -Path <Path_IIS_Logs> -Filter "*.log" | Select-String -Pattern 'Select-String -Pattern '(?=.*autodiscover)(?=.*powershell)'