Forum Discussion

SABBIRRUBAYAT's avatar
SABBIRRUBAYAT
Copper Contributor
Jan 01, 2022
Solved

Exchange Server Mail Stuck on Queue due to Microsoft Bug 01/01/2022

There is a live Bug going on in On premise Exchange. To resolve the issue Please Disable your Antimalware filter just for now so you can start your communication again. Please note that this is not final solution. this is just workaround for now

Please follow the below step:

 

  1. Go to This location “C:\Program Files\Microsoft\Exchange Server\V15\Scripts\” and run Disable-Antimalwarescanning.ps1     Follow Microsoft Article if you need help to run the command https://docs.microsoft.com/en-us/exchange/disable-or-bypass-anti-malware-scanning-exchange-2013-help
  2. Restart Transport Service
  3. New-SettingOverride -Name "DisablingAMSIScan" -Component Cafe -Section HttpRequestFiltering -Parameters ("Enabled=False") -Reason "Testing"

 

Following third step is not necessary. This step is necessary only if you have third party malware agent. 

  • LeeMEI's avatar
    LeeMEI
    Jan 01, 2022

    amaltsev Are you sure this is a cyberattack and not a bug?  According to most reports I've read, this is due to a failure converting to a new date and occurred at midnight UTC.

18 Replies

  • stopnik's avatar
    stopnik
    Copper Contributor

    SABBIRRUBAYAT - just a note that it's not a cyber attack, it's an int32 conversion issue with the date code of the 2022 signatures that Microsoft still hasn't fixed.

     

    I had also originally disabled malware scanning using the disable scripts last night to get things working, but the problem with this is that the malware definitions/engines are never updated for the FIP-FS service when you do that...it's better to set all your servers to bypass scanning instead.  I had tried this last night before the internet blew up (this is for 2013 but same thing for 2016/2019 - https://docs.microsoft.com/en-us/exchange/disable-or-bypass-anti-malware-scanning-exchange-2013-help).  The problem with that article is it doesn't say to restart the transport service after running the bypass command - but you have to do that for it to take effect. 

     

    If you do it this way, mailflow will work and your definitions will continue to be updated so that hopefully when Microsoft fixes this your servers will be updated so that when you remove the bypass things continue to work (rather than re-enabling scanning, then waiting for the engines to update which takes a long time/breaks your mail flow again).

    • SABBIRRUBAYAT's avatar
      SABBIRRUBAYAT
      Copper Contributor
      Thank you ..................... and apologize to identify this as cyber attack . thanks anyway
    • SABBIRRUBAYAT's avatar
      SABBIRRUBAYAT
      Copper Contributor
      Definitely you are right. I found the solution at the first hour and I started getting call from multiple companies at the same time. I was a little bit freaked out and thought this might be a cyber attack. I apologize for my mistake
    • azuser's avatar
      azuser
      Copper Contributor

      I only ran the two steps below, and the email started to flow after.
      1. Disable-Antimalwarescanning.ps1
      2. restart the transport service
      I did not run Set-MalwareFilteringServer <ServerIdentity> - BypassFiltering $true instead of the above two.

      Will the next Exchange patch release installation re·vert
      the antimalware agent back "on," as on the default installation, and again we have to disable the agent?

  • GOLINE_SA's avatar
    GOLINE_SA
    Copper Contributor
    It's a bug!
    According to @ceno666 the issue also seems to occur with the 220101002 update version as well. Could be related to, what I’m dubbing, the “Y2K22” bug. Refer to the comment from JulianSiebert about the “signed long” here: https://techcommunity.microsoft.com/t5/exchange-team-blog/december-2021-exchange-server-cumulative-updates-postponed/bc-p/3049189/highlight/true#M31885 The “long” type allows for values up to 2,147,483,647. It appears that Microsoft uses the first two numbers of the update version to denote the year of the update. So when the year was 2021, the first two numbers was “21”, and everything was fine. Now that it’s 2022 (GMT), the update version, converted to a “long” would be 2,201,01,001 - - which is above the maximum value of the “long” data type. Pernille-Eskebo: If you change it to an ‘unsigned long’, then the max value is 4,294,967,295 and we’ll be able to sleep easy until the year 2043!

    Paolo Caparrelli
    GOLINE SA
    IT Services
  • SteffjeNL's avatar
    SteffjeNL
    Copper Contributor

    SABBIRRUBAYAT Where do you see that there is an 'Cyber Attack' ?

     

    It seems there is a bug in the moduoe 'FIP-FS' if you look in your Eventlog

     

    The FIP-FS "Microsoft" Scan Engine failed to load. PID: 20132, Error Code: 0x80004005. Error Description: Can't convert "2201010004" to long.

     

    Looks like there are using long in there application
    with an max number of
    214748 3647 and they try (date looks like?) to parse and higher number 2201010004

    A solution without a reboot from servies is:

     

    Set-MalwareFilteringServer -Identity <<SERVERNAMEHERE>> -BypassFiltering $true

     

    Please note!! This is a temporary solution! get a good virus, mallware and spam scanner on your Exchange server from a third party!

     

     

    • LeeMEI's avatar
      LeeMEI
      Copper Contributor

      amaltsev Are you sure this is a cyberattack and not a bug?  According to most reports I've read, this is due to a failure converting to a new date and occurred at midnight UTC.

      • amaltsev's avatar
        amaltsev
        Icon for Microsoft rankMicrosoft

        LeeMEI please note that my answer relates only to the action steps proposed, nothing to do with the origin of the problem

Resources