MSExchangeMailboxReplication Service will not start

[chrisryland]

After attempting to install Security Update For Exchange Server 2016 CU15 (KB4540123) on our Exchange 2016 Server CU 15 Version 15.1 (Build 1913.5) which failed to install it disabled all the Exchange Services.  One of the Servers was able to get all the services back up running but the other unable to get the MSExchangeMailboxReplication Service to run. Get the message "the microsoft exchange mailbox replication service on local computer started and then stopped" Have fun through the https://docs.microsoft.com/en-us/exchange/management/health/troubleshooting-mrs-health-set

and searched the web for solution.

When run the Get Server Health is showing some of the MRS are unhealthy 

Get-ServerHealth <server> | ?{$_.HealthSetName -eq "MailboxMigration"}

CurrentHealthSetState : NotApplicable
Name : MRSDiagnosticsMonitor
TargetResource :
HealthSetName : MailboxMigration
HealthGroupName : ServiceComponents
AlertValue : Unhealthy
FirstAlertObservedTime : 2/10/2020 10:33:26 PM
Description :
IsHaImpacting : False
RecurranceInterval : 750
DefinitionCreatedTime : 13/10/2020 9:48:23 PM
HealthSetDescription :
ServerComponentName : None
LastTransitionTime : 2/10/2020 10:33:26 PM
LastExecutionTime : 22/10/2020 3:48:35 PM
LastExecutionResult : Succeeded
ResultId : 96325198
WorkItemId : 438493821
IsStale : False
Error :
Exception :
IsNotified : False
LastFailedProbeId : 2001237620
LastFailedProbeResultId : 25558101
ServicePriority : 1
Identity : MailboxMigration\MRSDiagnosticsMonitor\
IsValid : True
ObjectState : New

CurrentHealthSetState : NotApplicable
Name : MRSStartedMonitor
TargetResource :
HealthSetName : MailboxMigration
HealthGroupName : ServiceComponents
AlertValue : Unhealthy
FirstAlertObservedTime : 2/10/2020 10:37:07 PM
Description :
IsHaImpacting : False
RecurranceInterval : 750
DefinitionCreatedTime : 13/10/2020 9:48:23 PM
HealthSetDescription :
ServerComponentName : None
LastTransitionTime : 2/10/2020 11:02:07 PM
LastExecutionTime : 22/10/2020 3:49:11 PM
LastExecutionResult : Succeeded
ResultId : 96325392
WorkItemId : 1017824070
IsStale : False
Error :
Exception :
IsNotified : False
LastFailedProbeId : 484484869
LastFailedProbeResultId : 25558061
ServicePriority : 1
Identity : MailboxMigration\MRSStartedMonitor\
IsValid : True
ObjectState : New

this is stopping any mailbox moves for this server. 

Has anyone come across this or know how to resolve this?

[#2]

@chrisryland 

Hi

Can you please post some logs from the windows event log related to the service?

[#3]

@farismalaeb thanks for your reply

The only windows event log I can find around the service are under Applications and Services Logs > Microsoft > Exchange > Managed Availability. 

* Monitoring

* RecoveryActionLogs

* RecoveryActionResults

attached is the word doc with them in it hopefully it helps 

[#4]

@chrisryland 

To get more logs you need to run the following PowerShell command

Get-EventlogLevel -Identity "MSExchange Mailbox Replication\*" | Set-EventlogLevel -Level Expert

 Then restart the service and post the logs

I think they will be in the Application event log

[#5]

@farismalaeb 

System

- Provider       [ Name] MSExchange Common

- EventID 7003       [ Qualifiers] 16384

Level 4

Task 1

Keywords 0x80000000000000

- TimeCreated       [ SystemTime] 2020-10-28T01:09:33.968403000Z

EventRecordID 3353274

Channel Application

Computer

Security

-

EventData

w3wp

MSExchangeServicesAppPool

06:00:00

I32:ADR.C[INSTR]=49916;F:ADR.AL[INSTR]=1.458034;I32:ADS.C[INSTR]=42126;F:ADS.AL[INSTR]=1.927801;I32:ADS.C[SUPPR]=48;F:ADS.AL[SUPPR]=2.376469;I32:MB.C[SUPPR]=54532;F:MB.AL[SUPPR]=4.821628;I32:ATE.C[UNINSTR]=10748;F:ATE.AL[UNINSTR]=0.02633048;I32:VCGS.C[SUPPR]=4416;Dbl:VCGS.T[SUPPR]=106;Dbl:EXR.T[SUPPR]=189405;Dbl:MAPI.T[INSTR]=590128;I32:ATE.C[INSTR]=101135;F:ATE.AL[INSTR]=0.2423691;I32:RPCDB.C[INSTR]=83070;F:RPCDB.AL[INSTR]=1.241712;I32:RPCSVR.C[INSTR]=83070;F:RPCSVR.AL[INSTR]=1.241712;Dbl:EXR.T[INSTR]=422432;Dbl:CCpu.T[INSTR]=321484.375;Dbl:MBLB.T[INSTR]=18906041;I32:RPC.C[SUPPR]=54532;Dbl:RPC.T[SUPPR]=262933;Dbl:BudgUse.T[INSTR]=981696.362194508;I32:ROP.C[SUPPR]=1011294694;I32:RPC.C[INSTR]=227168;I32:MAPI.C[SUPPR]=947028;Dbl:RPC.T[INSTR]=584407;I32:ROP.C[INSTR]=45293799;I32:MAPI.C[INSTR]=2197334;Dbl:MAPI.T[SUPPR]=267316;I32:ATE.C[SUPPR]=104;F:ATE.AL[SUPPR]=0.1923077;I32:ADR.C[UNINSTR]=3185;F:ADR.AL[UNINSTR]=1.076108;I32:ADS.C[UNINSTR]=8205;F:ADS.AL[UNINSTR]=1.041992;Dbl:VCGS.T[MISSED]=0;I32:VCGS.C[MISSED]=72;I32:VCGS.C[INSTR]=24688;Dbl:VCGS.T[INSTR]=112;I32:MB.C[INSTR]=227168;F:MB.AL[INSTR]=2.572576;Dbl:ST.T[INSTR]=375345;Dbl:STCPU.T[INSTR]=222442;I32:STPR.C[INSTR]=4044;I32:RPCSVR.C[SUPPR]=16519;F:RPCSVR.AL[SUPPR]=1.23476;I32:RPCDB.C[SUPPR]=16519;F:RPCDB.AL[SUPPR]=1.23476;I32:STPPR.C[INSTR]=14970;Dbl:ST.T[SUPPR]=93004;Dbl:STCPU.T[SUPPR]=54981;Dbl:ADB.T[UNINSTR]=17;Dbl:MBLB.T[SUPPR]=67974;I32:STPPR.C[SUPPR]=907;I32:STPR.C[SUPPR]=673;Dbl:EXR.T[UNINSTR]=395;Dbl:MAPI.T[UNINSTR]=739;Dbl:RPC.T[UNINSTR]=730;I32:RPC.C[UNINSTR]=425;I32:MAPI.C[UNINSTR]=814;I32:ROP.C[UNINSTR]=-1856823415;Dbl:MBLB.T[UNINSTR]=134588;Dbl:ST.T[UNINSTR]=357;I32:MB.C[UNINSTR]=425;F:MB.AL[UNINSTR]=1.717647;I32:STPR.C[UNINSTR]=18;Dbl:STCPU.T[UNINSTR]=210;Dbl:VCGS.T[UNINSTR]=6;I32:VCGS.C[UNINSTR]=70;I32:RPCSVR.C[UNINSTR]=66;F:RPCSVR.AL[UNINSTR]=1.015152;I32:RPCDB.C[UNINSTR]=66;F:RPCDB.AL[UNINSTR]=1.015152;Dbl:EXRP.T[SUPPR]=564;I32:ADR.C[SUPPR]=10;F:ADR.AL[SUPPR]=2.72548;I32:ADW.C[SUPPR]=10;F:ADW.AL[SUPPR]=1.085844;Dbl:ADB.T[INSTR]=6;Dbl:ADB.T[SUPPR]=6;I32:BC.C[INSTR]=75;Dbl:BC.T[INSTR]=400.798898696899;Dbl:MBMC.T[INSTR]=16;Dbl:EXRS.T[SUPPR]=5;Dbl:EXRCP.T[INSTR]=4;Dbl:ADB.T[MISSED]=4

[#6]

@farismalaeb 

Hi this was to only log that could be related when I tried to start the service again. no other logs are showing for MSExchangeMailboxReplication and I have run the ps command

[#7]

@chrisryland 

Please check the following log for more information

C:\Program Files\Microsoft\Exchange Server\V15\Logging\MailboxReplicationService

[#8]

@farismalaeb  thanks for your help.

So the logs only go until the 2nd of Oct to the time when we tried to install the update that failed.

I've tried a few time to start the service to see if it records any logs but its not for some reason.

Attached are the logs of the time of the update. 

[#9]

@farismalaeb Issue has been resolved with the update to CU17