Exchange 2013 replication errors 1121

%3CLINGO-SUB%20id%3D%22lingo-sub-3271004%22%20slang%3D%22en-US%22%3EExchange%202013%20replication%20errors%201121%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3271004%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20all%2C%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ERunning%202%20Exchange%202013%20servers%20on%202012%20R2%20in%20a%20DAG.%26nbsp%3B%20We%20have%202%20mailbox%20databases.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EEvery%20few%20minutes%20am%20getting%2031%201121%20errors%20as%20below%2C%20with%20differing%20Request%20GUID's.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSTRONG%3EThe%20Microsoft%20Exchange%20Mailbox%20Replication%20service%20was%20unable%20to%20process%20a%20request%20due%20to%20an%20unexpected%20error.%3C%2FSTRONG%3E%3CBR%20%2F%3E%3CSTRONG%3ERequest%20GUID%3A%20'cbcde8dc-cee6-4016-849a-210d35c3a7a8'%20%3C%2FSTRONG%3E%3CBR%20%2F%3E%3CSTRONG%3EDatabase%20GUID%3A%20'25af4ecf-7834-4c3c-b748-c23799877a9c'%3C%2FSTRONG%3E%3CBR%20%2F%3E%3CSTRONG%3EError%3A%20Database%20'8ddbf292-7634-4189-b342-0a3bb62c78dc'%20doesn't%20exist..%3C%2FSTRONG%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20database%20GUID%20beginning%2025a...%20is%20one%20of%20our%20mailbox%20databases.%26nbsp%3B%208dd...%20does%20not%20exist%20(it's%20not%20the%20GUID%20of%20the%20other%20database).%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThings%20I've%20tried%3A%3C%2FP%3E%3CP%3E%3CSPAN%3ERemove-MoveRequest%3C%2FSPAN%3E%20command%20using%20the%20request%20GUID.%26nbsp%3B%20No%20such%20requests%20exist.%26nbsp%3B%20get-mailboxrequest%20returns%20nothing.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EADSI%20edit.%20I've%20read%20that%20requests%20can%20appear%20in%20%22Configuration%2F...%2FMailboxExportRequests%22%20but%20there%20is%20nothing%20there.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAnyone%20know%20where%20these%20requests%20might%20be%20sitting%20to%20so%20I%20can%20remove%20them%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ethanks%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EJ%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-3271004%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3E2013%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EExchange%20Server%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3278320%22%20slang%3D%22en-US%22%3ERe%3A%20Exchange%202013%20replication%20errors%201121%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3278320%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F3842%22%20target%3D%22_blank%22%3E%40Thomas%20Stensitzki%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%20Thomas.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI'll%20wait%20for%20a%20quiet(ish)%20time%20and%20start%20by%20removing%20one%20of%20the%20orphans%2C%20perhaps%20one%20that%20I%20can't%20see%20mentioned%20anywhere%2C%20then%20sit%20back%20and%20watch%20for%20any%20issues.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EIf%20that%20goes%20ok%20I'll%20slowly%20remove%20the%20others.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Echeers%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3276585%22%20slang%3D%22en-US%22%3ERe%3A%20Exchange%202013%20replication%20errors%201121%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3276585%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F539470%22%20target%3D%22_blank%22%3E%40whatwaht%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThis%20Cluster%20registry%20hive%20is%20often%20referred%20to%20as%20the%20Cluster%20Configuration%20Store.%20This%20information%20is%20used%20by%20the%20Windows%20Cluster%20Failover%20components.%3C%2FP%3E%0A%3CP%3ENormally%2C%20Exchange%20removes%20a%20Private-GUID%20entry%20after%20the%20removal%20of%20a%20database.%20The%20cluster%20members%20communicate%20using%20DCOM.%20Sometimes%20this%20communication%20is%20blocked%20by%20GPO%20settings%20or%20misconfigured%20firewalls.%26nbsp%3B%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3EThe%20Exchange%20Configuration%20node%20you%20mentioned%20(Config%2FServices%2FMS-Ex%2FUs%2FAdmin%20Groups%2FEx%20Admin%20Group%2FDatabases)%20contains%20the%20information%20used%20by%20Exchange%20itself.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EYou%20can%20remove%20the%20orphaned%20entries%20in%20the%20cluster%20hive%20manually%20at%20your%20own%20risk.%20You%20should%20double-check%20the%20GUIDs%20with%20your%20existing%20databases.%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3273190%22%20slang%3D%22en-US%22%3ERe%3A%20Exchange%202013%20replication%20errors%201121%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3273190%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F3842%22%20target%3D%22_blank%22%3E%40Thomas%20Stensitzki%3C%2FA%3E%3C%2FP%3E%3CP%3ESo%20there%20are%2011%20Private-GUID%20entries%20under%20the%20registry%20key%20you%20specified%2C%20including%20the%20guid%20mentioned%20in%20the%20errors.%26nbsp%3B%20Same%20on%20both%20servers.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EUnder%20%3CSTRONG%3EConfig%2FServices%2FMS-Ex%2FUs%2FAdmin%20Groups%2FEx%20Admin%20Group%2FDatabases%3C%2FSTRONG%3E%20there%20are%20only%20our%20existing%202%20databases%20listed.%20Unless%20there%20was%20somewhere%20else%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ethanks%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3272336%22%20slang%3D%22en-US%22%3ERe%3A%20Exchange%202013%20replication%20errors%201121%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3272336%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F539470%22%20target%3D%22_blank%22%3E%40whatwaht%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EPlease%20check%20the%20Cluster%20configuration%20node%20of%20each%20server%20using%20RegEdit%20if%20there%20is%20an%20orphaned%20database%20entry.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3EHKLM%5CCluster%5CMsExchangeIs%5C%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIdeally%2C%20there%20are%20only%20two%20entries%20matching%20your%20two%20existing%20databases%20in%20the%20format%20%3CEM%3E%5CPrivate-GUID%3C%2FEM%3E%3C%2FP%3E%0A%3CP%3EYou%20want%20to%20%3CEM%3Ejust%20check%3C%2FEM%3E%20if%20there%20is%20such%20an%20entry%20from%20an%20older%20database%20that%20has%20not%20been%20cleaned%20up%20properly.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EDid%20you%20check%20the%20Databases%20node%20on%20the%20AD%20configuration%20partition%3F%20Are%20there%20any%20orphaned%20database%20entries%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3272218%22%20slang%3D%22en-US%22%3ERe%3A%20Exchange%202013%20replication%20errors%201121%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3272218%22%20slang%3D%22en-US%22%3E%3CP%3EThanks%20for%20the%20reply%20Thomas.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWe%20are%20on%20CU23.%26nbsp%3B%20We%20haven't%20conducted%20any%20mailbox%20moves%20recently%20so%20am%20not%20sure%20what%20these%20requests%20would%20have%20been%20for.%26nbsp%3B%20I%20just%20noticed%20them%20but%20unforunately%20our%20logs%20only%20go%20back%20a%20week%20so%20am%20unsure%20when%20they%20started%20appearing.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3271191%22%20slang%3D%22en-US%22%3ERe%3A%20Exchange%202013%20replication%20errors%201121%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3271191%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F539470%22%20target%3D%22_blank%22%3E%40whatwaht%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWhat%20is%20the%20current%20CU%20state%20in%20your%20Exchange%20Organization%3F%3C%2FP%3E%0A%3CP%3EIs%20the%20issue%20related%20to%20cross-server%20moves%2C%20or%20does%20it%20occur%20even%20on%20the%20same%20server%20when%20moving%20a%20mailbox%20between%20databases%20active%20on%20the%20same%20server%3F%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E-Thomas%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Occasional Contributor

Hi all,

 

Running 2 Exchange 2013 servers on 2012 R2 in a DAG.  We have 2 mailbox databases.

 

Every few minutes am getting 31 1121 errors as below, with differing Request GUID's.

 

The Microsoft Exchange Mailbox Replication service was unable to process a request due to an unexpected error.
Request GUID: 'cbcde8dc-cee6-4016-849a-210d35c3a7a8'
Database GUID: '25af4ecf-7834-4c3c-b748-c23799877a9c'
Error: Database '8ddbf292-7634-4189-b342-0a3bb62c78dc' doesn't exist..

 

The database GUID beginning 25a... is one of our mailbox databases.  8dd... does not exist (it's not the GUID of the other database).

 

Things I've tried:

Remove-MoveRequest command using the request GUID.  No such requests exist.  get-mailboxrequest returns nothing.

 

ADSI edit. I've read that requests can appear in "Configuration/.../MailboxExportRequests" but there is nothing there.

 

Anyone know where these requests might be sitting to so I can remove them?

 

thanks

 

J

6 Replies

@whatwaht 

What is the current CU state in your Exchange Organization?

Is the issue related to cross-server moves, or does it occur even on the same server when moving a mailbox between databases active on the same server?

 

-Thomas 

Thanks for the reply Thomas.

 

We are on CU23.  We haven't conducted any mailbox moves recently so am not sure what these requests would have been for.  I just noticed them but unforunately our logs only go back a week so am unsure when they started appearing.

@whatwaht 

Please check the Cluster configuration node of each server using RegEdit if there is an orphaned database entry.

 

HKLM\Cluster\MsExchangeIs\

 

Ideally, there are only two entries matching your two existing databases in the format \Private-GUID

You want to just check if there is such an entry from an older database that has not been cleaned up properly.

 

Did you check the Databases node on the AD configuration partition? Are there any orphaned database entries?

@Thomas Stensitzki

So there are 11 Private-GUID entries under the registry key you specified, including the guid mentioned in the errors.  Same on both servers.

 

Under Config/Services/MS-Ex/Us/Admin Groups/Ex Admin Group/Databases there are only our existing 2 databases listed. Unless there was somewhere else?

 

thanks

@whatwaht 

This Cluster registry hive is often referred to as the Cluster Configuration Store. This information is used by the Windows Cluster Failover components.

Normally, Exchange removes a Private-GUID entry after the removal of a database. The cluster members communicate using DCOM. Sometimes this communication is blocked by GPO settings or misconfigured firewalls. 

The Exchange Configuration node you mentioned (Config/Services/MS-Ex/Us/Admin Groups/Ex Admin Group/Databases) contains the information used by Exchange itself.

 

You can remove the orphaned entries in the cluster hive manually at your own risk. You should double-check the GUIDs with your existing databases. 

@Thomas Stensitzki 

Thanks Thomas.

 

I'll wait for a quiet(ish) time and start by removing one of the orphans, perhaps one that I can't see mentioned anywhere, then sit back and watch for any issues.

 

If that goes ok I'll slowly remove the others.

 

cheers