Moving mailboxes, Exchange 2010 way
Published Jul 19 2010 05:56 PM 96K Views

Introduction

Exchange Server 2007 used the Move-Mailbox cmdlet to move mailboxes between mailbox stores. Move-Mailbox makes a RPC connection to both the source and target mailbox databases and then starts the move process. Exchange Server 2010 uses the Mailbox Replication Service (MRS), a service that runs on all Exchange 2010 Client Access Servers, for mailbox move operations. MRS handles all mailbox moves including offline and online moves.

 

Exchange 2007 vs. 2010

This is a high level comparison of Exchange Server 2007 vs. Exchange Server 2010. There is a radical change in mailbox moves from Exchange 2007 to Exchange 2010 environment. In Exchange 2010, we implemented online mailbox moves.

When you move a mailbox in previous versions of Exchange Server, the source mailbox gets locked and then the content is copied to the new mailbox on the target mailbox database. After the content is copied, the new mailbox is unlocked and the old one is deleted. This results in downtime for the user for the duration of the mailbox move operation. As long as you had smaller mailboxes, the downtime is not a big deal since this happens fairly quickly. With larger mailboxes (check out the Large Mailbox Vision whitepaper as well as Astrid's blog on the Top 10 Exchange Storage Myths), the downtime can be unacceptable.

In Exchange 2010, we implemented online mailbox moves and we also implemented changes to the Store in Exchange 2007 SP2 so that when you move and upgrade from Exchange 2007 to 2010 you will benefit from online mailbox moves.

The following terms are used in Exchange Server 2010 move operation:

  • Online mailbox move - Move mailbox operation wherein users are able to access their mailbox almost for the entire time of the operation except for the last part. Exchange Server 2010 uses online mailbox move.
  • Offline mailbox move is a move operation wherein users cannot access their mailboxes during the move.
  • Local Move is a move operation wherein the source and target mailboxes exist in the same forest and organization.
  • Remote Move is a move operation wherein the source and target are in different forests and organization.
  • Push is a move operation where the target is either an Exchange Server 2007 or Exchange Server 2003 and the source is an Exchange 2010 server in a local move. A push can also occur between Exchange 2010 where the source and the target are in different forests.
  • Pull is a move operation where the target is an Exchange 2010 server.

Which Exchange Versions supports what?

The following table outlines the type of mailbox moves supported by different Exchange Server versions.

Mailbox Moves and Personal Archives

Exchange Server 2010 introduced Personal Archives. An archive mailbox appears as an additional mailbox in Outlook 2010 or OWA. In Exchange 2010 RTM, an archive mailbox is moved along with the mailbox if one exists. Since archive mailboxes exist only in Exchange Server 2010, mailbox moves to legacy Exchange servers will fail if the mailbox being moved has an archive mailbox.

Exchange 2010 CMDlets for move mailbox

Mailbox moves can be performed using the EMC or the Shell (EMS. You can use the following CMDlets from the Shell to manage mailbox move requests. It's worth noting that suspending and resuming a move request can only be done through the Shell. Just like any other operation in Exchange 2010, you need to have certain permissions to perform the commands. The following table shows the RBAC permissions required for each cmdlet.

Note: RBAC permission can be granted to administrators either by assignment of a management role or membership in a built-in role group.

The Microsoft Exchange Mailbox Replication Service

The Microsoft Exchange Mailbox Replication Service (MRS) is a Windows service and is dependent only on the Microsoft Exchange Active Directory Topology service and Net.TCP Port Sharing service. There is a pre-requisite in Exchange Server 2010 setup that checks if the Net.TCP Post Sharing service is set to automatic. If it's not, setup fails on CAS Servers. MRS is built on the Windows Communication Foundation (WCF), a part of .Net Framework 3.0 stack. MRS uses a configuration file MSExhangeMailBoxReplication.Exe.Config on every Client Access Server for its configuration information. By default it's located in C:\Program Files\Microsoft Exchange Server\V14\Bin folder. The process associated with the service is MSExchangeMailBoxReplication.exe

How the Mailbox Replication Service works

When a move mailbox request is issued, the command creates a message in the System mailbox of the target database. From there MRS picks up the request and makes a MAPI.net connection to both the source and target databases. After a successful MAPI connection is made, MRS creates a mailbox in the target database and starts incremental synchronization of data. When it reaches to a point where it is about to complete the move, it locks the mailbox, updates Active Directory attributes, unlocks the mailbox and deletes the source mailbox.

Here is the flow of the move operation in detail for online move.

Local Online Mailbox Move:

  1. Administrator creates the move request using the New-MoveRequest command.
  2. The New-MoveRequest makes the following checks for mailbox being moved.
    • Gets the target and source mailbox server version
    • Checks the database versions to verify they are supported Exchange versions.
    • Determines the push or pull operation by the Exchange version information
    • Checks for an archive mailbox, if one is found, then adds it to the move request. Also checks to make sure you are not moving a mailbox with an archive to legacy Exchange system.
    • Checks the rule limit for legacy exchange servers
    • Checks mailbox quotas
  3. The New-MoveRequest command creates a request message in the target database's System Mailbox as special message.
  4. The following attributes are added to a user account for the mailbox in Active Directory. These attributes are used to store information about moving the mailbox and some are updated throughout the move.
    • msExchMailboxMoveBatchName
    • msExchMailboxMoveFlags
    • msExchMailboxMoveRemoteHostName
    • msExchMailboxMoveSourceMDBLink
    • msExchMailboxMoveStatus
    • msExchMailboxMoveTargetMDBLink

    These attributes will not be removed after the move request is completed unless a Remove-MoveRequest is run. If these attributes are not removed then another New-MoveRequest cannot be issued for the same mailbox.

  5. The New-MoveRequest command then "tickles" an MRS. A tickle is an operation where the command contacts an MRS directly to alert it to a new move request that is ready for pick up and processing. Which MRS is contacted is chosen at random from the CAS servers in the same AD site as the mailbox server where the target mailbox database is located.
  6. Mailbox Replication Service scans the mailbox databases for new interesting events. When it discovers the new interesting event, it then logs into the System Mailbox and gets information from the Move request messages.
  7. It then updates the message in the System Mailbox on the Mailbox Server's MRS that owns the moving of the mailbox.
  8. The Mailbox Replication Service will then update the msExchMailboxMoveStatus attribute on the mailbox object in Active Directory.
  9. It will then log into the source and target mailboxes using MAPI.Net and start the synchronization of the user data. This type of synchronization is also referred to as a heavy pipe operation.
  10. Once the initial synchronization is complete, most all of the mailbox data will be synchronized to the target mailbox. The Mailbox Replication Service will then lock the mailbox.
  11. Mailbox Replication Service will then complete the synchronization of the data including any new or changed items. This last synchronization data is typically not a full synchronization; instead it is a moving of changed and new items.
  12. The Mailbox Replication Service will then update the following attributes in Active Directory on the mailbox account to point to the new mailbox.
    • HomeMDB
    • HomeMTA
    • HomeServer
    • MSExchangeVersion (Set the appropriate Exchange Version)
    • Proxy Address (Typically changed in Cross forest moves)
  13. The move history is then written to the user's mailbox.

    In the following screenshot, we can see the location of the MailboxMoveHistory using MFCMAPI.

    Contents of the MailboxMoveHistory folder:

  14. The Mailbox Replication Service does not remove a move-mailbox request message from the System Mailbox. The message is removed when the move request is removed by the Remove-MoveRequest cmdlet.
  15. The Mailbox Replication Service then removes the source mailbox from the source database. It then changes the move status on msExchMailboxMoveStatus and msExchMailboxMoveFlags attributes to indicate that the move completed on the mailbox in Active Directory.
  16. Once the status has been changed to "Completed" the mailbox can be accessed again.

An Offline move works similar to the steps listed above with the exception that it will lock the mailbox database so no one can access the mailbox.

Mailbox Replication Service Queue

Each MRS keeps track of all move requests in its Active Directory site. It does this by scanning all System Mailboxes in the site. As mentioned earlier, each move request command creates a message in the System Mailbox of the target database. These messages are saved in the following folders:

  • MailboxReplicationService Move Jobs
  • MailboxReplicationService Move Reports
  • MailboxReplicationService SyncStates

The messages contain queue information about the move request. This queue information can be accessed using the Get-MoveRequestStatistics command with the MoveRequestQueue parameter.

Below is a screenshot of a System Mailbox opened using MFCMAPI that has move request messages. Shown are the 3 different folders that the MRS uses to store information about move requests.

System Mailbox in MFCMAPI

The MailboxReplicationService Move Jobs and MailboxReplicationService Move Reports folders contain information about the move request stored as messages within the folders. Each message in the folder represents a single mailbox move represented by the msExchMailboxGUID attribute of the mailbox enabled account. The below screenshot shows the contents of the MailboxReplicationService Move Reports folder.

Mailbox moves and database failures

By default Mailbox Replication Service waits 30 seconds before attempting to reconnect to a database if it encounters transient problems during a move operation. It will try to reconnect every 30 seconds until a successful connection or 60 retries. If it cannot connect after 60 retries then it puts the move request into a failed state. The default retry interval and maximum number of retries can be changed by editing the MAXRetries and RetryDelay values in the MsexchangeMailboxReplication.exe.config file.

Mailbox moves and High Availability

Move mailbox operations that involve databases in a DAG environment are different than move mailbox operations on standalone databases.

The active database, the passive database and log shipping are factors that affect move mailbox in a DAG environment. MRS checks with the Active manager component of the Exchange replication service before, during and right before completing a move request to see if the active copy is up, if log shipping is not lagging behind and if the passive copies are keeping up. The action taken depends on a property of the database called DataMoveReplicationConstraint. If this value is not set, then the move operation assumes SecondCopy option if the database has a copy. That is the move operation does not take into consideration log shipping and the passive copies. If this value is set the action depends on what the actual value is.

Possible values for the DataMoveReplicationConstraint property are:

  • None - The move operation treats the move just as it treats move mailbox operation on a standalone database. This is the default if the database is not replicated.
  • SecondCopy - If the database is replicated then at least one passive mailbox database copy must have the changes synchronized. This is the default value.
  • SecondDatacenter - If the database is replicated to two AD sites then at least one passive mailbox database copy in another AD site must have the changes replicated.
  • AllDatacenters - If the database is replicated to multiple AD sites then at least one passive mailbox database copy in each AD site must have the changes replicated.
  • AllCopies - If the database is replicated then all passive mailbox database copies must have the changes replicated.

The DataMoveReplicationConstraint property can be set by running the Set-MailboxDatabase with DataMoveReplicationConstraint parameter.

Mailbox Replication Service and High Availability Configuration

In addition to the DataMoveReplicationConstraint property of a database, the following two settings in msExchMailboxReplication.exe.config file also control the behavior of move mailbox that involves a DAG.

  • DataGuaranteeCheckPeriod - Controls how often the MRS checks with the active manager. The default value is 5 minutes with a minimum value of 30 seconds and maximum of 2 hours.
  • EnableDataGuaranteeCheck - When enabled the MRS checks with the active manager on the status of the mailbox databases. The default value is True.

Note: In order for the Mailbox Replication Service to check with active manager to see if the mailbox database is healthy and not behind processing log files both the EnableDataGuaranteeCheck in the MSExchangeMailboxReplication.exe.config and the DataMoveReplicationConstraint on the mailbox database need to be enabled. If they are enabled then the MRS will throttle back on processing the data transfer when the active manager reports database replication is not in a healthy state.

Database Availability Group Failover

During a move operation if the active database becomes unavailable then MRS contacts the active manager to see which copy will take over. MRS then will logon to the mailbox on the new database and will continue the move from where it left off. This is provided that the DataMoveReplicationConstraint property is set to other than none and also the database was not down for longer than 30 minutes. (Or there is another copy satisfying the constraint. If say the database has 3 copies, it's entirely possible that MRS will just continue working after a failover even if the original server is down). If DataMoveReplicationConstraint is set to none then MRS will try to connect to the same database every 30 seconds for the next 30 minutes. The 30 minute is from the maximum retry of 60 times every 30 seconds. Off course this value can be changed in the in msExchMailboxReplication.exe.config file.

Cross Forest Moves

Exchange 2010 has the ability to move mailboxes between Active Directory forests. The MRS is responsible for moving mailboxes to an Exchange 2010 Mailbox server. When it comes to moving mailboxes from one Exchange forest to an Exchange 2010 forest, there are two move types:

  • Remote - An Exchange 2010 Client Access (CAS) server is present in the source forest
  • Remote Legacy - There is no Exchange 2010 CAS server in the source forest

When there is no Exchange 2010 CAS server in an Exchange forest that is the source of a mailbox move, the MRS in the Exchange 2010 target forest is designed to process the move request in a manner similar to previous versions of Exchange. In this case the MRS communicates directly to the Active Directory (AD) Directory Service in the source forest, as well as the mailbox server where the source mailbox is located.

When the source forest is also an Exchange 2010 forest, the MRS is designed to move mailboxes between the forests using a new feature that simplifies and improves the process.

In previous versions of Exchange in order to move mailboxes between different Active Directory forests, the administrator would have to allow direct MAPI access to servers and configure trusts and give other administrators full access to each other's Exchange Organization. This way of moving mailboxes was not going to be effective with moving mailb oxes to Exchange Online and between forests in Exchange 2010.

To overcome these issues Exchange 2010 introduces the Mailbox Replication Proxy Service (MRSProxy). The Mailbox Replication Proxy service works in conjunction with the MRS to facilitate the required communication between the source and target servers in each Exchange 2010 forest. Each CAS server that has an instance of the MRS also has an instance of the MRSProxy service as part of the implementation. Essentially, Mailbox Replication Proxy Service is a web service for the Mailbox Replication Service. It is part of the Exchange Web Services (EWS). The Mailbox Replication Proxy service will proxy MAPI, ExRPCAdmin and LDAP requests between local and remote forests when moving mailboxes. These requests will be HTTP requests that the Mailbox Replication Proxy Service will proxy these requests to the Mailbox Replication Service. The Mailbox Replication Service then communicates with the mailbox servers and sends the data back through the Mailbox Replication Proxy Service. The Mailbox Replication Proxy Service then communicates back to the Mailbox Replication Proxy server that initiated the request.

Many thanks to Stephen Gilbert and Jonathan Runyon for providing detailed information on Local Online Mailbox Move and their contributions to this blog. Thanks to Otavio Pereira, Matt Richoux and Nasir Ali for their review.

- Hamza Hassen, Nagesh Mahadev

24 Comments
Not applicable
This is great information thank you for providing it.
Not applicable
Awesome that this is supported with Exchange 2007 SP2 as a source.  It will totally simplify our 2010 migration plans and possibly avoid the need for some 3rd party software we were considering.  Was this ever documented before?  We were under the impression that online moves would only be available after we were already on 2010.
Not applicable
@Oliver - yes, it's documented in Technet somewhere.

Interesting read, thanks guys. I'm concerned about the way online mailbox moves work though. Can you confirm if there is a restart requirement for MS Outlook (either 2007 or 2010) after an online mailbox move? Because that's exactly what happened to me..."Your administrator has made a change etc etc" which is exactly the sort of thing I though an online MBX move was supposed to change.

Thanks!
Not applicable
I was wondering if do we still have the System attendant mailbox in Exchange 2010

In the section
How the Mailbox Replication Service works
When a move mailbox request is issued, the command creates a message in the system attendant mailbox of the target database

It has been corrected later in this article

The New-MoveRequest command creates a request message in the target database's System Mailbox as special message

Gr8 Information Nagesh
Not applicable
We moved some Mailboxes from 2003 to 2010.

Two things I found:

- The "HomeServer" Atribut is missing at the User account I checked
- The "HomeMTA" Attribut is still filled with the old Server.

Is this a mistake in your blog? Or is this normal for Exchange 2003 moves?
Not applicable
Any guidance around tuning the throttling parameters to maximize mailbox move throughput in a large scale migration (15,000+)? In a consolidation scenario the target AD site might have only 2 - 2010 CAS servers (which are effectively your migration machines).   Will updating the concurrent move parameter from 5 to say 10 give you any substantial gains?  Or leaving it alone and testing be the best option ?  I would like the leverage Exchange 2010 vs. buying a 3rd party product and not worry about the CAS servers not being able to keep up with the load.
Not applicable
i've seen quite a while migrations where move requests from 2003 to 2010 are completing with warning.. seems like the last part of the move is failing, and we have to issue a manual "run cleanup agent" on the 2003 server.

is tweaking the MSExhangeMailBoxReplication.Exe.Config possible to bypass this ?
Not applicable
@Tom: So, we’d advise against simply doubling the number, as the load will actually be on the target server, not on the CAS. The number is mostly dictated by the mailbox average size. If most of them are small (<20Mb I’d say), they may be able to go as high as 20 (depending on how powerful their mailbox servers are). Again, CAS is the least of the worries here. CAS will only start to suffer if all moves have really large folder hierarchies (2000+ folders), in which case the memory pressure will build up. I’d say profiling and testing is the best option. It may be possible that during the initial phases of the migration the target servers can handle more simultaneous moves, and as more users start using their mailboxes on the target forest, the load available for moves will decrease, so they’ll have to throttle down.
Not applicable
@ilantz: There could be several reasons. If that’s the issue we’ve been seeing, then unfortunately no. It’s a bug on E2k3 that couldn't be fixed. IIRC getting users to log off their mailboxes before the move starts seems to mitigate it. Again every individual situation can be different.
Not applicable
Is there a way to NOT delete the source mailbox as there was in EX2007?  In 2007, the Move-Mailbox cmdlet is actually a copy and you add the -DeleteSourceMailbox to effect the "clean-up".  I'm not seeing anything along these lines for the New-MoveRequest cmdlet, which implies that if I start a migration that runs long (say, into business hours) and need to interrupt it, I have the split org and no way to do an AllowMerge for any delta data.
Not applicable
@nagesh - thanks ! i was actually looking also for a script that could issue the clean-up agent automatically on the 2003, that could be a HUGE "trade-off" it's a pain to go back to each db in the 2003 to have the move-mailbox "finish" ....
any Team suggestions here ? please !
Not applicable
@Christopher : The –AllowMerge behavior on the move-mailbox was a simple patch for the lack on online availability during the move. Since now the moves are online by default, this behavior should not be needed anymore. Any cross-forest deployment can be set for rich co-existence between the 2 forests, so that mailboxes on either forest have full access to all the features on the other forest (like free/busy, calendaring, …), so even if half of the mailboxes have been moved, there no reason to keep the migrated mailboxes on the source forest (it only complicates the user’s life, since when the account is moved by exchange AutoDiscover will prompt Outlook to reconfigure itself, while keeping the mailbox on the source will force them to manually fix the client when the source forest is being decommissioned). So, no, there’s not a way to prevent the source mailbox from being deleted. There will be something in SP1, but the left behind mailbox will be for data restore purposes only – in case the target mailbox suffers something terrible right after the move.
Not applicable
@Christopher : Forgot to mention, if there’s a need to only “stage” the move and then do as many switchovers as needed (say the rich coexistence cannot be set up for some reason), you can leverage –SuspendWhenReadyToComplete to only copy the data but not finalize the move and create the mailbox object on the target forest, and when all the mailboxes have been staged, you can then resume the moves and let MRS do the final merging and convert the mailbox objects.
Not applicable
@Ilantz : I'd suggest opening a case with CSS to isolate and provide you with an accurate solution to your specific issue.
Not applicable
Hi Coders...
Could you add to the "Move-Mailbox" cmdlet one additional parameter, which can be e.g. an array. So that you can set group of source mailboxes and move them to more than one target store.
Currently you can have more than one mailboxes as a source, but the destination must be always single store, why? (unless you use your own logic in a script(s)...)
Not applicable
@Petri : There’s no “move-mailbox” cmdlet. I assume you are talking about the MoveMailbox.ps1 script.

If that’s the case, you are free to change, but there’s already some capabilities to do what you want. You can use the -DatabaseMap and pass a map of source to target databases, so all users from dbA get moved to dbB, all users from dbC to dbD and so on. If you want random distribution, you just need to suspend the remaining databases from provisioning and don’t specify a database at all. Take a look at http://technet.microsoft.com/en-us/library/dd876878.aspx for more information on MoveMailbox.ps1 Script.
Not applicable
Hi,
I am doing a cross forest e2k3 to e2k10 remotelegacy mailbox move project. I am getting inconsistent results after the mailbox move completes with regard to how long it takes until the mb becomes readily available. There is a background process run by system attendant that needs to run before the migrated mb can be accessed (my target e2k10 servers have multiple dbs and are in a DAG) There is a forum post that supposedly has the fix-but its not really a fix since we have 8 databases and want to move mailbox to each of the 8 during each batch processing and dont want to wait a few hours. here is the post:http://social.technet.microsoft.com/Forums/en-US/exchange2010/thread/2b6fb514-34c8-488a-be4d-60c58dd...

Can someone who has the insight explain what is really going on in the background process- and is there anyway to force it after the move request is 100% complete to speed up db access?
thanks-Omar
Not applicable
@Nagesh, I was trying to follow this to move the Arbitration mailboxes, so that I could remove the first mailbox datastore.
http://markswinkels.nl/2009/05/19/exchange-2010-delete-the-first-mailbox-database/
You've said that there is no "Move-Mailbox" cmdlet, but this seems to have worked for some people.

I'm trying to move the mailboxes to a store on another server, as I have to decomission the first Exchange Server and I haven't been able to get this to work.

Any ideas?
Not applicable
Ignore my last comment!
I found the answer.
The MoveRequest script did the trick.
Not applicable
I know a random CAS is picked to move a mailbox...is there anyway you can specify a particular CAS?
Not applicable
i am doing a cross forest mailbox move 2k3 to 2k10. after the move, i am unable to access some mailboxes. it says cannot find mailbox for the user(in OWA). but if i remove the user i can see that the mailbox was moved and its it the disconnected mailboxes. when i reconnect the mailbox it works.
can someone tell me how i can avoid this issue
thanx
Not applicable
Is there any performance monitoring built into the bad boy?  In E2K3 you could have very easily dismounted a database with mailbox moves...
Not applicable
@Nagesh -

1.still wondering  what Christopher asked already - Is there a way NOT to delete source mailbox when doing e.g. cross-forest transition/migration Exc 2003 --> 2010?
Is there any changes to this in Exchange 2010 SP1 release?

It has been a very usefull "Rollback plan" in transitions, when source mailboxes has been intacted and rollback is just "not to do any changes" (SMTP routing, Outlook profile change,, etc.)
Now if the mailboxes will be purged from source system and in the target system happens some kind of a major disaster, like HW meltdown, there is no "light" rollback scenario" like we used to have...
(one solution -or should we say a "workaround"- would be to install Exchange 2007 first to the target domain, migrate there and then move the mailboxes to Exchange 2010 - however this means a little bit more hours to achieve)


2. Another thing I was wondering when doing cross-forest transition is the order of running PrepareMoveRequest script and ADMT with Exchange SP1 - is it still so that you run PrepareMoveRequest script first then ADMT account migration?

and is there a diffenrence when doing it to source Exchange Resource forest where you have disabled accounts with mailboxes comparing to normal scenario where you have accounts and Exchange in the same forest?

thx

/jc
Not applicable
@jiicee,
What I came up with in a cross forest migration (w2k3/e2k3-->W2k8r2/E2k10).

0. setup secondary email address space on both sides.
1. Run ADMT 3.1 (was in place before 3.2 came out)(with Sidhistory)
2.  run Enable-mailuser with identity and alias and externalemailaddress
3. Prepare move request -uselocalobject (key here is Updated the script to search the entire AD instead of looking in the users container or a specified OU)
4.  On the Exchange 2003 server- I export all mailboxes use Exchange Migration wizard to password protected PST files and secure the password paster file( this is my rollback/recovery option if needed since source MB gets dumped)
5. new-moverequest command (specify the secondary for e2k10 email address.
6. Miller time.
This process worked well for me- If you do preparemoverequest 1st to create the object- the issue I found is that all SMTP email addresses carry over- and we did not want that- we only wanted the secondary email address for routing.

My only issue-which still remains- some mailboxes are not available for logon for 1 to 3 hours. This is ok since we migrate at night and setup the migration to complete around 2am leaving us about 5 hrs local time until users come in the office.

In our scenario- we also put users in a security group when we migrate them- there is a GPO that is filtered to this group- that has a logon script that runs outlook with the prf switch to create them a new profile- leaving the old profile intact for reference behind the scenes.
Hope this helps.
Version history
Last update:
‎Jul 19 2010 05:56 PM
Updated by: