The 100 migration batches limit and how to not run into it
Published Apr 19 2019 08:00 AM 30K Views

As you might or might not know, in Exchange Online we have an upper limit of 100 migration batches. You can see this limit for MaxNumberOfBatches in the Exchange Online PowerShell Get-MigrationConfig: 100batches (To understand the MaxConcurrentMigrations limit of 300, you can check this blog post written by one of our migration experts, Brad Hughes.) 100 batches created at the same time should be enough for all O365 multi-tenant customers. But, as we have seen in support, sometimes there are support tickets opened because customers would see the error that says: “The maximum number of migration batches is already running. Please remove a batch before you add another one”. The main reason why this number of batches limit is in place is that having more than 100 batches causes performance problems and could cause outages for other customers in the multi-tenant system. Additional reading on resourcing for mailbox moves can be found here. Let’s say that you want to migrate 50,000 mailboxes to Exchange Online: you could create 50 batches with 1000 users each, and you would still have 50 batches available. If you need to complete all the migrations of users from Batch_1 at the same time, you can easily set -CompleteAfter at the migration batch level. Once this batch is completed, you could delete it to create room for another one. In real world, though, we have seen that our customers rarely want to complete large amount of mailbox migrations at the same time. This is usually the reason why large migration batches are not created but rather smaller batches are preferred. That way, all the migrated users from one migration batch at a time can be completed together. In this scenario, our recommendation is to have large batches for initial sync and if you need to complete the migration for users at a separate date, then you would complete the migration for groups of users in smaller batches, as needed. So how would you create small completion batches for already synced users from larger synced batches? Suppose you want to complete migration of users A, B and C, who reached Synced status and they are all contained in a large batch of 1000 users called Batch_1. To do this, first ensure that you have fewer than 100 batches created at the moment:

(Get-MigrationBatch).count

Then, proceed with PowerShell commands to get the users from the larger batch Batch_1 into the new smaller batch CompletionABC. Then complete this smaller batch and optionally, remove the completed migration batch:

New-MigrationBatch -Name CompletionABC -UserIds EmailAddressOfUserA, EmailAddressOfUserB, EmailAddressOfUserC -DisableOnCopy -AutoStart
Complete-MigrationBatch CompletionABC
Remove-MigrationBatch CompletionABC

Another method we sometimes see people using is setting -CompleteAfter on the move requests directly using Set-MoveRequest.  We don't recommend this method because the Migration Service may overwrite the per-request CompleteAfter setting with the one from the batch without notice. This method of individually completing the move requests should only be done when you’ve started the migration through PowerShell, using New-MoveRequest cmdlet, without creating migration batches to contain those move requests. I hope this post gives you a better idea of how to manage migration batches with many users being migrated and not have to run into the limit of number of batches. See you in the cloud! Special thanks to Brad Hughes and Nino Bilic who contributed to this blog post and their precious help in my daily support job. Mirela Buruiana
9 Comments
Not applicable
Thanks for the info. One question - do you need to run 'Start-MigrationBatch CompletionABC' between the New-MigrationBatch command and the Complete-MigrationBatch command? In my quick testing, the CompletionABC batch was in a 'Stopped' status after the New-MigrationBatch command and when I ran Complete-MigrationBatch, I received the following:

Are you sure you want to complete the migration batch "CompletionABC"? There are 3 pending items that won't be migrated because the batch hasn't been restarted since the items were added.

Ignoring the warning and responding 'Yes' didn't seem to do anything thing. The CompletionABC batch remained in a 'Stopped' status.

However, the test in which I ran ‘Start-MigrationBatch’ after ‘New-MigrationBatch’ and before ‘Complete-MigrationBatch’ completed as expected.

Not applicable
Thanks for the feedback, you are correct, we missed out the start-migrationbatch part. We will modify in blog the first command New-MigrationBatch with -AutoStart switch.
Not applicable
Thanks for the follow-up/update.

What is the intent of the -DisableOnCopy parameter? I notice that that the original migration user remains in the original migration batch as an Error with the reason:

MigrationUserMovedToAnotherBatchException: The user was moved to a new batch

Which makes sense but I'd really like the option to remove the user from the original batch. In fact, I follow up with Remove-MigrationUser to clean things up. This is fine when using EAC but, if I want to use PowerShell, I need to target the GUID of the MigrationUser as the Identity/SMTPAddress now matches multiple MigrationUsers:

The operation couldn't be performed because '' matches multiple entries.

Not applicable
Thanks for raising this. To answer your questions:

- The DisableOnCopy switch disables the original migration job item for a user if you're copying users from an existing batch to a new batch by using the UserIds or Users parameters. Reference https://docs.microsoft.com/en-us/powershell/module/exchange/move-and-migration/new-migrationbatch?view=exchange-ps

- This error MigrationUserMovedToAnotherBatchException is normal once we disable the migration user from the source batch and copied to another one.

- Removal of the Migration User from the Source Batch must be done manually, at least for the moment. This way we have a record of having moved them around. If you use PowerShell, then yes, as you mentioned, you need to specify an unique identifier of the migration user (remove-migrationUser -identity id) otherwise you get into this error "The operation couldn’t be performed because ” matches multiple entries.". It is maybe easier to remove the migration user from GUI.

Copper Contributor

This is a great feature, and have been testing it quite heavily.  It looks promising, however we've run into an issue that may be problematic.

 

User1 is in InitialBatch1

User2 is in InitialBatch2

 

We would like to move the 2 users out of InitialBatch1 and InitialBatch2, to a single batch InitialBatch3.

 

We are getting this error:

A unique batch must be used when copying users from one batch to another.

 

Have you come across this?

 

Thanks in advance!

 

Microsoft

@Ryan Schellenberg , sorry for the delay. The error for the unique batch is because you were trying to combine batches, you can't make users from different batches go into the same batch because we cannot copy settings from different source batches and make them into one target batch. In the blog example, I mentioned single, larger batch Batch_1 containing users A, B, C (all from the same batch).

Microsoft

Hi, my name is Mirela Buruiana and I am the main author of this blog. Reply to this comment if you want me to get notified about your question /comment.

Copper Contributor

Hello @Mirela_Buru ,

 

Is there a way to use a CSV for your new "Complete" batches, or are you limited to using only individual usernames?

 

Thanks in advance for any help!

Microsoft

@JBlalock , sorry for delay. You can do so something similar to this for UserIds parameter: 

Get-MigrationUser -BatchId batch1 | select MailboxEmailAddress | export-csv usersBatch2.csv -NoTypeInformation
# modify the CSV keep only the users you need 
$usersBatch2 = Import-Csv .\usersBatch2.csv
New-MigrationBatch -Name batch2 -UserIds $usersBatch2.MailboxEmailAddress
Start-MigrationBatch batch2
Version history
Last update:
‎Jul 01 2019 04:36 PM
Updated by: