Blog Post

Exchange Team Blog
3 MIN READ

The 100 migration batches limit and how to not run into it

The_Exchange_Team's avatar
Apr 19, 2019

As you might or might not know, in Exchange Online we have an upper limit of 100 migration batches. You can see this limit for MaxNumberOfBatches in the Exchange Online PowerShell Get-MigrationConfig: 100batches (To understand the MaxConcurrentMigrations limit of 300, you can check this blog post written by one of our migration experts, Brad Hughes.) 100 batches created at the same time should be enough for all O365 multi-tenant customers. But, as we have seen in support, sometimes there are support tickets opened because customers would see the error that says: “The maximum number of migration batches is already running. Please remove a batch before you add another one”. The main reason why this number of batches limit is in place is that having more than 100 batches causes performance problems and could cause outages for other customers in the multi-tenant system. Additional reading on resourcing for mailbox moves can be found here. Let’s say that you want to migrate 50,000 mailboxes to Exchange Online: you could create 50 batches with 1000 users each, and you would still have 50 batches available. If you need to complete all the migrations of users from Batch_1 at the same time, you can easily set -CompleteAfter at the migration batch level. Once this batch is completed, you could delete it to create room for another one. In real world, though, we have seen that our customers rarely want to complete large amount of mailbox migrations at the same time. This is usually the reason why large migration batches are not created but rather smaller batches are preferred. That way, all the migrated users from one migration batch at a time can be completed together. In this scenario, our recommendation is to have large batches for initial sync and if you need to complete the migration for users at a separate date, then you would complete the migration for groups of users in smaller batches, as needed. So how would you create small completion batches for already synced users from larger synced batches? Suppose you want to complete migration of users A, B and C, who reached Synced status and they are all contained in a large batch of 1000 users called Batch_1. To do this, first ensure that you have fewer than 100 batches created at the moment:

(Get-MigrationBatch).count

Then, proceed with PowerShell commands to get the users from the larger batch Batch_1 into the new smaller batch CompletionABC. Then complete this smaller batch and optionally, remove the completed migration batch:

New-MigrationBatch -Name CompletionABC -UserIds EmailAddressOfUserA, EmailAddressOfUserB, EmailAddressOfUserC -DisableOnCopy -AutoStart
Complete-MigrationBatch CompletionABC
Remove-MigrationBatch CompletionABC

Another method we sometimes see people using is setting -CompleteAfter on the move requests directly using Set-MoveRequest.  We don't recommend this method because the Migration Service may overwrite the per-request CompleteAfter setting with the one from the batch without notice. This method of individually completing the move requests should only be done when you’ve started the migration through PowerShell, using New-MoveRequest cmdlet, without creating migration batches to contain those move requests. I hope this post gives you a better idea of how to manage migration batches with many users being migrated and not have to run into the limit of number of batches. See you in the cloud! Special thanks to Brad Hughes and Nino Bilic who contributed to this blog post and their precious help in my daily support job. Mirela Buruiana
Updated Jul 01, 2019
Version 2.0
  • This is a great feature, and have been testing it quite heavily.  It looks promising, however we've run into an issue that may be problematic.

     

    User1 is in InitialBatch1

    User2 is in InitialBatch2

     

    We would like to move the 2 users out of InitialBatch1 and InitialBatch2, to a single batch InitialBatch3.

     

    We are getting this error:

    A unique batch must be used when copying users from one batch to another.

     

    Have you come across this?

     

    Thanks in advance!

     

  • Thanks for the info. One question - do you need to run 'Start-MigrationBatch CompletionABC' between the New-MigrationBatch command and the Complete-MigrationBatch command? In my quick testing, the CompletionABC batch was in a 'Stopped' status after the New-MigrationBatch command and when I ran Complete-MigrationBatch, I received the following:

    Are you sure you want to complete the migration batch "CompletionABC"? There are 3 pending items that won't be migrated because the batch hasn't been restarted since the items were added.

    Ignoring the warning and responding 'Yes' didn't seem to do anything thing. The CompletionABC batch remained in a 'Stopped' status.

    However, the test in which I ran ‘Start-MigrationBatch’ after ‘New-MigrationBatch’ and before ‘Complete-MigrationBatch’ completed as expected.

    • Deleted's avatar
      Deleted
      Thanks for the feedback, you are correct, we missed out the start-migrationbatch part. We will modify in blog the first command New-MigrationBatch with -AutoStart switch.
      • Deleted's avatar
        Deleted
        Thanks for the follow-up/update.

        What is the intent of the -DisableOnCopy parameter? I notice that that the original migration user remains in the original migration batch as an Error with the reason:

        MigrationUserMovedToAnotherBatchException: The user was moved to a new batch

        Which makes sense but I'd really like the option to remove the user from the original batch. In fact, I follow up with Remove-MigrationUser to clean things up. This is fine when using EAC but, if I want to use PowerShell, I need to target the GUID of the MigrationUser as the Identity/SMTPAddress now matches multiple MigrationUsers:

        The operation couldn't be performed because '' matches multiple entries.

  • Ryan Schellenberg , sorry for the delay. The error for the unique batch is because you were trying to combine batches, you can't make users from different batches go into the same batch because we cannot copy settings from different source batches and make them into one target batch. In the blog example, I mentioned single, larger batch Batch_1 containing users A, B, C (all from the same batch).

  • Hi, my name is Mirela Buruiana and I am the main author of this blog. Reply to this comment if you want me to get notified about your question /comment.

  • JBlalock's avatar
    JBlalock
    Copper Contributor

    Hello Mirela_Buru ,

     

    Is there a way to use a CSV for your new "Complete" batches, or are you limited to using only individual usernames?

     

    Thanks in advance for any help!

  • JBlalock , sorry for delay. You can do so something similar to this for UserIds parameter: 

    Get-MigrationUser -BatchId batch1 | select MailboxEmailAddress | export-csv usersBatch2.csv -NoTypeInformation
    # modify the CSV keep only the users you need 
    $usersBatch2 = Import-Csv .\usersBatch2.csv
    New-MigrationBatch -Name batch2 -UserIds $usersBatch2.MailboxEmailAddress
    Start-MigrationBatch batch2