Troubleshooting IMAP Migrations to Office 365
Published Jan 17 2019 12:53 PM 104K Views

During my day to day work as a part of support organization, I work with and help troubleshoot mailbox migrations very often. One type of migrations that we see quite often is IMAP migrations. I wanted to put together an overview of IMAP migration good practices as well as troubleshooting tips related to IMAP migrations. Hopefully, you find this useful! I realize that the second part of the post is a bit dry and might not be relevant unless you actively need to troubleshoot. A few things to get out of the way first:

  • We support migrations from IMAP servers to Office 365 Exchange Online using IMAP4 protocol.
  • We don’t support IMAP migrations between two Office 365 tenants. Our documentation for this scenario can be found here.
  • We also don’t support offboarding back to an IMAP server from Exchange Online using IMAP. Offboarding is only possible using Mailbox Replication Service (MRS) migrations using an MRSProxy endpoint.
  • We don’t recommend IMAP migrations from Exchange on-premises servers. In such scenarios, we recommend using a native migration method (Cutover or Minimal / Express Hybrid). We also don’t recommend Staged migration from Exchange 2007 or Exchange 2003 servers if you intend to keep directory synchronization in place after mailbox migration to Office 365. The recommended migration method when migrating from these old Exchange Servers when Directory Synchronization is required for your organization is Minimal Hybrid Configuration; this requires inserting an Exchange 2010 (for Exchange 2003 servers) or Exchange 2013 (for Exchange 2007 servers) and running Minimal HCW. This will give you a free Hybrid license and will allow you to perform an MRS migration (this means there is no need to recreate Outlook profiles). The Minimal HCW will also allow you to be in a supported scenario when you have Exchange Online Mailboxes and the AD users synced to Azure AD and you will need to manage your Office 365 Mailboxes using on-premises Exchange 2010 / Exchange 2013 Management tools (Exchange Admin Center and Exchange Management Shell). This is due to directory synchronization which requires administration of Exchange Online Mail Attributes from on-premises Exchange for your synced users. Managing Exchange Online Mailboxes from Exchange 2007 or Exchange 2003 servers after mailbox migration is not supported. More about Minimal and Express Migrations can be found here.

Here are some considerations that you need to take into the account when you perform an IMAP Migration:

  • Only items in a user's inbox or other mail folders can be migrated. You can't migrate contacts, calendar items, or tasks. (Note that we are working on a richer migration experience for G Suite to Exchange Online migrations, as mentioned here.)
  • By default, the maximum message size that can be migrated is 35 MB. This can be increased up to 150 MB.

There are 2 ways to perform an IMAP migration to Office 365:

  1. Using a graphical user interface: either Exchange Admin Center or Office 365 Admin Center
  2. Using PowerShell

Either method you choose, the IMAP migration needs an IMAP Migration Endpoint created first. Here are some screenshots from Office 365 Exchange Admin Center that show where to locate and create Migration Endpoints: ImapMig1 ImapMig2 And this is a PowerShell view of an IMAP Endpoint (from get-migrationendpoint) established with remote IMAP server on port 993 and using SSL: ImapMig3 If you have issues with creating the IMAP Endpoint, you can use Test-MigrationServerAvailability command while connected to Exchange Online PowerShell to test IMAP connectivity:

Test-MigrationServerAvailability -Imap -RemoteServer <Your IMAP server> -Port 993
Test-MigrationServerAvailability -Imap -RemoteServer <Your IMAP server> -Port 143

When you create the migration batch and specify the Office 365 user mailboxes where you will migrate content via IMAP, the migration service will create Migration Users and corresponding Sync Requests for each user you specified in the CSV File.

IMAPcorrected.jpg

Similarly, for Hybrid Remote Moves (Full or Minimal Hybrid Configurations), you would have Move Requests created for the users. This is how you would create the Migration Batch via Exchange Admin Center based on a CSV File where you specify the mailboxes you would migrate to:

ImapMig5 You can optionally exclude folders in the IMAP Migration Batch if you don’t want to migrate specific folders, or if you find yourself in scenarios where we have a corrupted IMAP folder. When troubleshooting failed IMAP migrations, Microsoft support will need at least one of the following XML reports from you:

Get-MigrationBatch -IncludeReport -DiagnosticInfo "showtimeslots, showtimeline, verbose" | Export-Clixml C:\temp\EXO_All_Batches.xml
Get-MigrationBatch <Specific Migration Batch Name> -IncludeReport -DiagnosticInfo "showtimeslots, showtimeline, verbose" | Export-Clixml C:\temp\EXO_Batch_X.xml
Get-MigrationUserStatistics <affected user SMTP> -IncludeSkippedItems -IncludeReport -DiagnosticInfo "showtimeslots, showtimeline, verbose" | Export-Clixml C:\temp\EXO_MigUserStats1.xml
Get-SyncRequest -Mailbox <affected user SMTP> | Get-SyncRequestStatistics -IncludeReport -DiagnosticInfo "showtimeslots, showtimeline, verbose" | Export-Clixml C:\temp\EXO_SyncReq.xml

However, depending on when (in which stage) things have failed, we might not always have a Sync Request for the user or even have a Migration Batch. In that case, you would run commands similar to these:

Get-MigrationEndpoint |FL
Test-MigrationServerAvailability -Endpoint <Identity of the IMAP endpoint from above>
Test-MigrationServerAvailability -Imap -RemoteServer <IMAP server> -Port 993
Test-MigrationServerAvailability -Imap -RemoteServer <IMAP server> -Port 143
Get-MigrationBatch
Get-MigrationUser
Get-SyncRequest -Mailbox <affected user>

If you do have a Sync Request created for the user that failed migration, then you should run Get-SyncRequestStatistics command, as described above, to export the XML report in order to retrieve more information on the cause of the issue. If you don’t have a Sync Request created, but you have a Migration User, you would run Get-MigrationUserStatistics command, as described above. Usually, when customers open support cases with support, for any migration to Office 365, this is because of 2 main reasons:

  • Migration failed, and they are unable to migrate one or more users to Office 365. Errors break down in two categories:
    • Permanent – the error that actually made the migration fail
    • Transient – errors which might slow down the migration to the point where it might fail at the end
  • Migration is slow or stalled due to Office 365 Resource Throttling or IMAP Server performance issues. These depend on many factors like source server performance and network related configurations, capabilities or issues. More info here. It can also happen that Office 365 will stall the migrations to protect Office 365 Servers health and also because migrations have a lower priority assigned than things like mail-flow tasks or client connectivity. More info on that can be found here.

Errors are our friends – how to troubleshoot in practice

Based on the failure you got in the Sync Request Statistics or Migration User Statistics, you can find out what is causing the issue and get more details on the migration error. When troubleshooting an IMAP Migration, find out if you have a Sync Request created for the user. If there is one, you would retrieve the Sync Request Statistics for it and store it in a variable or directly export it in an XML report to look at it into detail. To see if the sync request is created for the user, run the following command in Exchange Online PowerShell:

Get-SyncRequest -Mailbox <Affected User SMTP>

ImapMig6 Check the STATUS of the Sync Request, is it Failed / Synced / Syncing? Supposing it is Failed, you would then store the Sync Request Statistics into a variable, I used $syncstats in my example below.

$syncstats = Get-SyncRequestStatistics crystal@contoso.com -IncludeReport -DiagnosticInfo "showtimeslots, showtimeline, verbose"

We would then look at the failures. Here are some examples of commands to check various failures:

  • Retrieve Last Failure: $syncstats.Report.Failures[-1]
  • Retrieve the Last 2 Failures: $syncstats.Report.Failures | select -Last 2
  • Retrieve First Failure: $syncstats.Report.Failures[0]
  • Count all failures and group them by failure type: $syncstats.Report.Failures | group failuretype | ft -autosize
  • List all the failures with their details: $syncstats.Report.Failures

Similarly, if you don’t have a Sync Request but do have a Migration User created and we failed to create a Sync Request, we would need to gather the Migration User Statistics for that user in order to get more details. If you run Get-MigrationUser, you would list all migration users: their Identity, Name of the Migration Batch user is part of, Status and Last Synced time. Example (Get-MigrationUser): ImapMig7 We would then store the Migration User Statistics into a variable or export to XML.

# Store into a variable:
$ustats = Get-MigrationUserStatistics crystal@contoso.com -IncludeSkippedItems -IncludeReport -DiagnosticInfo "showtimeslots, showtimeline, verbose"
# Export to an .xml file:
Get-MigrationUserStatistics crystal@contoso.com -IncludeSkippedItems -IncludeReport -DiagnosticInfo "showtimeslots, showtimeline, verbose" | Export-Clixml C:\temp\EXO_MigUserStats1.xml

Next, we would use similar commands as we did to retrieve failures and internal failures from Sync Request Statistics. For example, to retrieve Last Failure from a Migration User Statistics, you would run a command similar to this: $ustats.Report.Failures[-1] Note that for troubleshooting IMAP failures, we prefer Sync Request Statistics instead of Migration User Statistics (if we have a Sync Request in place for the user). Next up, I have gathered some of more common (that we see in support) IMAP Migration Errors / Failures and what you can do to fix them.

1. Error: Imap server sent NO response to SelectCommand

Message : Imap server sent NO response to SelectCommand. Response code: '', message: 'Invalid mailbox name: Junk/'.
Message : Imap server sent NO response to SelectCommand. Response code: '', message: 'Invalid mailbox name: Sent/'.

This can be because we have invalid folder names with Forward Slash Characters. Reference here. If you don’t need to migrate these folders, you can exclude the folders from migration also.

2. Error: Mailbox folder hierarchy contains multiple roots: FolderHierarchyContainsMultipleRootsTransientException

Error: Mailbox folder hierarchy contains multiple roots: [Folder1: EntryID: [len=54, data=563D313B503D494D41503B46503D37393145423136314534463244383234433438443433413634463936393641354444333942383235], ParentID: [len=54, data=563D313B503D494D41503B46503D33304134424342304331383135314645423643353933414244423837374630393933413432393135], Type: Generic], [Folder2: EntryID: [len=54,

The "duplicate root" parent folder is the "SomeHiddenFolder" that is not returned by IMAP LIST command. So, when MRS runs LIST to enumerate folder hierarchy, it gets something like this: Inbox SentItems TopLevelFolder TopLevelFolder/NormalSubfolder SomeHiddenFolder/ Folder1 SomeHiddenFolder/ Folder2 Note that SomeHiddenFolder is *not* returned as a separate entry in the LIST command output. You would look in report.Failures to see the Folder Names impacted. In this example, it is Folder1 and Folder2. You would check the Folder Hierarchy of the affected user by this error, paying attention to folder names and their parent folder. Problem can also be duplicated folder names or parent folder to be a Public / Shared Folder (for those raising eyebrows right about now – access to public folders through IMAP was a thing in legacy versions of Exchange, for example Exchange 2003). You can also run the following script to LIST IMAP Folders for that mailbox: Get-ImapFolders.ps1 from GitHub and send us the output to see the IMAP Folders List together with the Sync Request Statistics XML. Another thing you can do is run Remote Connectivity Analyzer for IMAP test and select Exchange Server Tab (even if source IMAP server is not an Exchange server) and then IMAP Email under INTERNET Email Tests, fill in the affected user settings and then SAVE report as HTML and send us the file in order to see the IMAP Folders list. Ultimately, our support can’t do much for you in this situation other than identifying problematic folders. It is up to you to make the Folder "SomeHiddenFolder” a SELECTABLE folder and if needed, discuss with your vendor where original data is located. You could also try to create a new folder, selectable in IMAP and move the content from "SomeHiddenFolder” to this New Folder, including its subfolders, example Folder1 and Folder2.

3. Error TooManyLargeItemsPermanentException has occurred

As per our current documentation, the message size limit that we can move to Office 365 Exchange Online during an IMAP Migration is maximum 35MB. However, for IMAP migrations, where we need to have mailboxes created in Office 365 before we can migrate with IMAP, we can increase the message size limit up to 150MB on the target mailbox and this will allow email messages up to 150 MB size to be moved to Office 365 during IMAP Migrations.

Set-Mailbox -Identity alias@domain.com -MaxReceiveSize 150MB

This setting will also allow the user to receive larger email messages via transport (if the sender is capable). We don’t recommend lowering this limit back to 35 MB if you already migrated emails larger than 35MB. The Error "Fatal Error TooManyLargeItemsPermanentException has occurred." suggests that we reached the Large Item limit set on migration batch. Usually large item limit is set to 0 and if we encountered at least 1 email message in the source mailbox bigger than 35MB (default MaxReceieveSize on the Office 365 Mailbox), the limit would have been reached and migration is failed for that user. So you need to either Increase MaxReceiveSize (maximum 150MB) on the affected O365 User Mailbox to be able to migrate email messages larger than 35MB or increase the LargeItemLimit on the migration batch from GUI or from PowerShell with set-migrationbatch -LargeItemLimit XX or on the Sync Request (Set-SyncRequest -LargeItemLimit XX) so that we can skip migration of these large items and migration won’t fail because of this.

4. Error: Imap server sent NO response to LoginCommand. Response code: '', message: 'LOGIN failed.'.

MigrationPermanentException: Unable to log into account. --> Unable to log into account. --> The username or password for this account is incorrect, or IMAP access is disabled. --> Imap server sent NO response to LoginCommand. Response code: '', message: 'LOGIN failed.'.

This error suggests bad login (wrong username or password) or that IMAP protocol is disabled. If more or all users are affected, there might be issues with IMAP server certificate. Suggestions for this issue: 1) Check if you are able to configure IMAP profile in Outlook Desktop for the user 2) Check if you are able to connect to the IMAP Source Mailbox and List folders with one of these 2 methods:

  • Run Get-ImapFolders.ps1 Script from GitHub
  • Go to Remote Connectivity Analyzer for IMAP test and select Exchange Server Tab (even if source IMAP server is not an Exchange server) and then IMAP Email under INTERNET Email Tests, fill in the affected user settings as mentioned in the CSV file you used for Migration Batch and see if you are able to connect

3) Test-MigrationServerAvailability command in EXO PowerShell

Test-MigrationServerAvailability -Imap -RemoteServer <IMAP server> -Port 993
Test-MigrationServerAvailability -Imap -RemoteServer <IMAP server> -Port 143

4) If you use a Super User / Admin password, make sure the CSV file is created properly. See this and this. 5) If the source server is Gmail, this error message might happen because 2 step verification process/ app password which is required when migrating from it.

5. Errors that suggest RFC non-compliant source IMAP Servers.

For example, we might get a Permanent Failure after these transient failures which would make migration to fail for a specific user or more.

MigrationMRSPermanentException: Error: The job encountered too many transient failures ‎(61)‎ and is quitting. The most common failure is ImapCouldNotParseResponseException/ImapUnexpectedTokenFormatException with the hit count 57. --> Imap server sent a response that we could not understand. Command: ‎'WIR01992 UID FETCH 16967 ‎(INTERNALDATE UID BODY.PEEK[])‎ ‎'. Response: ‎'‎' >>‎'‎'. --> Unexpected response format - expected ‎')‎‎', actual ‎' ‎'.

This error suggests that the IMAP server returned a bad response format, it was expected a parenthesis after Date / Time of the messages received. In this situation, you would need to discuss with vendor where at the mailbox source to help you fix this issue.

6. Errors that suggest Source IMAP Server Bugs or Internal Errors

Message : Imap server reported an error during SelectCommand indicating that it encountered some bug: 'Internal error occurred. Refer to server log for more information. [2018-11-29 10:40:44] (0.002 + 0.000 + 0.001 secs).'.
DataContext : --------
Operation: ImapMailbox->GetFolderInternal<T>() for folder < >

The error message is generated at the selection of one or maybe more folders on the Source IMAP server (SELECT command) and according to the failure, the admin should check the logs that are generated on that IMAP server around the timestamp of the failure. To retrieve the Last Failure or the previous one before last, you can use the commands below:

$syncstats = Get-SyncRequestStatistics user@domain.com -IncludeReport
$syncstats.Report.Failures[-1]
$syncstats.Report.Failures[-2]

You can use one of these 2 methods below to LIST the Source IMAP Folders for the user and see their names:

  • Run Get-ImapFolders.ps1 Script from GitHub
  • Go to Remote Connectivity Analyzer for IMAP test and select Exchange Server Tab (even if source IMAP server is not an Exchange server) and then IMAP Email under INTERNET Email Tests, fill in the IMAP User Details and check the LIST of folders.

If the folder referenced in the error is found in the Sync Request Failure, you can try to Exclude it from IMAP migration batch when retrying migration. Many thanks to all that contributed to this blog post: Nino Bilic, Brad Hughes and Cristian Dimofte.

Mirela Buruiana

82 Comments
Not applicable
Hello there Support,

How do we troubleshoot IMAP migrations where the migration just sits there for hours and I tried stopping and restarting over and over multiple times and it just doesn't do anything? I checked for errors and none are reported and in fact hasn't made any progress for hours. Any thoughts?

Not applicable
Hello Yesai,

Please collect the XML reports mentioned in the blog and open a support case with us so that we can investigate why. Most important, gather the Sync Request Statistics with IncludeReport and DiagnosticInfo showtimeslots and look at these as well:

$syncstats.DiagnosticInfo

$syncstats.Report.InternalFailures

$syncstats.Report.Entries | % {[string] $_}

$syncstats |FL *total*, *transfer*

Please be aware that migrations to Office 365 have a low priority and you might see stalls on the EXO side. More info here: https://blogs.technet.microsoft.com/exchange/2018/06/25/resource-based-throttling-and-prioritization-in-exchange-online-migrations/

Not applicable
Thanks for sharing. This is really helpful.
Copper Contributor

Very useful. Thanks!

Microsoft

Hi, my name is Mirela Buruiana and I am the main author of this blog. Reply to this comment if you want me to get notified about your question /comment.

Microsoft

Hello, are you aware of documentation which would show that migration tool does migrations 2 way and can hard delete emails with no possibility of recovery from exchange online ( we are speaking about IMAP migration) ?

Microsoft

Hi @Kris_Gatchev , not sure what do you mean by "2 way"? This would be an "one way" type of migration as explained below. And I don't know if we have an official documentation that specifically describes this but I am assuring the information below is correct. 
During an IMAP migration we have a synchronization process between source and target, the changes are unidirectional, coming from on-premises IMAP server --> to Exchange Online. If during an IMAP migration, the end-users or admins delete content from source IMAP mailbox / server , the deletions will be synced to O365 as long as the synchronization process runs (you can see it with Get-SyncRequest -Mailbox <user>). The Synchronization is active as long as the migration is ongoing (migration batch is not deleted or the Sync Request is not being removed).  All the changes reported in source are synced to EXO and you can see it with Get-SyncRequest -Mailbox <user> | Get-SyncRequestStatistics -IncludeReport or from GUI.
If data has been inadvertently deleted from the source, the deletions are processed during initial or incremental sync as expected and MRS Sync Request is hard-deleting it from target as well. Engineering team will in the future address this, not to hard delete anymore the data deleted on source so that customers can recover the data from Exchange Online Mailbox, even if this is gone forever from source. For the time being, customers should try to recover it from the source side, where the original content has been initially deleted from.
Exchange Online MRS Sync Request only replicates the changes (deletions / additions / folder hierarchy changes etc) and it is not responsible for the (un)intended deletions which happened on source.

Copper Contributor

Can you tell me if the error: fatal error: QuotaExceededException has occurred" means we have overloaded the Office 365 mailbox? Suggested remedy? I assume online deleting of emails is the only option?

 

Microsoft

Assuming that the error complains about Mailbox Quota exceeded, yes, this means you reached the allocated space for the cloud mailbox (example 50gb/ 100gb). You can check Get-MailboxStatistics and Get-MailboxFolderStatistics | ft foldersize, targetquota,folderpath in EXO powershell. I am on my phone so apologies if I have typos in the cmdlets. Solution would be to reduce the size of the mailbox (start with deleting junk or duplicate data) or depending on the mailbox plan assigned, you can increase quota up to 100gb or have an archive mailbox to the primary mailbox. If you have migrated from Gmail, you might have duplicate data in the Archive folder (which would be equivalent of All Mail in Gmail). 

Copper Contributor

this is all it says: " fatal error: QuotaExceededException has occurred", it does not specify what mailbox but I assume it is the destination mailbox on Office. I have gone into that users account and enabled the archive mailbox in Office365, it seems like the easiest and quickest way to clean up the mailbox and create more space. I have 6 of 8 mailboxes synced, with 1 still syncing 20hrs later and the 1 with the failed error. After enabling the Archive mailbox on Office, I have restarted the migration batch and both say syncing. I am going to hope they are synced by tomorrow. If there is a case where a mailbox just won't complete sync, what are the ramifications of just changing MX records and redirecting client Outlooks apps to the new server? Would I just be losing older emails since it syncs from recent to old? How long should I wait for sync to complete successfully?  Thanks for your help !!!!!!!!!!

 

Copper Contributor
 
 
I am currently migrating to Office365 using IMAP, but Im getting the following error on some of the mail accounts., any idea's?
 
"3/03/2020 2:40:53 AM [ME2P282MB0498] Fatal error ImapBadResponseException has occurred."
Microsoft
@TheOnePom That means the source IMAP server sent a bad response to Office 365 Exchange Online servers when trying to do some regular IMAP commands or stuff needed for migration (like LIST, SELECT folders from the source IMAP mailbox). You can also run Get-ImapFolders.ps1 Script from GitHub and see if any issues there when listing the folders in the source imap mailbox.
If you want to find out why/ where the IMAP server sent a bad response, you can gather the Sync Request Statistics with IncludeReport and DiagnosticInfo verbose from Exchange Online PowerShell and check the full details of the error (especially DataContext in report.failures)
 
start-transcript
$syncstats = Get-SyncRequestStatistics user@domain.com -IncludeReport -DiagnosticInfo verbose
$syncstats.Report.Failures | group failuretype
$syncstats.Report.Failures | select -last 2
$syncstats.Report.InternalFailures
$syncstats.DiagnosticInfo
$syncstats.Report.Entries | % {[string] $_}
stop-transcript
Microsoft

@sharpshooter77 

Do you mean the Incremental Sync won't complete or Initial sync won't complete in an IMAP migration?
When the IMAP migration users are Synced (initial sync is done),  you would change the MX record to point to the cloud and emails will be getting delivered to the cloud mailbox. As long as you have a sync request going (get-syncrequest -mailbox <user>), this will sync every 24h the newest emails (incremental sync) and you can check last successful sync timestamp on the sync request and the cloud mailbox contents to see if changes are synced. You would delete the batch when all emails are getting delivered to O365 mailboxes and there are no more recent emails that should be synced to cloud during incremental sync (typically 72h after you changed MX, as per https://docs.microsoft.com/en-gb/office365/enterprise/powershell/use-powershell-to-perform-an-imap-m... 
 
So, if incremental sync won't complete, you will be missing in cloud mailbox the newest emails that are in the onprem mailbox.
If initial sync won't complete, then this could be entire onprem mailbox content missing from cloud mailbox.
 
Regarding Outlook clients, this depends on the primary smtp address domain that they should be configured on. That domain must point to autodiscover.outlook.com (cloud) for the cloud mailboxes so that Outlook can discover that the mailbox is hosted in O365.
Copper Contributor

Hi we are migrating from Zoho to Office 365, All the users migrated flawless with one exception, we have not been able to solve this error:

 

Error: TooManyTransientFailureRetriesPermanentException/ImapCouldNotParseResponseException/ImapUnexpectedTokenFormatException: Error: The job encountered too many transient failures ‎(61)‎ and is quitting. The most common failure is ImapCouldNotParseResponseException/ImapUnexpectedTokenFormatException with the hit count 60. --> Imap server sent a response that we could not understand. Command: ‎'WIR00006 LIST "" "%" ‎'. Response: ‎'* LIST ‎(\HasNoChildren)‎ "/" ""‎' >>‎'ALQUILER DE EQUIPOS"" * LIST ‎(\HasNoChildren)‎ "/" ""HOUSE CENTER"" * LIST ‎(\HasChildren)‎ "/" "[Gmail]" WIR00006 OK Success ‎'. --> Unexpected response format - expected ‎'CR‎', actual ‎'A‎'.

Microsoft

@compumax , this looks like the source IMAP server giving an unexpected answer. I think you have an actual folder named: ' >>‎ 'ALQUILER DE EQUIPOS which looks weird, try to check / rename that folder and see if the error still complains about it or consult with the vendor of that server.

Copper Contributor

@Mirela_Buru We solved by deleting the IMAP folders on the source IMAP server. The migration finally suceeded. Thank you

Copper Contributor

Hi! We are migrating from Zimbra to Office 365, an error often occurs in log "Fatal error ImapBadResponseException has occurred."
On the portal say "IMAP server send BAD response to FetchCommand. Response code: ", message: 'parse error: malformed sequence' 

Where is error?

 

Copper Contributor

Hi,

 

I'm doing a migration from Gmail to O365. Created IMAP migration points, but for some users and after a few days the sync fails with the following error: "Transient error RemoteNetworkErrorTransientException has occurred. The system will retry". It will retry for 60 times and then it stops the sync and no resume on the next day. This error happen often on the users with biggest mailboxes, but mailbox size does not reach 50GB, the biggest one in O365 is 33GB. For me this error network error is a bit strange, because the migration point is a connection between cloud services.

Also, I do not understand why, but mailbox size in O365 is doubled when compared with Gmail, I've noticed there is mail duplication on O365 archive folder, and this could explian the double size. How to avoid the fill of archive on the migration?

Microsoft

@vova4ka , the error is coming from the source IMAP server (Zimbra). You need to get the full error and see if you can understand what's the problem there on the the IMAP server, or check with the vendor the specific error. I wrote above how to get the failures from EXO PS, example:
start-transcript
$syncstats = Get-SyncRequestStatistics <your user> -IncludeReport -DiagnosticInfo verbose
$syncstats.Report.Failures | select -Last 2

$syncstats.DiagnosticInfo

stop-transcript

Microsoft

@ranlp , you should also take the Full Transient Error and see if we get more details, see the transcript that I put above.

For the email duplication, yes, is probably because of the ALL MAIL folder in GMAIL (that it is like a copy of your all Gmail mails) which is being copied to O365 Mailbox in the Archive Folder. You can probably hide the ALL MAIL from IMAP protocol on Gmail settings or you can exclude the folder when you create the Migration Batch in O365 EAC. Also, IIRC, if an email in Gmail has multiple labels, it will likely be copied to each folder tagged for. Then there were emails without any label in Gmail, for example Sent Items where the item is in that folder but not having a label for Sent Items, it will be copied in O365 both in Sent Items and Archive Folder.

 

Copper Contributor

@Mirela_Buru I only see one error (RemoteNetworkErrorTransientException), here is the info from LastFailure (I hope this can bring a light):

FailureType : TooManyTransientFailureRetriesPermanentException
FailureHash : af07
FailureCode : -2146233088
MapiLowLevelError : 0
FailureSide :
FailureSideInt : 0
ExceptionTypes : {TooManyTransientFailureRetries, MRS, MRSPermanent, Exchange...}
ExceptionTypesInt : {633, 10, 12, 1...}
WorkItem : IncrementalSync
Message : The job encountered too many transient failures (97) and is quitting. The most common failure is RemoteNetworkErrorTransientException/IOException/SocketException with the
hit count 60. --> We were unable to reach your email provider. Please check your server settings, or try again later. --> Unable to read data from the transport
connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected
host has failed to respond. --> A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed
because connected host has failed to respond
MessageData :
DataContext :
DataContextData :
StackTrace :
InnerException : RemoteNetworkErrorTransientException: We were unable to reach your email provider. Please check your server settings, or try again later. --> Unable to read data from the
transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because
connected host has failed to respond. --> A connection attempt failed because the connected party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond
UnknownElements :
UnknownAttributes :
XmlSchemaType :

Microsoft

Hello @ranlp,

It looks like a networking issue between our servers and the source. If it continues to happen, please open a support case so we can have a look.

You can check all the errors using the following commands:

 

$syncstats = Get-SyncRequestStatistics <your user> -IncludeReport -DiagnosticInfo verbose

$syncstats.Report.Failures | group FailureType | ft -AutoSize Count, Name

$syncstats.Report.Failures | where {$_.FailureType -like "Replace with failure name from above"} | fl

In regards to the doubled size, this is expected since the All Mail folder will be synced into the Archive folder from Cloud.
If you do not want to export that folder, you can exclude it from the IMAP sync from the migration or from GMail by hiding it from Settings->Labels tab-> Uncheck the box "Show in Imap"

 

Copper Contributor

I have a confusion, does a Parent folder ( not a sub folder of Inbox ), will it migrate in IMAP migration ? 

Microsoft

You mean at the same level with Inbox? Yes, it will migrate any email folder that is not hidden from IMAP or not excluded at the migration batch level. If you see it in Outlook IMAP account or in ExRCA.com IMAP test, we will migrate it.

Copper Contributor

Hello @Mirela_Buru,
We are currently migrating to Office365 using IMAP, but we get the following error on some of the mail accounts.

Imap server sent BAD response to FetchCommand. Response code: '',
message: 'Error in IMAP command UID FETCH: Invalid uidset'.

We understand that it is due to a message or folder that cannot be synchronized.
How could we identify the message or the folder?

Microsoft

@jnavarro1983 , check the full failure, especially DataContext inside the failure. Additionally, diagnosticinfo can reveal more stuff.

In EXO PS:
$syncstats = Get-SyncRequestStatistics <your user> -IncludeReport -DiagnosticInfo verbose
$syncstats.Report.Failures | select -Last 2

#Check Failures especially DataContext

$syncstats.DiagnosticInfo

Copper Contributor

@Mirela_Buru,  here is the info from $syncstats.Report.Failures | select -last 2.

We would like to know how to identify the message or folder with problems.

 

Timestamp : 10/9/2020 14:10:11

FailureType : ImapBadResponseException

FailureHash : 831e

FailureCode : -2146233088

MapiLowLevelError : 0

FailureSide : Source

FailureSideInt : 1

ExceptionTypes : {Exchange}

ExceptionTypesInt : {1}

WorkItem : CrawlAndCopyFolder

Message : Imap server sent BAD response to FetchCommand. Response code: '', message: 'Error in IMAP command UID FETCH: Invalid uidset'.

MessageData :

DataContext : --------

Operation: ImapSourceMailbox.CopyBatchOfMessages

EntryIDs: [count:1, [len=72,

data=563D313B503D494D41503B553D303B55563D313132393832303335353B4C463D45383031414632453130373346313842353638363045333741454331444132424644344642303832]]

--------

Operation: IMailbox.ExportMessages

REDACTED-4550cf30bb6928358f697b4008d2fc90@frsf.utn.edu.ar (IMAP)

OperationSide: Source

Flags: ReportBadItems

PropTags: (null)

--------

MessageCount: 1

EntryIDs: [count:1, [len=72,

data=563D313B503D494D41503B553D303B55563D313132393832303335353B4C463D45383031414632453130373346313842353638363045333741454331444132424644344642303832]]

--------

Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]

--------

Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]

--------

>>>> Current WorkItem: CrawlAndCopyFolder, Scheduled WorkItems: CrawlAndCopyFolder(P:0,R:1,S:0,C:10268); CrawlAndCopyFolder(P:1,R:0,S:0,C:10427);

CrawlAndCopyFolder(P:1,R:0,S:0,C:10360); CrawlAndCopyFolder(P:0,R:0,S:0,C:10395); CrawlAndCopyFolder(P:0,R:0,S:0,C:10369); CrawlAndCopyFolder(P:0,R:0,S:0,C:10448);

CrawlAndCopyFolder(P:2,R:0,S:0,C:9784); CrawlAndCopyFolder(P:2,R:0,S:0,C:10470,Cnt=20); DoPeriodicCheck(P:0,R:0,S:0,C:0); CrawlAndCopyFolder(P:4,R:0,S:1,C:10496,Cnt=6)

--------

Operation: ImapSourceMailbox.CopyBatchOfMessages

EntryIDs: [count:1, [len=72,

data=563D313B503D494D41503B553D303B55563D313132393832303335353B4C463D45383031414632453130373346313842353638363045333741454331444132424644344642303832]]

--------

Operation: IMailbox.ExportMessages

REDACTED-4550cf30bb6928358f697b4008d2fc90@frsf.utn.edu.ar (IMAP)

OperationSide: Source

Flags: ReportBadItems

PropTags: (null)

--------

MessageCount: 1

EntryIDs: [count:1, [len=72,

data=563D313B503D494D41503B553D303B55563D313132393832303335353B4C463D45383031414632453130373346313842353638363045333741454331444132424644344642303832]]

--------

Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]

--------

Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]

--------

>>>> Current WorkItem: CrawlAndCopyFolder, Scheduled WorkItems: CrawlAndCopyFolder(P:0,R:1,S:0,C:10268); CrawlAndCopyFolder(P:1,R:0,S:0,C:10427);

CrawlAndCopyFolder(P:1,R:0,S:0,C:10360); CrawlAndCopyFolder(P:0,R:0,S:0,C:10395); CrawlAndCopyFolder(P:0,R:0,S:0,C:10369); CrawlAndCopyFolder(P:0,R:0,S:0,C:10448);

CrawlAndCopyFolder(P:2,R:0,S:0,C:9784); CrawlAndCopyFolder(P:2,R:0,S:0,C:10470,Cnt=20); DoPeriodicCheck(P:0,R:0,S:0,C:0); CrawlAndCopyFolder(P:4,R:0,S:1,C:10496,Cnt=6)

--------

Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]

--------

Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]

--------

>>>> Current WorkItem: CrawlAndCopyFolder, Scheduled WorkItems: CrawlAndCopyFolder(P:0,R:1,S:0,C:10268); CrawlAndCopyFolder(P:1,R:0,S:0,C:10427);

CrawlAndCopyFolder(P:1,R:0,S:0,C:10360); CrawlAndCopyFolder(P:0,R:0,S:0,C:10395); CrawlAndCopyFolder(P:0,R:0,S:0,C:10369); CrawlAndCopyFolder(P:0,R:0,S:0,C:10448);

CrawlAndCopyFolder(P:2,R:0,S:0,C:9784); CrawlAndCopyFolder(P:2,R:0,S:0,C:10470,Cnt=20); DoPeriodicCheck(P:0,R:0,S:0,C:0); CrawlAndCopyFolder(P:4,R:0,S:1,C:10496,Cnt=6)

DataContextData :

StackTrace : at Microsoft.Exchange.Connections.Imap.ImapConnection.ProcessResponsesInternal(String commandType, IEnumerable`1 responses, String tag, Boolean isLogin, Boolean

isAuth, IImapBulkItemProcessor itemProcessor, Int32 estimatedMessageCount, Int32 estimatedFolderCount, FolderStatistics& folder, IList`1& messages, IList`1& folders,

SearchData& searchResults, ExtendedSearchData& esearchResults, ItemCopyResult& copyResult, ItemCreateResult& createResult, Boolean& receivedUpdate)

at Microsoft.Exchange.Connections.Imap.ImapConnection.ProcessMessagesResponses(String commandType, IEnumerable`1 responses, String tag, IImapBulkItemProcessor

itemProcessor, Int32 estimatedMessageCount, IList`1& messages)

at Microsoft.Exchange.Connections.Imap.ImapConnection.GetItems(MessageSet messages, FetchAttFlags attributes, Boolean useUid, IEnumerable`1 sectionIds, Nullable`1

changedSince, Boolean vanished, IImapBulkItemProcessor itemProcessor)

at Microsoft.Exchange.MailboxReplicationService.ImapSourceMailbox.<>c__DisplayClass82_0.<ProcessMimeStreams>b__0()

at Microsoft.Exchange.MailboxReplicationService.ImapSourceMailbox.AcquireThrottlingDuration(IThrottleable throttleable, Action action, Boolean

trackImplicitlyThrottlingDuration)

at Microsoft.Exchange.MailboxReplicationService.ImapSourceMailbox.ProcessMimeStreams(List`1 messages, IFolderProxy folderProxy, BodySectionId bodySection, List`1

badItems, Boolean reportBadItems)

at Microsoft.Exchange.MailboxReplicationService.ImapSourceMailbox.CopyBatchOfMessages(List`1 messages, IFxProxyPool proxyPool, List`1 badItems, Boolean

reportBadItems)

at Microsoft.Exchange.MailboxReplicationService.MailboxWrapper.Microsoft.Exchange.MailboxReplicationService.IMailbox.ExportMessages(List`1 messages, IFxProxyPool

destProxies, ExportMessagesFlags flags, PropTag[] propsToCopyExplicitly, PropTag[] excludeProps, List`1 badItems)

at Microsoft.Exchange.MailboxReplicationService.MessageExporter.<>c__DisplayClass6_1.<ExportMessageBatch>b__0()

at Microsoft.Exchange.MailboxReplicationService.TransmitterWithHistory.TrackAction(Object pipelineStart, Action action)

at Microsoft.Exchange.MailboxReplicationService.MessageExporter.ExportMessageBatch(IMailbox mailbox, List`1 messages, Func`2 getProxyPool, ExportMessagesFlags

flags, PropTag[] propsToCopyExplicitly, PropTag[] excludeProps, EntryIdMap`1 transientFailuresMap, ItemPropertiesBase perBatchSettings, FailureRec& failure,

EntryIdMap`1& badItemMap)

at Microsoft.Exchange.MailboxReplicationService.MessageExporter.ExportMessagesWithBadItemDetection(IMailbox mailbox, List`1 messages, Func`2 getProxyPool,

ExportMessagesFlags flags, PropTag[] propsToCopyExplicitly, PropTag[] excludeProps, TestIntegration testIntegration, List`1 badItems, EntryIdMap`1

transientFailuresMap, OperationFlags operation)

at Microsoft.Exchange.MailboxReplicationService.MailboxMerger.CopySingleMessageBatch(FolderCopyContext copyContext)

at Microsoft.Exchange.MailboxReplicationService.MailboxMerger.CopyMessageBatch(FolderContentsMapper mapper, List`1 itemsToCopy, FolderMapping fm, Boolean

skipRequestRefresh)

at Microsoft.Exchange.MailboxReplicationService.MailboxMerger.MergeFolderContentsPaged(FolderMapping folder, ISourceFolder srcFolder, FolderContentsCrawler

sourceFolderCrawler, IDestinationFolder destFolder, TimeSpan maxOperationDuration, CopyDataFlags copyFlags)

at Microsoft.Exchange.MailboxReplicationService.MailboxMerger.CopyFolderPropertiesAndContents(FolderMapping folder, ISourceFolder srcFolder, FolderContentsCrawler

sourceFolderCrawler, IDestinationFolder destFolder, Boolean copyProperties, TimeSpan maxOperationDuration, CopyDataFlags copyFlags)

at Microsoft.Exchange.MailboxReplicationService.SyncBaseJob.CopyFolderPropertiesAndContents(FolderMapping folder, ISourceFolder srcFolder, FolderContentsCrawler

sourceFolderCrawler, Boolean shouldCopyProperties, TimeSpan maxOperationDuration, CopyDataFlags copyFlags)

at Microsoft.Exchange.MailboxReplicationService.SyncBaseJob.CrawlAndCopyFolder(FolderCrawlingInfo folderCrawlInfo)

at Microsoft.Exchange.MailboxReplicationService.Job.ExecuteWorkItemCallback(WorkItem workItem)

InnerException :

UnknownElements :

UnknownAttributes :

XmlSchemaType :

 

Microsoft

@jnavarro1983 , the folder seems INBOX as per this entry below:
Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]

But since we are not able to fetch the message, we cannot really know / get an actual identifier of it, we just know the invalid UID is 0

V=1;P=IMAP;U=0;

 

Copper Contributor

@Mirela_Buru How can I identify that message? Is there any command, tool or whatever to determine which is the exact message that generates the error?

Microsoft

@jnavarro1983 , since we cannot fetch it, we are not able to identify it. I can just tell you that it has am invalid uid, that is 0. But is this error failing your migration or the message is just marked as bad item in $syncstats.report.baditems ?

Copper Contributor

Hi @Mirela_Buru 

 

I have some doubts:

 

1-Archived e-mails will be migrate from IMAP to O365?

2-I have plans to migrate the accounts during the weekend. Let's suppose I have an account with 25gb and it takes several days to complete, It's not clear to me if I can change the MX record before the last account completes the synchronization, it seems that all the company will be impacted in case one account is delayed. I don't find a clear documentation for such situation.

3-Is there a way that I can re-sync the accounts in case someone creates a folder after the synchronizazion is done, therefore that folder was not sysynchronized? what will happen to the e-mails received during the sync?

4-How can I increase the limit of 35MB of each e-mail?

 

Thanks in advance

Microsoft

@AndreSantos 

1. In the Gmail definition of "archived", yes, archived mail (mail with no labels) is migrated to the "Archive" folder in Exchange Online. Otherwise, not sure what specifically you mean by "archived".

2. You should wait until all users are ready to start using their new mailboxes before you change the MX record. Typically this would be after the initial sync has completed for each of the mailboxes involved. Otherwise, they may only have access to their old content on the source side until the initial synchronization completes.

3. As long as the migration batch is in place, it will continue doing scheduled incremental syncs (usually daily). This means any changes from the source will be replicated to the Exchange Online mailbox.  After you do cutover the MX record, you'd usually leave the batch around for a few days so that any mail delivered up to the point of the MX cutover can be synchronized to O365 before you delete the batch and disable the sync.

4. After provisioning the mailboxes, you can use Remote PowerShell to connect to Exchange Online. You can then run: Set-Mailbox user@contoso.com -MaxReceiveSize 150MB to increase the maximum message size up to 150MB.

 

Copper Contributor

Hi @Mirela_Buru 

We have another problem doing a migration from an on-premises IMAP server to Exchange Online, and we are getting several errors from the batches being migrated:

1st Error:
Error: UserAlreadyBeingMigratedException: The user ‎'*** Email address is removed for privacy ***‎' already has a pending request. Please remove the existing request and resume the current batch or start a new batch for this user. --> Name must be unique per owning mailbox. There‎'s already a request with the name ‎'Simple Migration Merge Request‎' owned by mailbox ‎'AMPR01A009.PROD.OUTLOOK.COM/Microsoft Exchange Hosted Organizations/xxxxx.onmicrosoft.com/**email account removed**‎'

Tried with this resolution:
PS C:\WINDOWS\system32> Get-MoveRequest -Identity '***email removed****'

Couldn't find a move request that corresponds to the specified identity '***email removed****'.

+ CategoryInfo : NotSpecified: (:) [Get-MoveRequest], ManagementObjectNotFoundException

+ FullyQualifiedErrorId : [Server=SN6PR01MB5168,RequestId=3963635d-d8ed-4793-bc15-dd495bcc5440,TimeStamp=10/5/2020

4:49:43 PM] [FailureCategory=Cmdlet-ManagementObjectNotFoundException] 595D492,Microsoft.Exchange.Management.Migr

ation.MailboxReplication.MoveRequest.GetMoveRequest

+ PSComputerName : outlook.office365.com

If we try with:

PS C:\WINDOWS\system32> $syncstats = Get-SyncRequestStatistics ***email removed**** -IncludeReport -DiagnosticInfo verbose

PS C:\WINDOWS\system32> $syncstats.DiagnosticInfo

<Job>

<SkippedItemCounts Corrupt="0" Missing="0" Large="0" Other="0" />

<context>

<C K="BatchName" V="MigrationService:LOTE TIC + DEPARTAMENTALES" />

<C K="DatabaseGuid" V="1d34703a-234c-4329-a0dd-27b4a6d175c6" />

<C K="DatabaseName" V="NAMPR01DG361-db007" />

<C K="EmailAddress" V="***email removed****" />

<C K="FatalFailureRetryCount" V="0" />

<C K="JobFeatures" V="BinaryJobFeatures, UseRequestGuidForMoveReport" />

<C K="MailboxGuid" V="777ee8f4-b0f4-4a97-abfa-0f45562a202f" />

<C K="OrganizationName" V="xxxxxxx.onmicrosoft.com" />

<C K="RecipientTypeDetails" V="UserMailbox" />

<C K="RemoteServerName" V="xxxxxxxxxxx" />

<C K="RequestCreatedTimestamp" V="10/5/2020 12:57:49 PM" />

<C K="RequestFlags" V="CrossOrg, Pull, Suspend" />

<C K="RequestGuid" V="fbeeee80-228a-4df2-a29e-b4d8c393ef34" />

<C K="RequestInternalFlags" V="SkipKnownCorruptions,UseDataConsistencyScore" />

<C K="RequestJob" V="lem\IMAP:***email removed****" />

<C K="RequestLastSuccessfulSyncTimestamp" V="10/5/2020 1:11:09 PM" />

<C K="RequestState" V="Canceled" />

<C K="RequestStatus" V="Suspended" />

<C K="RequestStyle" V="CrossOrg" />

<C K="RequestType" V="Sync" />

<C K="RequestWorkloadType" V="Onboarding_IMAP" />

<C K="SyncProtocol" V="Imap" />

<C K="UserRamp" V="80" />

...

We get that the email is associated with an old batch but we cannot delete or remove that old batch, tried creating a new batch and removing it from there but it didn't affect.


2nd Error:
Error: MigrationPermanentException: El nombre debe ser diferente para cada buzón. Ya existe una solicitud con el nombre ‎'IMAP:***email removed****‎' que pertenece al buzón ‎'NAMPR01A009.PROD.OUTLOOK.COM/Microsoft Exchange Hosted Organizations/xxxxxxxxx.onmicrosoft.com/NAME‎'. --> El nombre debe ser diferente para cada buzón. Ya existe una solicitud con el nombre ‎'IMAP:***email removed****‎' que pertenece al buzón ‎'NAMPR01A009.PROD.OUTLOOK.COM/Microsoft Exchange Hosted Organizations/xxxxxxxxx.onmicrosoft.com/NAME‎'.

For the 2nd error we don't get any clear reason on why it's failing besides the log information.

Please advise,

 

Microsoft

@jnavarro1983 - IMAP migrations are done with a SyncRequest. Try looking for the existing request with:
Get-SyncRequest -Mailbox <mailbox>
and removing it with
Remove-SyncRequest -Identity <identity from Get-SyncRequest>

Microsoft

@jnavarro1983 , for the 1st error, this looks like a Cutover / Staged Migration (Merge request) for that user. Did you previously migrate using Staged/Cutover (Outlook Anywhere)?  You can check get-migrationuser to list all migration users and get-migrationbatch to list them all. If there is no migration user / batch, then please open a support case with us to investigate that.

Copper Contributor

@bradhugh @Mirela_Buru 

Here is the info from command,
Get-SyncRequest -Mailbox ***Email address is removed for privacy***

RunspaceId : 3ac7ef3c-5085-4403-8c79-8cdf9f6d4c33
Mailbox : NAMPR01A009.PROD.OUTLOOK.COM/Microsoft Exchange Hosted
Organizations/***Organization is removed for privacy***
EmailAddress : ***Email address is removed for privacy***
SyncProtocol : Imap
WorkloadType : Onboarding_IMAP
RemoteServerName : ***
RemoteServerPort : 993
SmtpServerName :
SmtpServerPort : 0
AuthenticationMethod : Basic
SecurityMechanism : Ssl
AcceptUntrustedCertificates : False
DisplayName :
OAuthProvider : None
LastSuccessfulSyncTimestamp :
LastUpdateTimestamp : 05/10/2020 12:57:50
LastFailureTimestamp :
InitialSyncCompletionTimestamp :
FailureType :
TargetRootFolder :
InternalFlags : UseDataConsistencyScore
DoNotLeaveOnServer : False
RemoteSendEnabled : False
IsQuarantined : False
IsOauthSync : False
UserName :
SmtpUserName :
Name : IMAP:***Email address is removed for privacy***
RequestGuid : fbeeee80-228a-4df2-a29e-b4d8c393ef34
RequestQueue : NAMPR01DG361-db007
ExchangeGuid : 00000000-0000-0000-0000-000000000000
Flags : CrossOrg, Pull
BatchName : MigrationService:LOTE TIC + DEPARTAMENTALES
Status : Queued
Protect : False
Suspend : False
Direction : Pull
RequestStyle : CrossOrg
OrganizationId : NAMPR01A009.PROD.OUTLOOK.COM/Microsoft Exchange Hosted
Organizations/****
SourceMailboxSize : 0
MailboxRopCount :
WhenChanged : 05/10/2020 9:57:50
WhenCreated : 05/10/2020 9:57:49
WhenChangedUTC : 05/10/2020 12:57:50
WhenCreatedUTC : 05/10/2020 12:57:49
Identity : NAMPR01A009.PROD.OUTLOOK.COM/Microsoft Exchange Hosted
Organizations/***
IsValid : True
ObjectState : New

Then removing with Remove-SyncRequest -Identity <identity from Get-SyncRequest>
but when we execute command Get-SyncRequest -Mailbox *** Email address is removed for privacy *** again we get the same result with status: Queued

Copper Contributor

Hello to all Guys,

a very nice KB, congratulations!.

 

I need some clarification about IMAP migration from a Server Exchange. My situation is that:

  • i have already setup an Hybrid Exchange from one organization ( with all users migrated but the server it's present yet as recommend from MS);
  • I need to setup another migration from another Exchange Server, where this server is in another Forest, another location, antoher AD Domain, but same Azure Ad connect;

So, Microsoft tell me that i cannot setup multi hybrid exchange configurations for one single same tenant, in this case, i'm implementing migration via IMAP, but i read here that isnot recommend, so, which the best pratices in this particular situation?.

 

Thanks for your time and your work!.

Nicolas

Microsoft

Multi-Exchange Hybrid Deployment with single tenant has been supported for some years now, please check our official doc: https://docs.microsoft.com/en-us/exchange/hybrid-deployment/hybrid-with-multiple-forests 

Probably MS confused to the new Multi-Tenant > Single On-Premises Hybrid topology: https://support.microsoft.com/en-my/help/4583653/september-2020-update-to-exchange-hybrid-configurat... 

Copper Contributor

@Mirela_Buru thanks for your reply.

 

And, in the case i can't install HCW for other causes, why MS not recommend to use IMAP?

 

Thanks

Microsoft

@PasquaN , because IMAP from Exchange is a poor migration method compared to Hybrid Exchange or even Cutover Migration (IMAP only copies email data), you can check this table from here to make an idea on what is being migrated vs what is not in each method: https://docs.microsoft.com/en-us/fasttrack/data-migration#source-environments 

Also, with IMAP there is downtime, Outlook and EAS profile recreation. And IMAP is more prone to errors than MRS migration.

Copper Contributor

Thanks @Mirela_Buru 

Microsoft

You're welcome, Nicolas!

Copper Contributor

Hello, 

i'm trying imap migration, until today i have never got problems with this operation, but from about 3 days i obtain these errors: 

 

UserAlreadyBeingMigratedException: The user 'asd@asd.it' already has a pending request. Please remove the existing request and resume the current batch or start a new batch for this user. --> Name must be unique per owning mailbox. There's already a request with the name 'IMAP:asd@asd.it' owned by mailbox 'EURPR08A001.prod.outlook.com/Microsoft Exchange Hosted Organizations

 

i 've already follow all your steps above, but nothing. 

 

And these is the diagnosticinfo and includereport from other error:

 

by Get-MigrationBatch: SyncedWithErrors
by Get-SyncRequest: Status: Failed
by Get-SyncRequestStatistics <smtp-identity>

WorkItem : CrawlAndCopyFolder
Message : Imap server sent BAD response to FetchCommand. Response code: '', message: 'The specified message set is invalid.'.
MessageData :
DataContext : --------
Operation: ImapSourceMailbox.CopyBatchOfMessages
EntryIDs: [count:8, [len=68, data=563D313B503D494D41503B553D31333439363B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333439353B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333439333B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333438343B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333437373B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333437363B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333437353B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=64,
data=563D313B503D494D41503B553D303B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]]
--------
Operation: IMailbox.ExportMessages
REDACTED-6d40a82f6c2d7609f6850ef4583917@XXXXXXXXXXXXXXXXXX.com (IMAP)
OperationSide: Source
Flags: ReportBadItems
PropTags: (null)
--------
MessageCount: 8
EntryIDs: [count:4, [len=68, data=563D313B503D494D41503B553D31333439363B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333439353B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333439333B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333438343B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]]
--------
Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]
--------
Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]
--------
>>>> Current WorkItem: CrawlAndCopyFolder, Scheduled WorkItems: CrawlAndCopyFolder(P:19,R:0,S:0,C:10231); CrawlAndCopyFolder(P:74,R:0,S:0,C:15785); CrawlAndCopyFolder(P:0,R:0,S:0,C:10944); CrawlAndCopyFolder(P:40,R:0,S:0,C:10898);
CrawlAndCopyFolder(P:2047,R:0,S:0,C:10564); CrawlAndCopyFolder(P:2003,R:0,S:0,C:11983); CrawlAndCopyFolder(P:2249,R:0,S:0,C:10264); CrawlAndCopyFolder(P:1927,R:0,S:0,C:25776); CrawlAndCopyFolder(P:4834,R:0,S:0,C:18538);
CrawlAndCopyFolder(P:2786,R:0,S:0,C:11300)
--------
Operation: ImapSourceMailbox.CopyBatchOfMessages
EntryIDs: [count:8, [len=68, data=563D313B503D494D41503B553D31333439363B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333439353B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333439333B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333438343B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333437373B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333437363B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333437353B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=64,
data=563D313B503D494D41503B553D303B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]]
--------
Operation: IMailbox.ExportMessages
REDACTED-6d40a82f6c2d7609f6850ef4583917@XXXXXXXXXXXXXXXXXX.com (IMAP)
OperationSide: Source
Flags: ReportBadItems
PropTags: (null)
--------
MessageCount: 8
EntryIDs: [count:4, [len=68, data=563D313B503D494D41503B553D31333439363B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333439353B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333439333B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]; [len=68,
data=563D313B503D494D41503B553D31333438343B55563D31343B4C463D41324446363342394431364331414639433344373338384639444136394431423139353645383144]]
--------
Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]
--------
Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]
--------
>>>> Current WorkItem: CrawlAndCopyFolder, Scheduled WorkItems: CrawlAndCopyFolder(P:19,R:0,S:0,C:10231); CrawlAndCopyFolder(P:74,R:0,S:0,C:15785); CrawlAndCopyFolder(P:0,R:0,S:0,C:10944); CrawlAndCopyFolder(P:40,R:0,S:0,C:10898);
CrawlAndCopyFolder(P:2047,R:0,S:0,C:10564); CrawlAndCopyFolder(P:2003,R:0,S:0,C:11983); CrawlAndCopyFolder(P:2249,R:0,S:0,C:10264); CrawlAndCopyFolder(P:1927,R:0,S:0,C:25776); CrawlAndCopyFolder(P:4834,R:0,S:0,C:18538);
CrawlAndCopyFolder(P:2786,R:0,S:0,C:11300)
--------
Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]
--------
Folder: type Generic, wkf Inbox, entryId [len=5, data=496E626F78], parentId [len=10, data=49706D53756274726565]
--------
>>>> Current WorkItem: CrawlAndCopyFolder, Scheduled WorkItems: CrawlAndCopyFolder(P:19,R:0,S:0,C:10231); CrawlAndCopyFolder(P:74,R:0,S:0,C:15785); CrawlAndCopyFolder(P:0,R:0,S:0,C:10944); CrawlAndCopyFolder(P:40,R:0,S:0,C:10898);
CrawlAndCopyFolder(P:2047,R:0,S:0,C:10564); CrawlAndCopyFolder(P:2003,R:0,S:0,C:11983); CrawlAndCopyFolder(P:2249,R:0,S:0,C:10264); CrawlAndCopyFolder(P:1927,R:0,S:0,C:25776); CrawlAndCopyFolder(P:4834,R:0,S:0,C:18538);
CrawlAndCopyFolder(P:2786,R:0,S:0,C:11300)
DataContextData :
StackTrace : at Microsoft.Exchang ..............

 

We aren't able to understand what is wrong in inbox like a message or folder structure.

Could you help us?.

Thanks

Microsoft

@PasquaN , for UserAlreadyBeingMigratedException, run the following:
Get-SyncRequest -Mailbox <user>


- If no Sync Request is returned but you get this error, open a support case with us.
- If a Sync Request is returned for that user and this is for IMAP Onboarding Migration please remove it with the following command
Get-SyncRequest -Mailbox <user> | Remove-SyncRequest
Check if the Sync Request was removed or still there:
Get-SyncRequest -Mailbox <user>

If still there, open a case with us. If disappeared, retry IMAP migration.

Microsoft

@PasquaN , for the second error ('The specified message set is invalid.'), this is almost similar to this one above https://techcommunity.microsoft.com/t5/exchange-team-blog/troubleshooting-imap-migrations-to-office-... (Error in IMAP command UID FETCH: Invalid uidset'.) Something on the Source IMAP server not allowing us to fetch the messages. Seems Inbox folder also here. Unfortunately I cannot tell more on the cause. You would need to check the IMAP server logs. 

Copper Contributor

@Mirela_Buru thanks very much!! :smile:

 

At this time the migration is going on. Just last question, for better undestarnd some parameters inside the report of user statistics.

this rows:

09/10/2020 15:31:15 [VE1PR08MB5566] Stage: CopyingMessages. Percent complete: 95.
09/10/2020 15:31:15 [VE1PR08MB5566] Copy progress: 84122/65445 messages, 35.54 GB (38,161,759,871 bytes)/6.173 GB (6,628,556,800 bytes), 0/0 folders completed.

 

What stand for? In the specific the "84122 messages" and "35.64GB".

 

Thanks a lot

Microsoft

@PasquaN , I think that in this specific case, you might have had a lot of reconnections / failures and since IMAP migrations don't have a sync state, it would recopy again the data when reconnected. Or maybe some shard mailbox merge. I don't know by heart but I don't see it a problem. 

Iron Contributor

 A good post but I don't see my error listed :sad:

 

I'm trying to migrate some users from a "mail.uk2.net"  IMAP service to Exchange Online.  All users have Microsoft Apps for Business licenses with the exception of myself.  As the global admin. I have the Teams Preview license  and also I have added  two Outlook migration roles, discussed in the docs.  

In preparation, I have added the MX and Txt records to Office 365 for the source host. Then I tried the migration wizard but this just errors with a whole list of possible causes then don't make sense. 

Thought I would have a go at  the PowerShell

 

Test-MigrationServerAvailability -Imap -RemoteServer  mail.uk2.net -Port 993 -Security Ssl
<#
RunspaceId      : d9718fa4-147e-49b6-a3f5-bf26de82e24e
Result          : Success
Message         :
SupportsCutover : False
ErrorDetail     :
TestedEndpoint  : mail.uk2.net
IsValid         : True
Identity        :
ObjectState     : New

New-MigrationEndpoint -IMAP -Name IMAPEndpoint  -RemoteServer  mail.uk2.net -Port 993 -Security Ssl

 New-MigrationBatch -Name IMAPBatch1 -SourceEndpoint IMAPEndpoint -CSVData ([System.IO.File]::ReadAllBytes("c:\dev\Clients\Myclient\admin\migratedusersonly.csv")) -AutoStart

Domain controller LNXP265A001DC06.GBRP265A001.PROD.OUTLOOK.COM is overloaded. Please try again later.

Anybody seen this "lovely" error?

Update: the error is actually temporary tho can reoccur at any time.  I have 'progressed' to the next error.  Both the user's password and the SSL connection seem fine.

 

MailboxEmailAddress       : firsname.surname@domain.co.uk
RecipientType             : Mailbox
SkippedItemCount          : 0
SyncedItemCount           : 0
MailboxGuid               : 6e8b8fae-b69d-4857-a455-?????????????
MailboxLegacyDN           : /o=ExchangeLabs/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn??????-firstname.lastname
RequestGuid               : 00000000-0000-0000-0000-000000000000
LastSuccessfulSyncTime    :
Status                    : Failed
StatusSummary             : Failed
MigrationType             : IMAP
State                     : Failed
Flags                     : None
WorkflowStep              : DataMigration
WorkflowStage             : Injection
TriggeredAction           : None
DataConsistencyScore      :
HasUnapprovedSkippedItems : 
ErrorSummary              : The username or password for this account is incorrect, or IMAP access is disabled. --> The username or password for this account is incorrect, or IMAP 
                            access is disabled. --> Imap server sent NO response to AuthenticateCommand. Response code: '', message
IsValid                   : True
ObjectState               : Unchanged

 

 

Microsoft

Hi @Daniel Westerdale , yes, the first error seems a transient stall on our side (Office 365 resource health throttling).

For the second error, check suggestions from Error #4 from this blog, it is an authentication issue.

Iron Contributor

@Mirela_Buru  

 

Thanks very much for your suggestion . All steps ( except CSV)  in #4 all worked for me . I am currently testing batches to see if this in the correct format for UK2.NET. Currently the batch I am testing has passed validation and just seems to stick with syncing 0 items . Update:  Thanks again,  for advice @Mirela_Buru . -the  issue was the user login  details for the source mailboxes in the csv file.

Co-Authors
Version history
Last update:
‎Nov 21 2023 03:34 PM
Updated by: