BUG ? exchange mailbox migration batch silently skipping mails when copying from source via imap

[RolandK50]

hello,

i'm currently testing office365/exchange-online as we perhaps want to migrate from our legacy mail server to exchange online.

i'm testing migration with the integrated migration solution.

the migration batch succeeds and tells 5947 Items synced and no items skipped - with data consistency score: perfect

after comparison of source and target, i found that there are 5 out of 2699 emails missing from my mailing-list folder after migration, i.e. they have not been synced and have silently been skipped.

this is reproducible, even if i freshly completely re-copy the folder from local archive to the imap folder on the legacy system. it's not a "visibilty bug", i tested with different email clients.

if i attach exchange target mailbox and imap source mailbox in thunderbird and copy folder with thunderbird, there is no message getting skipped.

this looks like a bug to me, causing mistrust into this migration tool.

unfortunately, i don't get imapsync unix commandline tool working because of oauth2 authentication.

any clue how this can be further debugged ?

regards
roland

[#2]

Does this come up with any pointers?
Get-MigrationUserStatistics email address removed for privacy reasons -IncludeSkippedItems -IncludeReport

[#3]

@Michel de Rooijnope

source (shell on unix mailserver inside migration2 mailbox) :

[root@mail pve-devel.mdir]# find . -type f |wc -l
2699

thunderbird, source mailbox attached mailbox via imap:

target:
thunderbird, target mailbox attached via imap:

osx mail, target mailbox attached via imap:

PS C:\Windows\system32> Get-MigrationUserStatistics email address removed for privacy reasons -IncludeSkippedItems -IncludeReport

Identity Batch Status Items Synced Items Skipped
-------- ----- ------ ------------ -------------
email address removed for privacy reasons mymigrationbatch1 Synced 2695 0

4 Mails silently skipped / missing

this is reproducible. and making me nervous.  i have 70 accounts and more then 1,5mio mails to migrate...

[#4]

so, apparently the problem is some deduplication feature.

apparently migrate batch is removing sort of duplicate elements

the problematic folder has mails from two differnt, but related mailing lists, where developers send emails to both sometimes - and these have been "deduplicated".

i'm curious though, why this happens - as this are different mails with different subjects

[#5]

it seems mails getting deduplicated by included message-id - different subject line or content does not seem to matter.