We've seen a few issues recently where members of DGs (Distribution Groups) in mixed-mode Exchange 200x and 5.5 seem to randomly and mysteriously disappear. We thought we'd share one known root-cause and also show you how to prevent the problem while doing your migration, if for whatever reason you are still running Exchange 5.5 =). One easy way to prevent this problem (and a few others) is to ALWAYS use the Exchange 2003 post SP1 cross-site mailbox migration wizard if you ever have to move mailboxes in a mixed-mode environment.
With that said, typically the problem occurs when you use a directory export/import to move mailboxes between Exchange 5.5 sites. For example, if you have two Exchange 5.5 sites Site S (Source) and Site D (Destination) and you would like to move all the mailboxes in Site S to Site D you might perform the following steps. (When we refer to a DL we mean a Distribution List on the Exchange 5.5 side and when we refer to DG we mean a Distribution Group on the Active Directory side)
- Extract all the DLs of the mailboxes you intend to migrate
- Export the directory Information from the source Exchange 5.5 server in Site S
- Delete the source mailboxes in Site S
- Wait for the two Exchange 5.5 sites to synchronize the mailbox deletions. You may force DRC (Directory Replication Connector) replication at this stage to speed things up. Also wait for the Active Directory Account to become mailbox-disabled.
- Do a directory import of the mailbox information in Site D to recreate the mailboxes.
- Move the mailboxes from the Exchange 5.5 server in Site D to the Exchange 200x server in the same site
The problem occurs after step 4. The following key points should help you understand why.
- Replication between Exchange 5.5 sites is governed by USN-Changed values. Any changes made to an object in a directory in one site only replicate to other sites if that object's USN-Changed value is higher than the corresponding value in some other site.
- For the special case of a mailbox deletion, you would expect the USN-Changed value for any DL that the mailbox is a member of to increment after the mailbox deletion. This is, however, not necessary because when you delete a mailbox from its 'home' site, the member property of the DL changes and the read-only copy of the mailbox in all the other sites gets deleted. When the read-only copy is deleted in all the other sites the DL membership is updated as well. The salient point is without incrementing a DL's USN-Changed, all the necessary changes for the sites to be in-sync are properly accounted for. This works well for a pure Exchange 5.5 environment but creates a problem for a mixed-mode one with an ADC (Active Directory Connector) in the mix.
- As we know, the ADC compares the msExchServer2HighestUSN value on the RCA (Recipient Connection Agreement) to the USN-Changed value of an object in the Exchange 5.5 directory to determine whether the Exchange 5.5 object should be replicated to the AD. If the USN-Changed on an object is greater than the msExchServer2HighestUSN on the RCA, the object has been changed in the Exchange 5.5 directory and needs to be updated in the AD. If msExchServer2HighestUSN is greater than or equal to USN-Changed, no changes have occurred on the 5.5 object that need to be updated (replicated to) on the AD side. See the following knowledge base article for further details 253840.
In this situation, since the DLs' USN-Changed isn't incremented in the 5.5 directory when you delete the mailboxes, the corresponding DG (Distribution Group) membership in the AD isn't updated.
If you later make changes that increment the USN-Changed on the DL, the entire object (including the earlier deletions) is replicated to the AD side and so some members seem to randomly disappear from the DG's members list. While replication in Exchange 5.5 is object-based and replication in the AD is attribute-based, Exchange 5.5 to AD replication is still object-based (think lowest common denominator). The solution to this problem is to force the USN-Changed on the DL to increment after performing the deletions in step 3. A DL's USN-changed increments under the following scenarios:
- If you use the Exchange 5.5 Administrator program to open the DL's properties, modify any value and click Apply or OK (Or if you modify the DL's properties by doing an directory export/import with the standard header fields)
- If you use the Exchange 5.5 Administrator program to open a mailbox's properties and remove a DL from the list of DLs that the mailbox is a member of
- If the DL is updated by DRC or ADC replication changes from other 5.5 sites or from the AD respectively
Again, a DL's USN-Changed does not increment when you use the Exchange 5.5 administrator program to delete a mailbox from the DLs membership. To force the USN-Changed value on the DL to increment you need to make a 'dummy' change that falls under a, b or c after step 3. Our new steps to ensure we avoid the problem would therefore be:
- Extract all the DLs of the mailboxes you intend to migrate
- Export the directory Information from the source 5.5 server in Site S
- Delete the source mailboxes in Site S
- Use the Exchange 5.5 Administrator program to modify the "notes" field on all the DLs that contained the deleted mailboxes to force an increment of USN-Changed value. This step is critical
- Wait for the two Exchange 5.5 sites to synchronize the mailbox deletions. You may force DRC (Directory Replication Connector) replication at this stage to speed things up. Also trigger RCA replication and make sure that the 'member of' list for the AD user accounts that correspond to the deleted mailboxes is empty (except built-in groups such as Domain Users etc)
- Do a directory import of the mailbox information in Site D to recreate the mailboxes
- Move the mailboxes from the Exchange 5.5 server in Site D to the Exchange 200x server in the same site.
- Add the migrated mailboxes to the corresponding DLs manually using the Exchange 5.5 Administrator program. This step may not seem necessary but it is. Usually an administrator may notice that the member list is still present on the AD after performing the previous steps and therefore assume that nothing more needs to be done on the Exchange 5.5 side. Later when some other change increments the USN-Changed the deletions replicate to the AD and members seem to randomly disappear from DGs.
Everything should work fine and dandy at this point and you needn't worry about 'ghosts' modifying your DGs!
- Jasper Kuria and William Yang
