EXMerge is your friend
Published Jul 01 2004 10:58 AM 17.1K Views

One of the best utilities ever created for Exchange is EXMerge. You can get the latest version of this tool here, where it is listed as the Mailbox Merge Wizard:
 
http://www.microsoft.com/exchange/downloads/2003.asp
 
What EXMerge does may sound simple and may not seem to be all that generally useful: It can copy (or move) messages, rules and forms from an Exchange mailbox into an Outlook .PST file. And it can copy messages from an Outlook .PST file back to an Exchange mailbox.  (If you're new to Exchange and Outlook, a .PST file is a message store you can create in Outlook regardless of whether you use Exchange. You can deliver mail to it from an Internet mail account, or create multiple ones to archive and organize messages. In current versions of Outlook, you can create a .PST file from File, New, Outlook Data File.)
 
As EXMerge has evolved over the years, it has developed very sophisticated filtering capabilities. You can filter the messages EXMerge operates on by selecting or excluding folders, or by date and time ranges. You can tell it to look for a message in all Exchange mailboxes with a certain subject line or attachment name. When it finds such a message, it can copy it, move or delete it. This can be useful in a variety of situations, including cleaning up email virus messages before naive users act on them.
 
EXMerge can operate on a single mailbox, selected mailboxes or all mailboxes in an Exchange store. It can also process a batch of .PST files, matching them up correctly with Exchange mailboxes when importing data from .PST files to an Exchange server.
 
EXMerge is smart about duplicating messages: If you export all the contents of a mailbox twice to the same .PST file, you won't end up with two copies of each message. When going from .PST file to Exchange server mailbox, EXMerge is just as smart about not importing duplicates. One side effect of this excellent duplicate detection is that you can use EXMerge as a "brick backup" solution. A brick backup is a backup of a single mailbox instead of an entire database. Some third party backup solutions provide brick backup capabilities. If your backup solution doesn't, then you can use EXMerge instead. Because it is smart about duplicates, it will only export new messages to an existing .PST file each time you run it, thus providing "incremental" backup capability.
 
EXMerge is also very scriptable, which makes it easy to use it for backup or automated archiving. To learn how to script EXMerge, consult the thorough documentation that accompanies the utility. It's not just a readme file, but a comprehensive user's manual. EXMerge supports a very sophisticated .INI file that you can use to configure almost all settings, thus allowing you to execute EXMerge for a variety of different uses with a simple command line that points to a particular .INI file. You can even pick settings in the user interface, and have them saved to automatically generate an appropriate .INI file.
 
EXMerge uses a wizard-style graphical interface. You probably won't need the documentation to figure out how to use it, with one exception: You need to understand the difference between a "one step" and a "two step" merge. To understand that, you should know something about EXMerge's history.
 
EXMerge was initially developed to assist with disaster recovery of Exchange databases. Its initial purpose was to merge messages between two Exchange databases. It was originally developed in Microsoft PSS by Kali Buhariwalla, who maintained and improved it year after year. Recently, EXMerge was "adopted" by the Exchange product group, and Kali has become a Principal Consultant for Microsoft.
 
Why should you want to merge data between databases? Suppose an Exchange database has been damaged and can't be mounted without being repaired first. Repair can take hours. In the meantime, you want to restore mail service to users. So you bring up a blank database for them, and take the damaged database to another server for repair. Exchange databases are portable between servers (subject to a few rules), and after repair you can mount the repaired database on the other server. But your problem now is that you have two databases, and users can only connect to one at a time. Users have new messages in the new database and old messages in the old database. To finish the job, you have to get all messages back into a single database.
 
EXMerge comes to the rescue by extracting messages from database A to a PST file and then importing them to database B. The .PST file is used merely as an intermediary staging area. This is what happens when an administrator selects a "one step" merge. In the "two step" merge, the administrator chooses either to extract data from Exchange mailboxes (Step 1) or to import data from .PST files (Step 2). The steps are separated so that you can preserve the .PST files in case you want them for more than just a staging area (perhaps for a brick backup). Also, what if your test server and your production server are on disconnected networks and you can't reach both the source and destination servers simultaneously? Then you can do Step 1, transport the PST files to the destination server, and then do Step 2 when ready.
 
Until SP1 of Exchange 2003, the only way to extract mailbox data from a Recovery Storage Group was to use the Exchange 2003 version of EXMerge. In SP1, some of EXMerge's functionality has been integrated into the Exchange System Manager (ESM), and you can now select and right click to merge whole mailboxes between databases--this lets you do a basic "one step" merge, and this is the task you'll most often want to do during a disaster recovery. However, the ESM merge feature lacks filtering and other EXMerge features that administrators have found so useful for purposes other than disaster recovery.
 
Even if you can't think of a current use for EXMerge, you should download it and familiarize yourself with the way it works. EXMerge should be in every Exchange administrator's toolkit.

- Mike Lee

16 Comments
Not applicable
Could you elaborate a bit more on how it prevents duplicate messages? I'm curious to know what criteria it uses to determine duplicate messages from seperate copies of the same database. (like a backup of a server from 1 month ago and a fresh backup of the same server)
Not applicable
Each item in your mailbox is defined by a set of MAPI properties (PR_SOMETHING_OR_OTHER). The size of a message is a particular property (PR_MESSAGE_SIZE). So is the delivery date, when it was last modified, and even the text of the message (PR_BODY). There are also properties that are for uniquely identifying the message and comparing two messages to see if they are the "same." PR_SEARCH_KEY can be used to find correlated messages, and then you can do additional property comparisons to determine whether they are the same message (however you are defining "same" for your purposes). EXMerge not only determines if two messages are the same, but checks for later versions of the same message, and will replace an older version with an updated version as appropriate.
Not applicable
ExMerge is a great tool....when it works....I used it to migrate from a POP3 environment to a brand new Exchange 2003 environment this past january. Well, for reasons that no one can explain or seem to be recorded anywhere on the net, when importing with ExMerge all the mail apears to come over, but a few messages in each mailbox are what I'll call corrupt. They show up in the folder listing but can't be opened in either the preview pane or by double-clicking them. What is the big deal you ask? Well, first of all the problem didn't come to light until we'd migrated all 150 users since no one goes back and checks ALL of their old mail to make sure it will open and remember, the messages appear to be there in the folder list. Second, lots of tools don't work when you have corrupt messages in your mailbox--think "archive" and more importantly offline-access. I'm a huge fan of Exmerge but it screwed me pretty hard on this migration. I'm still finding mailboxes with corrupt messages and then only thing to do is delete the offending message and then go get it off the original PST and put it in the user's mailbox directly. Before you ask, the wonderful dublicate thing does not work to fix this issue. In fact, after much testing i came to discover that it generally corrupts the same messages on import, but there is no pattern as why particular messages get corrupted while others do not. I also tryed "repairing" the PSTs before I imported them but that too made no difference. So, before you rest your entire migration strategy on a single tool like I did, be darn sure that you check more than just the fact that messages show up in the folder list....just my $.02
Not applicable
Yes, Exmerge is an excellent tool and all Exchange admins should know how to use it.

Does anyone know if and when ExMerge will support UNICODE based PSTs? (this would allow you to create .PSTs greater than 2GB.)
Not applicable
I don't know if EXMerge will ever support unicode psts, but the new merge functionality that comes with E2k3 SP1 allows merging of mailboxes that are larger than 2gb.

}-]
Not applicable
Well, ExMerge is quite cool, but recently i had a problem with a E2K Server:
In Pre-SP1 E2K there was an Information Store created in the specific language of the client:

MailBox/Höchste HierarchieStufe/InBox....
instead of
MailBox/Top Of Informationstore/InBox....

AFAIK this was corrected/changed in SP1.
While restoring an (old grownup) Exchange MailBox info a newly setup SP3 Server everything was restored into the language dependend store. The newly SP3 was accessing the english version only.
So, the mailbox was full, but nobody was able to access it.
I tried to access it with ExMerge also, but not even the cool ExMerge was able to access it (or maybe i missed a parameter)...

Not applicable
Exchange-faq.dk - Din portal til Microsoft Exchange Server information
Not applicable
How much de-duping can ExMerge really do, when it's used as a migration or merge tool between one old server store and one new? The scenario is that you've got an old 5.5 store on ServerA, you 'exmerge' (for want of a better phrase describing the act of using ExMerge!) everybody's mailbox to a pst file each somewhere, and then 'exmerge' them all into a shiny new 2003 store on ServerB. ServerA's store had a single instance ratio of say, 1.8 to 1. What will ServerB's single instance ratio be with the new version of ExMerge?

In the past (like with everybody's Exchange 2000 migration), it was always 1 to 1 in the new store; in other words single instance storage of all objects had effectively been removed by the use of a PST delta in the ExMerge two step process...

Cheers
Not applicable
EXMerge does not preserve single instance storage, and it therefore causes a certain amount of growth in database file size. The reason that EXMerge can't preserve single instance is that it operates on each mailbox independently, without knowledge of what is in other mailboxes. EXMerge's duplicate detection is scoped to the level of a single mailbox, not the whole database.

How much will your database grow because of this? It varies widely, and depends on the mail habits of your users. More often than not, the single instance storage ration of an Exchange database is very low, approaching 1. The reason for this is that the "half life" of a message is inversely related to the number of people it is sent to (meaning, the more people a message is sent to, the more quickly people are likely to delete the message). This is explained briefly in this Microsoft KnowledgeBase article:

http://support.microsoft.com/?id=198673

As a rule of thumb, Microsoft PSS has recommended anticipating up to a doubling of database size after a full EXMerge of all user data.
Not applicable
Jesse, if you still have one of those .PST files that EXMerge couldn't handle, would you be interested in forwarding it to Microsoft for analysis?
Not applicable
Thanks for clarifying Mike - much appreciated. I clearly see that de-duplicating and maintenance of single instance are two separate concepts.

A thing of note on single instance is that in general I'd agree that mature stores do tend to approach 1:1 in the real world, but there are cases where your user habits will reflect your organisational culture. An example of where this comes into play with single instance ratios is within law firms that don't apply mailbox quotas. The combination of the lawyers' tendencies to store everything forever (ostensibly just in case), along with the tendency to do team-based work and share documents/revisions via email; is the reason that we see many Exchange stores in such places with si ratios approaching 3 or 4:1. These kinds of examples are surprisingly more common than I'd expected...just depends as you say, on your users habits.

Cheers
Not applicable
Mike, I'd be happy to supply them. How do I get them to you?
Not applicable
Great tool yes, but also a pain to intergrate into your automated processes. What realy is needed is a ExMerge object which can be used directly from your dot.net code. Any chances for this?
Morten
Not applicable
Jesse

Did you have any problems with your .edb and .stm files going out of sync for that mail store. I know trying to restore mailboxes after that happened cause me a to have quite a few messages that were unopenable.

John P
Not applicable
We ran exmerge last night on all the mailboxes on our whole network to delete a certain message. now random mailboxes have been cleaned out. Nothing in inbox, deleted items, sent items, or calander. Is this related?
Not applicable
Nick,

Depending on the settings that you ran Exmerge with - if you specified that Exmerge should archive only messages that have a specific attachment or the specific subject line - then I would definitely not expect it to archive anything else. I have not seen an issue like that yet anyway. In any event, as this was actually "archive from the store into the PST file" operation, you should be able to open up the PST files for those mailboxes (that Exmerge created) and check to see if there are messages other than what you indended in there.
Version history
Last update:
‎Jul 01 2019 02:58 PM
Updated by: