Protecting Exchange data with Microsoft System Center Data Protection Manager (DPM)
Published Mar 20 2008 01:47 AM 3,267 Views

Microsoft System Centre Data Protection Manager (DPM) is one of the suite of System Centre products to be released last year. DPM offers a new approach to backing up our Exchange data and introduces a number of new alternatives to traditional approaches to backup, particularly when used alongside Exchange Server 2007. Before you can understand what these new approaches are it is useful to understand how DPM works when used to protect Exchange server data.

RTO's, RPO's and Retention Ranges
To understand how to configure DPM appropriately it is important to work towards three key pieces of information; a Recovery Point Objective (RPO), a Recovery Time Objective (RTO) and a Retention Range. In other words how much data loss is acceptable to a point in time; for how long can I be without service and or data; and finally how long do I want to keep protected data. It might be necessary to work towards multiple RTO's and RPO's. For example an RTO for recovering from database corruption might be shorter than an RTO defining a time objective for recovering from site failure. Suffice to say it is important to understand what all our objectives are for data protection before we can begin to configure DPM. It is also worth noting here that it in some cases an RTO or RPO need not necessarily be met by DPM. It may be that the use of continuous replication or an extended 'deleted item retention' period is sufficient. Once we have a set of RTO's and RPO's we need to understand how long to keep hold of this data. For most implementations it is likely that two retention ranges will need to be defined; one for the short term and another for longer term data retention. The most likely scenario is a short term retention range of say one week where DPM will retain Exchange data on disk. A longer term retention range might define a strategy for storing data to satisfy industry regulations governing long term data retention; 7 years for example. Longer term data retention might be to tape where the data is then moved offsite. Within DPM one or more Protection Schemes define our protection strategy and a Protection Group, in the context of Exchange protection, is one or more Storage Groups to which a particular protection scheme applies.
Protecting Exchange Data
There are 2 main processes used by DPM to protect Exchange mailbox and public folder data which I will focus on in this blog - the Express Full Backup and the Transaction Log Synchronisation. These ensure that changes to database pages and newly committed transaction logs, are regularly synchronised with the data held by DPM. Before either of these processes takes place however, we need to take a copy of the data held in volumes on the Exchange Server to the DPM Server to create the initial baseline. There are actually a number of ways of achieving this first copy. When creating a protection group DPM will offer the administrator the chance to let DPM take a VSS copy immediately, or at a later point in time, or the chance to manually copy the data to the DPM volumes. Once this process is complete ongoing protection will be enabled with a combination of Express Full and Transaction Log Synchronisation as follows:
Express Full
The term 'Express Full' is used by DPM to highlight the unique characteristics of this method of data protection. It is an express backup since it only takes changes and is therefore fast but the restorable end result is the same as the classic full backup; hence 'Express Full'. The Express Full backup is the process by which DPM ensures that changes to pages within the Exchange database and committed transaction logs are copied to the DPM server ensuring that the data held there is consistent and recoverable. A combination of a volume filter, a volume bitmap and the VSS 'Copy-On-Write' method is used to create 'shadow copies' of the Exchange database and transaction logs. Each time a change to a disk block is made on the database or transaction log volume, the fact that the block has changed is recorded in a volume wide bitmap. This is merely a quick 'bit flip' and does not impact the performance of the protected server. Periodically, typically each night, an express full backup is initiated by the DPM server and sent to the DPM agent (VSS Requestor) on the Exchange Server. The Exchange VSS Store Writer (there are now two for Exchange Server 2007; one built into the Store 'the Store Writer' and one built into the replication service 'the Replication Writer' discussed in more detail later) ensures that the data on disk is consistent and administrative actions and write operations against database and transaction log volumes are then suspended. Standard VSS snapshots of the protected volumes are now taken. The snapshot itself will be completed in a matter of seconds and is a combination of two main processes. The first is to build a volume filter which matches block level changes with database pages and transaction logs to identify what data is to be backed up. The second is to start tracking changes that will occur during the transfer of data to DPM. Once this process is complete write I\O is thawed and the Exchange Server can continue to serve write requests. At this point transaction logs or pages that we know have changed as a result of their appearance on the volume filter begin to be sequentially copied to the DPM server. ...so what happens when clients now want to create a new calendar appointment or write a new email? This is where VSS 'Copy-on-write' comes in. When a change to the volume occurs during the time in which changes are being backed up to DPM, and critically, before the change is physically written to disk, the disk block that is about to be modified is read and written to a difference area. (Remember the .pat file?) By doing so DPM ensures that it has a record of all the changes to the original data held on disk blocks which have changed since the last express full backup plus a record of the blocks that would have been changed\overwritten during the backup. When the backup is completed the page-level integrity of the information store database is verified (or checksummed) using eseutil with the /k switch. Transaction logs are truncated and the backup completes successfully if no discrepancies are found. If the integrity check fails the backup is aborted and the transaction logs are not truncated.
Transaction Log Synchronisation
The Express Full operation would typically occur every night but a short term protection scheme would also define how often transaction log synchronisations would occur. These occur by default every 15 minutes and use a VSS incremental synchronisation to ensure that committed, sequential transaction logs are copied to DPM. Again the Exchange Writer ensures that the data on disk is consistent and any committed transactions held in memory are flushed to disk. Administrative actions and write operations against database and transaction log volumes are then suspended. The VSS writer is notified and the incremental snapshot is taken. Once released any changed transaction logs will be transferred to the DPM recovery point volume. Transaction logs are truncated and the backup completes successfully if no discrepancies are found. A combination of Express Full and transaction log synchronisations ensures that the DPM server contains a complete consistent copy of the database together with a corresponding set of transaction logs in sequence, providing the administrator with multiple recovery points from which a restore can be initiated. The above scenario is DPM protecting data on a standard mailbox role server. Of course with Exchange Server 2007 it is now possible to run a mailbox role server with continuous replication - LCR, CCR or with Service Pack 1, SCR. The first point to make is that it is currently not supported to directly protect an SCR target database or an LCR replica. Protecting either the LCR active database or the CCR replica is possible however and should be the preferred method for most DPM deployments where either of these continuous replication methods is being used. Protecting the replica as opposed to the active database in a CCR implementation is made possible by the introduction of the Exchange Server 2007 VSS Replication Writer. Essentially the Exchange VSS Store Writer is responsible for the backup and restore of the active database and the Replication Writer is responsible for the backup of the replica database. A restore of a backup of a replica database, however, is controlled by the VSS Store Writer.
Some final thoughts...
Exchange data protection has in the past fallen somewhere between the messaging team and the teams that manage the backups and tapes in general. DPM should enable many more messaging teams to take over responsibility for the protection of the data that they are responsible for and I believe does now start to introduce an opportunity to reconsider traditional approaches to protecting Exchange data. For example how does DPM fit into an environment where CCR and SCR are already deployed? Do we still need our tape infrastructure? A combination of continuous replication and DPM protection to disk might perhaps satisfy all of our recovery objectives without the need for traditional tape based solutions. For many companies it won't, but the decision to implement DPM might be a good point to generate accurate and viable RTO's, RPO's and retention ranges for your implementation to stimulate these types of discussions. I'd like to thank Ruud Baars and Ben Appleby who helped me put this together, as well as several members of DPM team who taught us few things in the review process! - Doug Gowans
25 Comments
Not applicable
Note that the 'copy_on_write' mechanism on Exchange side is ONLY used during synchronization and that shadowcopy (type AUTODELETE) is removed when the synchronization process is complete. Shadow copies to recover from are maintained on the DPM server, not on the Exchange server.
Not applicable
Great Post! Some questions:
How does the feature "Enable SAN based recovery using hardware snapshots" interact during DPM restore? What are the best practices for Exchange DB and Log volumes in this sense? Are in this case all the snapshots (incremental and full) not discarded? Are only SANs supporting the Virtual Disk Service compatible with this model?
Regards,
Fausto
Not applicable
It seems the reasons given to the implement CCR are also be used for DPM.  I do know of a few shops leveraging CCR as a solution in both HA and backup space.  
Not applicable
Doug Gowans from the Microsoft Premier Support Team in Dubai has posted a detailed article summarizing how DPM2k7 functions in relation to Exchange 2007.  What I find of value in this Blog Post is the summarization of how DPM2k7 incorporates Volume Shadow Copy (the VSS Writer) function into its inherent Backup function.
Not applicable
thank you very much. the diagrams really made it easy for me to understand. by the way, DPM at present does not support windows server 2008, right?
Not applicable
Absolutely a great piece of information.  Nice work, Doug, Ruud and Ben.  I could not have said it better -- and I get paid to explain how DPM works. <grin>

Not applicable
How does this stack up against a solution that utilizes host-based replication for protecting Exchange like Double-Take?
Not applicable
This is great info, but can you give us an article about restoring from DPM and what can be done?  Backups are great, but what really matters is restore.
Not applicable
The backup functionality works great, now let’s look at recovery.

I believe people will find that without third party apps and a huge amount of time and process recovering mailbox’s is a painful and unrealistic process.  

Please check official MS documentation.  Once again the MS world and the real world are two different things, WHY?

Don’t get me wrong I’ve been with DPM for awhile now, cut my teeth on Beta.  What DPM “is”,  “What MS thinks DPM is” and what DPM "could be" are three very different things unfortunately.

I hope this gets looked at and corrected soon.  
Not applicable
Great article on how DPM and Exchange work, and can work together.  

Unfortunately, Milo's comment hits the real problem right on the head.  DPM's strengths and abilities are greatly diminished with the Exchange recovery steps.

Please see discussions in Microsoft Communities on Data Protection Manager.  We're still waiting on real answers from Microsoft.
Not applicable
Milo and Alf,

The DPM team would like to talk to you about your experiences with Exchange recovery. If you are interested, please email me at ninob AT microsoft DOT com and I'll get you connected.
Not applicable
i image this is listed some where else but i have yet to locate it. i was wondering if it is a requirement of DPM to use dynamic disks? we use different iSCSI SANs and were donsidering DPM. i know there are issues with dynamic disks and iSCSI, from what i seen of webcasts and articles about DPM they all mention converting the disk you are storing the protected data on to dynamic. so in short is it actually a requirement of DPM?

thanks
Not applicable
When will DPM support Exchange 2007 on Windows 2008??
Not applicable
When will DPM support Exchange 2007 on Windows 2008??
Not applicable
Well, I did as suggested and contacted MS with our complaints.  The response I received was, in my opinion, positively shameful.  After pointing out what the white paper said regarding Exchange recovery, I was informed that the white paper was "pre-release" and that it would be removed until it could be corrected.  Looks like MS is more concerned with covering their backs regarding the misleading documents than they are with responding to their users.
Not applicable
I am the DPM PM who conversed with Alf.  When we discovered that the paper had some pre-release guidance that was not updated at RTM, we pulled the paper to revalidate its claims and steps.  Our plan is to repost the paper before end of the month.  
It wasn’t  about ‘covering our backs’ – it was about being responsive and fixing an error that the community found for us, as quickly as possible.  As I explained to Alf - new features in software takes longer than correcting a whitepaper's errors.  <grin>
Not applicable
I am having problems with a replica becoming inconsistent. In this state all future backups fail.  After a few failures you are now unable to restore to the Exchange Server since it is not the most recent backup.

I would like a doc on proper DPM troubleshooting.  I would also like to see performance information. What is the highest throughput tested at Microsoft between the Exchange Server and the DPM server; both backup and restore (MB/sec).
Not applicable
Does DPM support backing up Exchange Server 2007 SP1 mailbox servers that run on Windows Server 2008. I'm hoping for an answer here :)

Not applicable
I also would like an answer to Shijaz question



Does DPM support backing up Exchange Server 2007 SP1 mailbox servers that run on Windows Server 2008. I'm hoping for an answer here :)
Not applicable
I have DPM protecting a CCR replica running Windows Server 2008.  It is not a fully provisioned server, but the backup and restoration work just fine in testing.
Not applicable
You should install KB950082 for protecting Win2k8 based servers.

Regards,
Harsh
Not applicable
Server admins who have optimistically upgraded to 2008 6 months after release are killed again. See, that's how Microsoft rewards their loyal, early adopters. Like us. "Sorry, there are no supported backup situations in a freshly upgraded WS2008 network."

Run. Run as fast as you can. And don't stop until ever last bit of MS software is out of your network - then you can be safe.
Not applicable
This is in reply to a post back in March......................

The final update and closure to this thread.

DPM works fine.  Step back and think about what the problems is.

Use Outlook Dumpster to recover items.

Use Exchange System Manager to recover deleted mailboxes.

The trick is to make sure your retention period meets your needs.

peace
Not applicable
When will the roll up package 2 be released ?
Not applicable
How does long term to tape work?  If we're doing short term to disk, does long term to tape mess with the log files?  If it does, wouldn't that mess with the short term to disk or short term to tape?
Version history
Last update:
‎Jul 01 2019 03:35 PM
Updated by: