Exchange Circular Logging and VSS Backups
Published Aug 18 2010 12:10 PM 61K Views

In discussions with some of our friends over in the DPM team (System Center Data Protection Manager), we’ve found that there is a rather high incidence of VSS errors in installations where DPM is being used to back up Exchange 2010. Turns out that in many instances, this is because of an incorrect configuration of circular logging on the mailbox databases being backed up, so we thought we would quickly discuss circular logging, what it is, and how it affects VSS backup scenarios.

Circular logging has been around for a long time in the Exchange world. I found KB147524, which references Exchange 4.0, and discusses circular logging. When you configure an Exchange database with circular logging turned on, Exchange doesn’t wait until a backup occurs to truncate transaction log files. Rather, as soon as the log files have been played forward into the database, Exchange is free to delete those transaction logs. In Exchange 2003 and before, this was always handled by the Information Store service.

With Exchange 2007 and later there is actually another form of circular logging, known as continuous replication circular logging (CRCL). There is a great discussion of this in the Continuous Replication Deep Dive white paper, so I’m not going to delve into the mechanics of how it works, other than to state that the system ensures that logs are not truncated on the source until all copies agree with the deletion.

Circular Logging is useful in a few scenarios:

  • Customers leverage circular logging on mailbox databases that do not contain any user data.
  • Customers leverage circular logging on mailbox databases within lab environments.
  • Customers leverage circular logging on mailbox databases in Exchange 2010 when they utilize the Exchange Native Data Protection built into the product because there is no traditional VSS backup solution used to manage the log file truncations.

Customers occasionally leverage circular logging on mailbox databases when performing mass mailbox moves to a single mailbox database target within a small period of time to minimize the capacity impacts that could cause an outage event on the target database. If you are deploying an Exchange solution that will continue to leverage a VSS backup infrastructure and enable circular logging it is imperative that you remember to disable circular logging prior to your next backup window, otherwise your next full or incremental backup will fail. For example, some VSS solutions perform incremental backups of Exchange data by capturing the transaction log files generated since the previous backup that managed/truncated the transaction logs (a full backup, or the previous incremental backup). If you have circular logging turned on, when the incremental backup is performed, the log files that are expected to be there since the previous backup are not there – they have been truncated – causing the backup to fail.

So, what are we saying here? We know that there are some valid and supported scenarios where circular logging is very useful. The idea here we want to reinforce is that when you are performing VSS backups that rely on the transaction logs, make sure that your normal run state is with circular logging turned off. If you have a reason to turn circular logging on when utilizing VSS incremental backups that rely on the transaction log files, remember to turn it back off as soon as reasonable, and understand that while circular logging is on that your incremental backups will fail to complete as expected!

-- Robert Gillies

17 Comments
Not applicable
I've had issues where I did enable circular logging for mailbox moves, then disable it again.  DPM failed for reasons mentioned above.  I dismounted and remounted the database (with circular logging off) and DPM was happy again.  What gives?
Not applicable
Mike,

Each time circular logging is enabled or disabled, the database has to be dismounted and mounted back for the changes to take effect.  See http://www.howexchangeworks.com/2010/05/mailbox-database-copy-and-circular.html
Not applicable
Suppose I am using DAGs to ensure I don't lose data. Is that compatible with circular logging?
Not applicable
Mark,

Circular logging can must be disabled to create a database copy on another server.  After the database has been copied, you can enable circular logging.  CRCL, mentioned above, will ensure that no log is truncated until all copies agree.

You would probably only enable circular logging on DAGs in a backup-less scenario where you have so many copies of the database you're comfortable with not having backups.
Not applicable
Doh! I meant, "Circular logging must be disabled to create a database copy on another server."  See http://technet.microsoft.com/en-us/library/dd298080.aspx
Not applicable
That TechNet article says you must have circular logging disabled to create the database copy, but can disable it later.

If I am using enough DAGs to be satisfied I don't need backups, enabling circular logging would be a good idea. Right? Becuase, with no backup running, the log would otherwise never truncate...
Not applicable
Hi, can  help me  with  a problem,  i have  enabled Circular Logging   and i make backup  with  Windows Server Backup extension  of exchange 2010,  the firsts  backup   ended  fine, but  now   it is  ended  with problems  en re build  fo system  and  system state,  if  i disabled circular login it must be  better?, Windows backup extension  of exchange 2010 truncate transacction log if  i disabled circular logging?.

Regards.
Not applicable
Mark, If you have enough copies (recommendation is a minimum of 3), you can deploy what we call Exchange Native Data Protection where you turn on CRCL to manage the transaction log files.  There are a lot of other things to consider before going down that

path.  Here is a good article that discusses the things to think about:  http://technet.microsoft.com/en-us/library/dd876874.aspx#FleMaiPro



Eduardo, with any data protection solution other than the Exchange Native Data Protection (where you use multiple copies on the DAG along with Single Item Restore and other Exchange 2010 capabilities to protect your data), you need to disable circular logging

on all of your databases and allow your backup solution to manage transaction logs.  This problem we're documenting above that was brought up by the DPM team is not unique to DPM - it could apply to any backup solution out there.

Not applicable
Gents, you need to remember that while DAG's allow you to have multiple copies which can be configured with some delay in place between copies these will not allow you to recover back data for all compliancy requests. Backups are therefore a very important part of you data compliancy strategy and hence turning off circular logging needs careful thought.

If you plan to use anything other than Full copies each day you will have to turn circular logging off otherwise you won’t be able to restore back multiple incremental backups and hence roll forward.


MS,  DAG's are the best thing since sliced bread, well done.
Not applicable
We have circular logging enabled on one of our exchange servers (2007).
I am now thinking to disable it as we have exchange aware backup app which will (and is capable of) truncate logs.

Do I need to :

1) Disable (uncheck) Circular Logging
2) Dismount the Mailbox DB and PF DB
3) Mount the DBs again.

Q. Would I need to stop the exchange services as well or I can do the above 3 steps without stopping any service ?

Q. If disable circular logging and restart the machine, would that enable it or do I have to Dismount / Mount the DBs before the restart ?

Will be grateful for your assistance on this !
Not applicable
Restarting the machine would certainly make the change in circular logging status be picked up in any case, and that means that if you have to reboot to install the agents for your backup product that might be a good time to do this.

As for whether you need to restart anything, it depends if you are in a CCR cluster or not (or a DAG in 2010).  If you are, you are in CRCL - Continuous Replication Circular Logging - and that is managed by the replication service.  I believe that this will pick up the change without dismounting and remounting the database.  

If you are not in CRCL - if you are in traditional circular logging (no CCR, no DAG) - then you definatly need to dismount and remount the database for the circular logging changes to take affect.
Not applicable
I have taken #1 Full Backup- It backed up EDB & log 1-20 and purged logs 1-20, then took incremental backup(#3) - It backed up logs 21-30, and purged logs 21-30. I enabled circular logging. It generated logs 31 - 50 and overwritten logs 31-40. I take #2 Full backup - It backup edb & logs 41-50 and purged those. If I take incremental backup(#4) after #2 Full backup, will it look for logs starting from 31(Continuation of incremental backup(#3) ) or log starting from 51?
Not applicable
Sukum, I think you're missing a few points.  When you turn on circular logging, Exchange transaction log files are not "overwritten" the same way that events are in event viewer and the even lots.  What circular logging means is that when Exchange has completed all operations needing the transaction log files (which has different meanings for non-replicated, replicated HA copies and replicated lagged copies), Exchange will truncate those logs.  It deletes them upon truncation.

What this means, is that when you enable circular logging in your example above, the full backup following wouldn't have logs 41-50 to back up (they would have been truncated) and the following incremental backups (#4 and beyond) will fail because there are not transaction log files that can be backed up (other than ones that have not been committed to the database).

Does that make better sense?
Not applicable
Thank you for your comment.  I believe the question was bit confusing. Sorry about it.

Let me put this way ,

>> I use Netbackup 7.0 and Exchange 2010 RU3
>> I have edb file with committed logs – 20 to 50
>> I took FullBackup(#1)
>> It backed up edb file and logs – 20 to 50. After successful backup, it deleted 20 to 50
>> Then I had committed logs – 51 to 100
>> I took incremental backup(#2). It deleted 51 to 100 logs after successful backup.
>> Then I had committed logs – 101 – 200.
>> My logs file disk was running out of disk space.
>> So I deleted logs 101 to 150
>> Then I took incremental backup(#3) and it failed with event that log 101 missing (it was expected behavior)
>> I took full backup(#4) and it was successful and deleted the logs 151-200
>> I had committed logs – 201 to 300
>> I took incremental backup(#5), but it failed with event that log 101.

>> My expectation is, since I have taken fullback(#4) and it backed up EDB and log files – 151 to 200, the subsequent incremental backup should look for logs starting from 201, but in my case it failed with event that log 101 was missing.
>> Why it is so and how I can succeed with my future incremental backups. What should I do to succeed with incremental backup?
Not applicable
Sukum,

I am not sure what would cause that - whether it is a VSS issue or a Netbackup issue.  I have never used Netbackup against Exchange Server 2010 and am not qualified to answer questions about how it works.

I would suggest talking to Netbackup support first, and from there that might lead to a Microsoft CSS support call.

In the future, though, I would suggest that the way to properly truncate logs when you have an issue where log space is running out would be to run an incremental backup and let the backup program manage the log file truncations.
Not applicable
I agree with you, Robert. Thank you once again.I have one more quick question- After incremental backup, a set of logs are deleted by Exchange. Can you please help me to understand how the exchange knows these are the logs it has to delete. Will backup application update exchange that I made a copy these log files  and succeeded in backup, now you go ahead delete the logs?
Not applicable
Sukum,

That is a good question.  It all comes down to how the Exchange VSS Writer handles the backup scenario.  Here is some documentation:  http://msdn.microsoft.com/en-us/library/bb204080(EXCHG.140).aspx

Remember that Exchange is a transactional database.  What this means is that any time a change is made to an Exchange database, it must be written to the transaction log files.  Until it is written, it didn't happen.  These transactions are stored in memory in the database cache as "dirty pages" (meaning that they are updated since they were last read from disk).  Which pages are "dirty" is tracked in the Version Store (you might have heard of "version buckets", which is a grouping in the version store).  Then, some time later, a process called "the lazy writer" comes along and writes these changed pages from the database cache to the database.  (NOTE:  This is a fairly simplified version of the process!)

There is a special file called the checkpoint file that points to the oldest transaction log file that has any data in it that has not been written to the database.  This is used by Exchange to recover a database after a crash - the database knows that it can just "play forward" the transactions in the log files after the checkpoint and build a database that has all of the updates and changes made before it crashed and lost the dirty pages in the cache.  (Once again, oversimplication is possible...)

Now, when you do a backup of the database (a full backup), VSS will either get a full backup of the database, or a block-wise partial backup that can be combined with an older full backup to create what the database looks like at the time of the backup (DPM would call this an express full).  After that completes, the VSS writer tells Exchange "I backed up the database, you can truncate up to the checkpoint file."

As for incrementals, at least the way DPM does them (I am not familiar with other backup software), the VSS backup will copy the transaction log files that have been generated since the last truncation.  Then, the VSS writer informs Exchange that it can truncate because a successful incremental backup occurred, and Exchange truncates up to the checkpoint.

Exchange keeps track of the backups that have occurred, as does the backup software.  It is the responsibility of the backup software to inform Exchange that the backup was successful, that the data was validated (assuming they validate the backup data), and that the log files can be truncated to a certain point.

Does this help?
Version history
Last update:
‎Aug 18 2010 12:10 PM
Updated by: