Diagnosing Failures in Windows Server Backup - Part 1 (VSS/SPP Errors)

[Jose Barreto]

First published on TECHNET on Sep 16, 2009

Many customers have asked questions in our forums on how to understand and troubleshoot errors reported by Windows Server Backup. While we document the event IDs logged by Windows Server Backup (for example, see http://technet.microsoft.com/en-us/library/cc734488(WS.10).aspx), we do not document the error codes (HRESULT) displayed by Windows Server Backup. This series of articles clarifies these errors codes to help system administrators troubleshoot Windows Server Backup errors.

Windows Server Backup relies on multiple layers in the operating system to function. It can show errors when any of those layers report an error. If the origin of the error is inside Windows Server Backup, Windows Server Backup provides direct recommendations on resolutions. The challenge in troubleshooting is when the problem is in a layer not in Windows Server Backup, but which emerges and is shown by Windows Server Backup.

In this article, I’ll show you how to troubleshoot errors displayed by Windows Server Backup, when the origin of the error is in an underlying layer such as Volume Shadow Copy Service (VSS), Shared Protection Point (SPP), or other applications that plug into VSS framework. (For more information about troubleshooting VSS, see http://technet.microsoft.com/en-us/library/cc734243(WS.10).aspx )These issues map to following error codes: 0x8004*, 0x81001*, 0x807800A1, 0x8078006B, 0x8078008A and 0x8078001F-0x80780025. For a detailed list of these errors refer to Appendix A  below in this article. This article applies to Windows Server 2008 and Windows Server 2008 R2.

Below are the steps that you should follow to troubleshoot errors where the cause is reported by an underlying layer to Windows Server Backup. Typically this involves using Event Viewer. To open Event Viewer, click Start , click Run , and then type:

eventvwr

General Troubleshooting Steps:

1.      Check the event log for failure events logged by Windows Server Backup.

· How : Open Event-Viewer and review at the logs at: Windows LogsApplication where the source is Backup.

2.      Identify which application or layer caused the issue.

· How : The event in previous step will usually indicate which layer or application caused the failure. For example, where VSS  is the cause, you will see a message such as Check VSS event log for details .” .Check the event log of the problem application reported by the failure event.

3.      Analyze events belonging to the application to understand the root cause. The events may point to issues in a different application.

· How : Search for events logged by the application found in step 2. For example, for errors caused by I VSS, this means, searching for all events where the value in the Source column is VSS .

4.      Repeat step 2 and step 3 until you get to the root cause of the problem. Correct the problem.

Following this method to troubleshoot errors, I will walk you through a customer reported issue where we need to do two iterations to get to root cause. You can use this example to understand how to apply the steps above. Or, if you are lucky and get the same error as in this example, you can use it directly. J Sample Issue:

When running Windows Server Backup, you receive the following error: A Volume Shadow Copy Service operation failed.

C:TEMP>wbadmin start backup -include:d:,i: -backuptarget:f: -quiet

wbadmin 1.0 - Backup command-line tool

(C) Copyright 2004 Microsoft Corp.

Retrieving volume information...

This will back up volume NTFSTest(D:),Local Disk(I:) to f:.

The backup operation to F: is starting.

Creating a shadow copy of the volumes specified for backup...

The backup operation stopped before completing.

Summary of the backup operation:

------------------

The backup operation stopped before completing.

Detailed error: A Volume Shadow Copy Service operation failed.

Troubleshooting Steps:

· Step 1: Check the event log for failure events logged by Windows Server Backup.

On checking the Backup application event log, the following message is found:

The backup operation that started at '?2009?-?07?-?09T06:30:12.721000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348129'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.

· Step 2: Identify which application or layer caused the issue.

From the backup event it is evident that the issue is caused by a VSS error.

· Step 3: Analyze events belonging to the application to understand the root cause.

On checking the VSS application event logs, the following warning event is found:

A VSS writer has rejected an event with error 0x800423f3, The writer experienced a transient error.  If the backup process is retried,

the error may not reoccur.

. Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer.

Operation:

PrepareForSnapshot Event

Context:

Execution Context: Writer

Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}

Writer Name: Microsoft Hyper-V VSS Writer

Writer Instance ID: {25785983-e057-4b58-8d90-d0a7ef6c1a22}

Command Line: C:Windowssystem32vmms.exe

Process ID: 10632

· Back to Step 2: Identify which application or layer caused the issue.

From the VSS event it is evident that the issue is caused by Hyper-V error.

· Step 3: Analyze events belonging to the application to understand the root cause.

On checking HyperV application event logs at Applications and Services LogsMicrosoftWindowsHyper-V-VMMS , the following warning event is found:

Failed to create the backup of virtual machine 'TestVM'. (Virtual machine ID XXXX)

· Back to Step 2: Identify which application or layer caused the issue.

From the Hyper-V event it is evident that the issue is caused by a virtual machine named TestVM.

· Step 3: Analyze events belonging to the application to understand the root cause.

On checking the application event logs inside the guest machine, TestVM, the following VSS event is found:

There is not enough disk space to create the volume shadow copy on the storage location. Make sure that, for all volumes to be backup up, the minimum required disk space for shadow copy creation is available. This applies to both the backup storage destination and volumes included in the backup.

Minimum requirement: For volumes less than 500 megabytes, the minimum is 50 megabytes of free space. For volumes more than 500 megabytes, the minimum is 320 megabytes of free space.

Recommended: At least 1 gigabyte of free disk space on each volume if volume size is more than 1 gigabyte.

· Step 4: Correct the problem.

The above error indicates that freeing up space on the physical volume hosting TestVM will solve the issue.

Appendix A

Windows Server Backup Error List

HRESULT	HRESULT (in hex)	Error Message