VM creation may fail and stall with "Copying 0KB of 16GB"

[System-Center-Team]

First published on TECHNET on Jul 20, 2009

I had an interesting case the other day and since I didn’t see it documented anywhere I thought I’d mention it here in case anyone else runs into it. What can happen is that when trying to copy a template to a Hyper-V host in SCVMM 2008 the process may start, create the VHD file on the host, but never start the copying process. It may simply display "Copying 0KB of 16GB" and never display any errors but not progress either.

In order to test BITS, we used BITSadmin to move a VM from one host to another. When doing this we got the following BITSadmin output:

BITSADMIN version 3.0 [ 7.0.6001 ]
BITS administration utility.
(C) Copyright 2000-2006 Microsoft Corp.
GUID: {4D5224E7-52CB-431D-9737-F714CC69CB3A} DISPLAY: 'https://Server2.cxdomain.com:443/0e7d10e5a5ec0ed493997d75af0d6da7e623b930/e:A5cvirtual disksA5cTESTVMA5c2003StdTemplate.vhd/A5cA5cFBENSYSCTR.cxdomain.comA5cVMMTemplatesA5c2003StdTemplateA5c2003StdTemplate.vhd'
TYPE: UPLOAD STATE: ERROR OWNER: NT AUTHORITY\SYSTEM
PRIORITY: FOREGROUND FILES: 0 / 1 BYTES: 0 / 5616214528
CREATION TIME: 6/18/2009 3:18:05 PM MODIFICATION TIME: 6/18/2009 3:18:35 PM
COMPLETION TIME: UNKNOWN ACL FLAGS:
NOTIFY INTERFACE: UNREGISTERED NOTIFICATION FLAGS: 3
RETRY DELAY: 60 NO PROGRESS TIMEOUT: 0 ERROR COUNT: 16
PROXY USAGE: NO_PROXY PROXY LIST: NULL PROXY BYPASS LIST: NULL
ERROR FILE:    https://Server.domain.com:443/0e7d10e5a5ec0ed493997d75af0d6da7e623b930/e:A5cvirtual disksA5cTESTVMA5c2003StdTemplate.vhd/A5cA5cFBENSYSCTR.domain.comA5cVMMTemplatesA5c2003StdTemplateA5c2003StdTemplate.vhd -> F:\VMMTemplates\2003StdTemplate\2003StdTemplate.vhd
ERROR CODE:    0x80072efe - The connection with the server was terminated abnormally
ERROR CONTEXT: 0x00000005 - The error occurred while the remote file was being processed.
DESCRIPTION: VMM LAN Transfer Job
JOB FILES:
0 / 5616214528 WORKING https://Server2.domain.com:443/0e7d10e5a5ec0ed493997d75af0d6da7e623b930/e:A5cvirtual disksA5cTESTVMA5c2003StdTemplate.vhd/A5cA5cFBENSYSCTR.domain.comA5cVMMTemplatesA5c2003StdTemplateA5c2003StdTemplate.vhd -> F:\VMM Templates\2003StdTemplate\2003StdTemplate.vhd
NOTIFICATION COMMAND LINE: none
Peercaching flags
Enable download from peers      :false
Enable serving to peers         :false
CUSTOM HEADERS: NULL
Listed 1 job(s)

The 0x00000005 corresponds to: BG_ERROR_CONTEXT_REMOTE_FILE .

VMM Debug View Trace:

By taking a DebugView Trace on the VMM Server we saw the following exception:

00015758                        214.61161804              [61316]    at Microsoft.Carmine.WSManWrappers.ErrorContextParameterHelper.ThrowTranslatedCarmineException(ErrorInfo ei, Exception ex)
00015759                        214.61161804              [61316]    at Microsoft.Carmine.WSManWrappers.WsmanAPIWrapper.Invoke(String actionUri, String url, Hashtable parameters, Type returnType, Boolean isCarmineMethod)
00015760                        214.61161804              [61316]    at Microsoft.Carmine.WSManWrappers.AzManUtility.SetRoleAssignment(WsmanAPIWrapper wsmanObject, String Scope, String RoleName, String[] Accounts, Boolean AccountsAreSIDs, ErrorInfo& ErrorInfo)
00015761                        214.61161804              [61316]    at Microsoft.VirtualManager.Engine.HyperVAuthStore.SetVMRoleAssignment(Host host, VMData vm, String[] roles, String[] scopes)
00015762                        214.61161804              [61316]    at Microsoft.VirtualManager.Engine.HyperVAuthStore.UpdateConfiguration()
00015763                        214.61161804              [61316] *** Carmine error was: HostAgentFail (2912); HR: 0x80070490

Cause

This can occur if there is an invalid certificate on the VMM server for the host.

Resolution

To resolve this issue follow these steps:

1. Remove the troublesome host from VMM. In the VMM console, click on HOSTS and right-click on the troublesome host, select remove host. Follow the dialog.
2. Wait a few minutes then go to the host and see if the agent is uninstalled. Depending on what kind of host you configured it to be, it might not be uninstalled. Go ahead and uninstall the VMM Agent from the host if it is still there.
3. Back to the VMM Server. Run CertMgr.msc on the VMM Server. Now you are looking to find the certificate for the HOST, that might (or not) still be on the VMM Server. It will most likely be under "Trusted People", but might be under a different category.  The certificates name will be the name of the host. Right-click on it and EXPORT it as a backup. Then DELETE the certificate for the host. If the removal deleted the certificate already, we are happy too.
4. Go back to your VMM Console and add the host back into VMM.
5. Check back on the host that the VMM Agent installed.
6. Check on the VMM Server when running certmgr.msc that the certificate for that host has been added back.
7. Re-run the creation of a VM.

Justin Luyt | Senior Support Engineer