Hello

maybe someone know where my deployment goes wrong... My deployment always ends up with 

2024-06-10 09:32:35 Warning  [DeploymentService:InvokeEnvironmentChecker] Task: Invocation of interface 'ValidateNetwork' of role 'Cloud\Infrastructure\EnvironmentValidator' failed: 

Type 'ValidateNetwork' of Role 'EnvironmentValidator' raised an exception:

No MSFT_NetIPAddress objects found with property 'InterfaceAlias' equal to 'Ethernet'.  Verify the value of the property and retry.

I am trying to deploy a single node with a single NIC.

Hi, posting here out of desperation. I am consistently seeing the issue mentioned in this article where the local administrator group is not populated with the correct accounts and NT service accounts. The first failure this causes is at the "Install the update orchestrator" stage where we get this error

WatsonBuckets": null }, "Message": "AzureStack File Copy Agent is not running.", "Type": "HealthCheckException", "CallStack": "HealthCheckException: AzureStack File Copy Agent is not running.\r\n" } }, { "ErrorType": "ResourceFailureLocalGroupMembership", "ErrorInfo": { "Exception": { "ClassName": "Microsoft.AzureStack.Infrastructure.Orchestration.AgentLifecycleAgent.ResourceManagement.Exceptions.LocalGroupMembershipException", "Message": "Failed to set up membership for NT SERVICE\\AzureStack File Copy Agent", "Data": null, "InnerException": { "ClassName":

You can watch the event logs whilst the deployment running for it constantly trying to create this service, which I assume is failing due to it not being able to add the service account to the local admin group.

Permissions have been verified for the deployment user (its in the administrators group on both nodes). I have a long running case open with MS but we are going around in circles at the moment. 

I notice this exact issue is mentioned here and thought i'd reach out to see if anyone else is having / has had this and knows of a cause of fix. I am nearly in double digits for the times i've flattened and rebuilt / re-deployed this cluster!

All help greatly appreciated whilst I still have some hair

These steps helped here:

Workaround:
- removing LCM extension via PS or Portal
- creating the mentioned HKLM Key (as removing will delete it) via PS
- reboot the nodes via PS
- register LCM extension via PS

Root Cause:
The function Test-Nodeinitialization runs before or during LCM extension creation. It will test for a registry key which is not present before the LCM deployment is not already succeded.
Only after the successful creation of LCM extension this registry key is created and contains the expected String Value.

Solution:
The exception in the function should not terminate the deployment of the LCM extension but rather check for the existance of this registry key (Test-Path), and if it does not exist should create the registry key.

xtension Message: Enable-Extension.ps1 : Installing LCM extention failed: 
Extension Error: Get-ItemProperty : Cannot find path 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\LCMAzureStackStampInformation' because it 
does not exist.
At C:\NugetStore\Microsoft.AzureStack.Solution.LCMControllerWinService.10.2402.0.15\content\LCMControllerWinService\Dep
loymentScripts\LCMControllerWinService.psm1:128 char:30

 

the error refers to this function below and in fact the complete hive is empty. And on a fresh 23H2 box it does not exist.
NOTHING in the whole LCMControllerWinService.psm1 module does actually create this hive in the registry.
Especially NOT the function Install-LCMController.

Please try https://github.com/DellGEOS/AzureStackHOLs/tree/main/lab-guides/01a-DeployAzureStackHCICluster-CloudBasedDeployment in WestEU.

LCMControllerWinService.psm1 reference 

function Test-NodeInitialization
{
    $existingService = Get-WmiObject Win32_Service | where {$_.Name -eq 'LcmController'}
    $eceLiteDir = Split-Path $existingService.PathName
    $eceLite = Join-Path $eceLiteDir "EnterpriseCloudEngine.psd1"
    $InitializeAction = "InitializeDeploymentService"
    $initializationStatus = (Get-ItemProperty -path Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\LCMAzureStackStampInformation -Name InitializationComplete).InitializationComplete
    if (![string]::IsNullOrEmpty($initializationStatus) -and ($initializationStatus -eq "Complete"))
    {
        Write-Verbose "Node Initialization successfull."
    }
    elseif (Test-path -path $eceLite)
    {
        Import-Module $eceLite
        $actionProgress = Get-ActionProgress -ActionType $InitializeAction
        if (!$actionProgress)
        {
            throw "Node Initialization not started."
        }
        elseif ($actionProgress -and $actionProgress.Attribute("Status").Value -eq "Success")
        {
            Write-Verbose "Node Initialization successfull."
        }
        elseif ($actionProgress -and $actionProgress.Attribute("Status").Value -eq "Error")
        {
            Write-Verbose "Node Initialization failed with error: $($actionProgress.Attribute("Status").Parent.Value)"
            throw "Node Initialization failed with error: $($actionProgress.Attribute("Status").Parent.Value)"
        }
        elseif ($actionProgress -and $actionProgress.Attribute("Status").Value -eq "InProgress")
        {
            Write-Verbose "Initialization action plan status:$($actionProgress.Attribute("Status").Value)"
            throw "Node Initialization in progress."
        }
        else
        {
            Write-Verbose "Node Initialization failed."
        }
    }
    else
    {
        throw "Node Initialization failed."
    }
}

mtbmsft  Islam Gomaa on our Azure Stack HCI Slack there are several reports spanning for months that most failing resource in pre-deployment is LCM

This one is also handled here:
https://techcommunity.microsoft.com/t5/fasttrack-for-azure/common-deployment-challenges-and-workarounds-for-hci-23h2/ba-p/4044172


Can you confirm that the telemetry collected with every 23H2 deployment and (failed) attempt is collected even at this command?

   Invoke-AzStackHciArcInitialization

----

 
Having the following outputs:

Azure Portal Azure Stack HCI Wizard page 1
Validate Selected Server:

Resource validation failed. Details: [{"Code":"ValidationFailed","Message":"Arc extensions installed on Arc Machine /subscriptions/a69d12f1-e62b-49b1-a483-6bc11b28a923/resourceGroups/AzSHCI-Clu-112-rg/providers/Microsoft.HybridCompute/machines/ASNode1 are 
- DeviceManagementExtension
- EdgeRemoteSupport
- TelemetryAndDiagnostics 

while required list of mandatory arc extensions are 
- DeviceManagementExtension, 
- LcmController,
- TelemetryAndDiagnostics, 
,"Target":null,"Details":null},

So LCMController is the culprit once again, also visible on both fresh nodes
CU: 23H2 MBR 709
LCM Version: 30.2402.0.15
Scriptversion: 10.2402.0.15

The error log in the extension is extremely helpful and verbose,but for me looks like an internal code issue.

Microsoft.AzureStack.Orchestration.LcmController
Status
Failed
Automatic upgrade
Not supported
Version
30.2402.0.15
Status level
Error
Status message
Extension Message: Enable-Extension.ps1 : Installing LCM extention failed: 
Extension Error: Get-ItemProperty : Cannot find path 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\LCMAzureStackStampInformation' because it 
does not exist.
At C:\NugetStore\Microsoft.AzureStack.Solution.LCMControllerWinService.10.2402.0.15\content\LCMControllerWinService\Dep
loymentScripts\LCMControllerWinService.psm1:128 char:30
+ ... onStatus = (Get-ItemProperty -path Registry::HKEY_LOCAL_MACHINE\SOFTW ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (HKEY_LOCAL_MACH...tampInformation:String) [Get-ItemProperty], ItemNotFo 
   undException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemPropertyCommand
 
Get-ItemProperty : Cannot find path 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\LCMAzureStackStampInformation' because it 
does not exist.
At C:\NugetStore\Microsoft.AzureStack.Solution.LCMControllerWinService.10.2402.0.15\content\LCMControllerWinService\Dep
loymentScripts\LCMControllerWinService.psm1:128 char:30
+ ... onStatus = (Get-ItemProperty -path Registry::HKEY_LOCAL_MACHINE\SOFTW ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (HKEY_LOCAL_MACH...tampInformation:String) [Get-ItemProperty], ItemNotFo 
   undException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemPropertyCommand
 
Transcript started, output file is C:\MASLogs\Install-LCMController_20240223-023842Z.log
VERBOSE: Loading module from path 
'C:\NugetStore\Microsoft.AzureStack.Solution.LCMControllerWinService.10.2402.0.15\content\LCMControllerWinService\Deplo
ymentScripts\LogmanHelpers.psm1'.
VERBOSE: Importing function 'Register-LogManScheduledTask'.
VERBOSE: Start-LCMControllerServiceLogman : Registering LogMan ScheduledTask.
VERBOSE: Register-LogManScheduledTask : starts logman script started.
VERBOSE: Register-LogManScheduledTask : Creating scheduled task: Start logman for LcmController service
VERBOSE: Register-LogManScheduledTask : Scheduled task Start logman for LcmController service created, register it to 
task scheduler

TaskPath                                       TaskName                          State     
--------                                       --------                          -----     
\                                              Start logman for LcmController... Ready     
VERBOSE: Register-LogManScheduledTask : Scheduled task Start logman for LcmController service is now registered
VERBOSE: Start-LCMControllerServiceLogman : Starting LogMan ScheduledTask.
VERBOSE:  : Installing LCMController Service.

Status      : Stopped
Name        : LcmController
DisplayName : LcmController

[SC] ChangeServiceConfig2 SUCCESS
VERBOSE:  : LcmController service created.
VERBOSE: LcmController starting.

__GENUS          : 2
__CLASS          : __PARAMETERS
__SUPERCLASS     : 
__DYNASTY        : __PARAMETERS
__RELPATH        : 
__PROPERTY_COUNT : 1
__DERIVATION     : {}
__SERVER         : 
__NAMESPACE      : 
__PATH           : 
ReturnValue      : 0
PSComputerName   : 

VERBOSE:  : Waiting 300 seconds for LcmController to start, attempt 1 of 5 ...
VERBOSE:  : LcmController service Running.
VERBOSE: Loading module from path 
'C:\NugetStore\Microsoft.AzureStack.Solution.LCMControllerWinService.10.2402.0.15\content\LCMControllerWinService\Enter
priseCloudEngine.psd1'.
VERBOSE: Exporting function 'Get-EceInterfaceParameters'.
VERBOSE: Exporting function 'Test-EceInterface'.
VERBOSE: Exporting function 'Get-DeploymentActionPlanLog'.
VERBOSE: Exporting cmdlet 'Trace-ECEScript'.
VERBOSE: Exporting cmdlet 'Set-EceSecret'.
VERBOSE: Exporting cmdlet 'Get-EceConfiguration'.
VERBOSE: Exporting cmdlet 'Get-ActionProgress'.
VERBOSE: Exporting cmdlet 'Get-JsonTemplate'.
VERBOSE: Exporting cmdlet 'Invoke-EceAction'.
VERBOSE: Exporting cmdlet 'Join-RoleTemplate'.
VERBOSE: Exporting cmdlet 'Import-EceCustomerConfiguration'.
VERBOSE: Exporting cmdlet 'Set-RoleDefinition'.
VERBOSE: Importing cmdlet 'Get-ActionProgress'.
VERBOSE: Importing cmdlet 'Get-EceConfiguration'.
VERBOSE: Importing cmdlet 'Get-JsonTemplate'.
VERBOSE: Importing cmdlet 'Import-EceCustomerConfiguration'.
VERBOSE: Importing cmdlet 'Invoke-EceAction'.
VERBOSE: Importing cmdlet 'Join-RoleTemplate'.
VERBOSE: Importing cmdlet 'Set-EceSecret'.
VERBOSE: Importing cmdlet 'Set-RoleDefinition'.
VERBOSE: Importing cmdlet 'Trace-ECEScript'.
VERBOSE: Importing function 'Get-DeploymentActionPlanLog'.
VERBOSE: Importing function 'Get-EceInterfaceParameters'.
VERBOSE: Importing function 'Test-EceInterface'.
VERBOSE: Initialization action plan status:InProgress
VERBOSE: Node Initialization in progress.
Transcript stopped, output file is C:\MASLogs\Install-LCMController_20240223-023842Z.log

This has been very helpful in figuring out the install process and navigating the logs thank you.

mtbmsft Islam Gomaa is there a guidance or script that helps to cleanup objects in the Azure Portal aswell for removed or failed deployments?

Today found couple of applications.

Entra ID > Enterprise Applications:
Remove filter (All Applications)
clustername.arb

Great Work mtbmsft