Creating and exporting a collection with variable properties

[JulianMilano]

My script logs into multiple Cisco UCS servers and extracts the CPU information, storing it in a collection which is then written to a CSV file. My problem is that I have a loop which creates a set of properties for each CPU in the server and adds it to the collection. However some servers have 2 CPUs and some have 4. When my code exports the data to CSV, I only get 2 CPUs of data for ALL servers, even though I can see 4 CPUs defined in the collection as I step thru the code.

Thus, as the code runs and collects CPU data, the collection ($IMCRackServerInventory) looks like this:

Server01, CPU1_ID, CPU1_Model, CPU2_ID, CPU2_Model
Server02, CPU1_ID, CPU1_Model, CPU2_ID, CPU2_Model
Server03, CPU1_ID, CPU1_Model, CPU2_ID, CPU2_Model, CPU3_ID, CPU3_Model, CPU4_ID, CPU4_Model
Server04, CPU1_ID, CPU1_Model, CPU2_ID, CPU2_Model

But then the collection is exported to CSV and I only get the following in the CSV:

Server01, CPU1_ID, CPU1_Model, CPU2_ID, CPU2_Model
Server02, CPU1_ID, CPU1_Model, CPU2_ID, CPU2_Model
Server03, CPU1_ID, CPU1_Model, CPU2_ID, CPU2_Model
Server04, CPU1_ID, CPU1_Model, CPU2_ID, CPU2_Model

So it seems to be only writing the first 2 CPU details to file as not all records have 4 entries.

Is there a way around this?

My Code:

#
# ================================FUNCTION DECLARATION START====================================================================
#
 
#
# Check the required columns for this script to run, are present in the user-selected CSV import file.
Function CheckCSVColumnsExist
    {
    Param(
        [Object]$CSVImportFile,
        [Array]$ColumnsToCheck = ''    
        )
 
    $ColumnHeaders = (Import-Csv $CSVImportFile | Get-Member -MemberType NoteProperty).Name
    $MissingColumnHeaders = @()
    ForEach( $ColumnToCheck in $ColumnsToCheck)
        {
        $MissingColumnName = New-Object PSObject
        If ($ColumnHeaders -match $ColumnToCheck )
            {
            # Nothing to do.
            }
        Else
            {
            $MissingColumnName | Add-Member -type NoteProperty -Name 'Column_Name' -Value $ColumnToCheck
            $MissingColumnHeaders += $MissingColumnName
            }
         
        }
    Return $MissingColumnHeaders
    }
   
# This function shows the Open File dialog to the user so the user can
# select the import CSV file.
#
Function Get-FileName($InitialDirectory)
    {
    [System.Reflection.Assembly]::LoadWithPartialName("System.windows.forms") | Out-Null
  
    $OpenFileDialog = New-Object System.Windows.Forms.OpenFileDialog
    $OpenFileDialog.initialDirectory = $initialDirectory
    $OpenFileDialog.filter = "CSV (*.csv) | *.csv"
    $OpenFileDialog.ShowDialog() | Out-Null
    $OpenFileDialog.FileName
    }
 
#
# ================================FUNCTION DECLARATION FINISH===================================================================
#
 
#
# ================================SCRIPT START==================================================================================
#
#
Import-Module Cisco.IMC

# The input CSV file contains the details required for the script
# to log into each Cisco UCS server.
#
# Get the CSV import file name and path:
$ImportCSVFile = Get-FileName

Write-Host "Script starting." -ForegroundColor Green
 
# Check if the user cancelled the request.
if ($ImportCSVFile -eq "")
    { # They did!
    Throw "No file selected. Ending script"
    }

# Load the CSV file contents.
$UCSRackServerReport = Import-CSV $ImportCSVFile -ErrorAction SilentlyContinue
# Ask the user for the CIMC credential details.
$IMCServerCredential = Get-Credential admin

$IMCRackServerInventory = @()
# Read each entry in the import CSV file, then try to logon to the 
# CIMC of each one and collect hardware details.
ForEach ($vRow in $UCSRackServerReport)
    {
    $Item = New-Object PSObject
    # Get the server CIMC IP.
    $RackServerIMCIP = $vRow."IP Address"
    # Get the server User Label and Host Name.
    $RackServerIMCUserLabel = $vRow."User Label"
    $RackServerIMCHostName = $vRow."Host Name"
    # If the User Label is blank, use the Host Name for console output updates.
    If ([string]::IsNullOrWhiteSpace($RackServerIMCUserLabel) )
        {
        $StringLabel = "[$RackServerIMCHostName/$RackServerIMCIP]"
        }
    Else
        {
        $StringLabel = "[$RackServerIMCUserLabel/$RackServerIMCIP]"
        }
    Try
        {
        # Let's connect to the CIMC.
        $Connected = $True
        Write-Host "$StringLabel Attempting a connection to server IMC." -ForegroundColor Yellow
        $IMCHandle = Connect-IMC -Name $RackServerIMCIP -Credential $IMCServerCredential -NotDefault -ErrorAction Stop
        }

    Catch
        {
        # Did not connect due to connection issue.
        Write-Host "$StringLabel Error: $($PSItem.ToString())" -ForegroundColor Red
        If ($PSItem.Exception.ToString().Contains("Unable to connect"))
            {
            # Set connection flag false.
            $Connected = $False
            }
        }
    # Did we connect to the CIMC?
    If (!$Connected)
        {
        # No we did not. Record this event.
        If ([string]::IsNullOrWhiteSpace($RackServerIMCUserLabel) )
            {
            $NewUserLabel = $RackServerIMCHostName
            }
        Else
            {
            $NewUserLabel = $RackServerIMCUserLabel 
            }
        # Set collection details and the properties not read in will be set to "Not Connected".
        $Item | Add-Member -type NoteProperty -Name 'UserLabel' -Value $NewUserLabel
        $Item | Add-Member -type NoteProperty -Name 'Model' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'Serial' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'ServerID' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'CIMC_IP' -Value $RackServerIMCIP
        $Item | Add-Member -type NoteProperty -Name 'Vendor' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'IMCName' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'DN' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'Rn' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'CPUID_1' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'CPUModel_1' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'OperState_1' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'Presence_1' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'SocketDesignation_1' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'CPUVendor_1' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'CPUIMC_1' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'Dn_1' -Value "Not Connected"
        $Item | Add-Member -type NoteProperty -Name 'Rn_1' -Value "Not Connected"
        }
    Else
        {
        # We connected OK.
        Write-Host "$StringLabel Connection to server IMC established." -ForegroundColor Cyan
        Write-Host "$StringLabel Gathering Rack Unit data." -ForegroundColor Cyan
        # Get rack server hardware details.
        $IMCServerUnit = Get-IMCRackUnit -Imc $IMCHandle
        # Check if User Label is blank and if so, use the IMC field as the server label.
        If ([string]::IsNullOrWhiteSpace($IMCServerUnit.UsrLbl) )
            {
            $NewUserLabel = $IMCServerUnit.Imc
            }
        Else
            {
            $NewUserLabel = $IMCServerUnit.UsrLbl
            }
        # Record rack server hardware details.
        $Item | Add-Member -type NoteProperty -Name 'UserLabel' -Value $NewUserLabel
        $Item | Add-Member -type NoteProperty -Name 'Model' -Value $($IMCServerUnit.Model)
        $Item | Add-Member -type NoteProperty -Name 'Serial' -Value $($IMCServerUnit.Serial)
        $Item | Add-Member -type NoteProperty -Name 'ServerID' -Value $($IMCServerUnit.ServerId)
        $Item | Add-Member -type NoteProperty -Name 'CIMC_IP' -Value $RackServerIMCIP
        $Item | Add-Member -type NoteProperty -Name 'Vendor' -Value $($IMCServerUnit.Vendor)
        $Item | Add-Member -type NoteProperty -Name 'IMCName' -Value $($IMCServerUnit.Imc)
        $Item | Add-Member -type NoteProperty -Name 'DN' -Value $($IMCServerUnit.Dn)
        $Item | Add-Member -type NoteProperty -Name 'Rn' -Value $($IMCServerUnit.Rn)
        Write-Host "$StringLabel Gathering Processor data." -ForegroundColor Cyan
        $CPUCounter = 1
        # Get the rack server processor details.
        $IMCServerProcessors = Get-IMCProcessorUnit -Imc $IMCHandle
        # For multiple processors, cycle thru each object found.
        ForEach ($IMCServerProcessor in $IMCServerProcessors)
            {
            # Dynamically create the property name with "_#" at the end for each processor. Record processor details.
            $FieldName = "CPUID_$CPUCounter"; $Item | Add-Member -type NoteProperty -Name $FieldName -Value $($IMCServerProcessor.Id)
            $FieldName = "CPUModel_$CPUCounter"; $Item | Add-Member -type NoteProperty -Name $FieldName -Value $($IMCServerProcessor.Model)
            $FieldName = "OperState_$CPUCounter"; $Item | Add-Member -type NoteProperty -Name $FieldName -Value $($IMCServerProcessor.OperState)
            $FieldName = "Presence_$CPUCounter"; $Item | Add-Member -type NoteProperty -Name $FieldName -Value $($IMCServerProcessor.Presence)
            $FieldName = "SocketDesignation_$CPUCounter"; $Item | Add-Member -type NoteProperty -Name $FieldName -Value $($IMCServerProcessor.SocketDesignation)
            $FieldName = "CPUVendor_$CPUCounter"; $Item | Add-Member -type NoteProperty -Name $FieldName -Value $($IMCServerProcessor.Vendor)
            $FieldName = "CPUIMC_$CPUCounter"; $Item | Add-Member -type NoteProperty -Name $FieldName -Value $($IMCServerProcessor.Imc)
            $FieldName = "Dn_$CPUCounter"; $Item | Add-Member -type NoteProperty -Name $FieldName -Value $($IMCServerProcessor.Dn)
            $FieldName = "Rn_$CPUCounter"; $Item | Add-Member -type NoteProperty -Name $FieldName -Value $($IMCServerProcessor.Rn)
            $CPUCounter += 1
            }
        Write-Host "$StringLabel Gathering Firmware data." -ForegroundColor Cyan
        # Get the firmware package details.
        $IMCFirmwarePackage = Get-ImcFirmwareRunning -Imc $IMCHandle | Where {$_.Deployment -eq "system" -and $_.Type -eq "blade-controller"}
        # Add the firmware package details to the collection.
        $Item | Add-Member -type NoteProperty -Name 'BMCFirmwarePackage' -Value $($IMCFirmwarePackage.Version)
        }
    $IMCRackServerInventory += $Item
    Write-Host "$StringLabel Data collection completed." -ForegroundColor Blue
    # Close the connection to the server's CIMC.
    If ($IMCHandle -ne $Null )
        {
        $CloseIMCHandle = Disconnect-IMC -Imc $IMCHandle
        }
    Write-Host "$StringLabel Closing session to server IMC." -ForegroundColor Blue
    Write-Host ""
    }
# Write the collected data to CSV file.
$IMCRackServerInventory | Export-CSV -Path "UCS_IMC_RackServer_Inventory.csv" -NoTypeInformation -UseCulture
Write-Host "Script Finished." -ForegroundColor Green

[#2]

You will have to add a check if $IMCServerProcessors 3 and 4 are there and if not, then add a not available in those columns

[#3]

@Harm_VeenstraSo what you are saying is that I need to log into each server first, determine the max number of CPUs any one server has, then go and create the same number of properties for each server and for those servers with less CPUs, just fill the "extra" properties with "Not Available"?

[#4]

Looking at it again now... What is in your CSVImportFile?

[#5 · Accepted solution]

@JulianMilano 

You've got a couple of options, but first, let's look at your question on why you're only getting the first two when in your debugging, you're seeing higher counts (such as the four CPU example.)

Export-Csv constructs the header dynamically using the first object in the pipeline and doesn't alter it afterwards. If your first object is based on a 2-CPU processor then that's what your header will look like for all rows, and more importantly, all subsequent objects that come through the pipeline will only be checked for object properties matching the header, meaning even if entries for CPUs above 2 exist, Export-Csv will simply not look for them.

Example script

$Data = @(
    [PSCustomObject] @{
        Row1 = "Value1";
        Row2 = "Value2";
    },
    [PSCustomObject] @{
        Row1 = "Value1";
        Row2 = "Value2";
        Row3 = "Value3";
        Row4 = "Value4";
    }
)

$Data | Export-Csv -NoTypeInformation -Path "D:\Data\Temp\Forum\forum.csv";

Output

"Row1","Row2"
"Value1","Value2"
"Value1","Value2"

Conversely, if I reverse the order of the data rows, we see the header change to reflect it:

Output

"Row1","Row2","Row3","Row4"
"Value1","Value2","Value3","Value4"
"Value1","Value2",,

So, that's why you're not seeing your additional data when using Export-Csv.

The easiest way to combat this is to pre-stage a specified number of CPU properties on each object, and then during the iterations, assign the values to those properties.

For example, changing your existing lines 177 to 193, to something like this:

        # Get the rack server processor details.
        $IMCServerProcessors = Get-IMCProcessorUnit -Imc $IMCHandle

        # Pre-stage a fixed number of CPU columns (in this case, 4.)
        for ($index = 1; $index -le 4; $index++){
            Add-Member -InputObject $item -NotePropertyName "CPUID_$index" -NotePropertyValue $null;
            Add-Member -InputObject $item -NotePropertyName "CPUModel_$index" -NotePropertyValue $null;
            Add-Member -InputObject $item -NotePropertyName "OperState_$index" -NotePropertyValue $null;
            Add-Member -InputObject $item -NotePropertyName "Presence_$index" -NotePropertyValue $null;
            Add-Member -InputObject $item -NotePropertyName "SocketDesignation_$index" -NotePropertyValue $null;
            Add-Member -InputObject $item -NotePropertyName "CPUVendor_$index" -NotePropertyValue $null;
            Add-Member -InputObject $item -NotePropertyName "CPUIMC_$index" -NotePropertyValue $null;
            Add-Member -InputObject $item -NotePropertyName "Dn_$index" -NotePropertyValue $null;
            Add-Member -InputObject $item -NotePropertyName "Rn_$index" -NotePropertyValue $null;
        }

        # For multiple processors, cycle thru each object found.
        for ($index = 1; $index -le $IMCServerProcessors.Count; $index++)
        {
            $item."CPUID_$index" = $IMCServerProcessors[$index-1].Id;
            $item."CPUModel_$index" = $IMCServerProcessors[$index-1].Model;
            $item."OperState_$index" = $IMCServerProcessors[$index-1].OperState;
            $item."Presence_$index" = $IMCServerProcessors[$index-1].Presence;
            $item."SocketDesignation_$index" = $IMCServerProcessors[$index-1].SocketDesignation;
            $item."CPUVendor_$index" = $IMCServerProcessors[$index-1].Vendor;
            $item."CPUIMC_$index" = $IMCServerProcessors[$index-1].Imc;
            $item."Dn_$index" = $IMCServerProcessors[$index-1].Dn;
            $item."Rn_$index" = $IMCServerProcessors[$index-1].Rn;
        }

There are other alternatives but they come with additional complexity - more than is worth the small amount of benefit of not having superfluous CPU columns is worth.

At the end of the day, if you plan to use Export-Csv then your first pipeline object simply has to have present all the columns that can possibly feature in the resulting CSV, or else you are going to miss out on data.

Cheers,

Lain

[#6]

It's just a list of Cisco UCS CIMC server names and IMC IPs. The script logs into the CIMC management (also called iLO, DRAC, etc) and exports the server details.

[#7]

Fantastic explanation- thanks Lain!