Forum Discussion
Script Optimization
Hello Peeps,
I have a script that read in 2 files.
File A contains FDQN
File B contains HOSTNAME + ENV
The pseudo code is as follows:
For each line in File A
Split HOST from FDQN
For each line in File B
if match HOST to HOSTNAME, set env = ENV
Using the above logic, i have 2 loops, outerloop A and innerLoop B, the time taken is close to 8 hours, as the number of entries is close to 27000 in each file (27000 x 27000 loops).
How can i make this faster?
2 Replies
- Manfred101Iron Contributor
Hello beno_low,
You didn’t attach a example input file, so I was not able to test the script below. But I think you can speedup your script al lot by using your RAM instead of reading and writing to disk in each loop. Try something like the code below. I moved the Get-Content $args[1] in line 25 and load that into the $cmdb_content var. The Out-file at line 94 is also moved outside your primairy loop.
WARNING: You are using $env in your script, this is not a best practice since Powershell is also using $Env to store enviroment variables! More info: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables?view=powershell-7
############################################################################## # Filename : gen_host_env.ps1 # Usage : .\gen_host_env CS_host CMDB_host # Both input files must be in text format # Output : $Final_Result = @() #$Header = 'CMDB_Host,CMDB_Env,CS_Host,CS_Env,CS_FQDN,L1_Domain,L2_Domain'+"`r`n" #Write-Host $Header 'CMDB_Host,CMDB_Env,CS_Host,CS_Env,CS_FQDN,L1_Domain,L2_Domain' | Out-File -Encoding ascii -Append .\crowdstrike_host.txt $cmdb_content = Get-Content $args[1] $output = @() foreach ($fqdn in (Get-Content $args[0])){ $comp = $fqdn.Split(".") $hname = $comp[0] $l1 = $comp[1] $l2 = $comp[2] $l3 = $comp[3] $l4 = $comp[4] $l5 = $comp[5] $l6 = $comp[6] foreach ($cmdb_entry in $cmdb_content) { $cmdb_split = $cmdb_entry.Split(",") $cmdb_host = $cmdb_split[0] $cmdb_env = $cmdb_split[1] #Write-Host $cmdb_host if ($hname -imatch $cmdb_host) { $env_frm_cmdb = $cmdb_env break } else { $cmdb_host = 'NO match' $env_frm_cmdb = 'undetermined' } } #Write-Host $hname #Write-Host $l1 #### First Part of FQDN match criteria hostname.prod|pat|dev|.xyz########### if ($l1 -match 'prod'){ $env = 'prod' } elseif ($l1 -match 'splunk'){ $env = 'prod' } elseif ($l1 -match 'corp'){ $env = 'prod' } elseif ($l1 -match 'dev') { $env = 'dev'} elseif ($l1 -match 'englab') { $env = 'dev'} elseif ($l1 -match 'sit') { $env = 'sit'} elseif ($l1 -match 'pat') { $env = 'pat'} #### Second Part of FQDN match criteria hostname.abc.prod|pat|dev|.xyz########### elseif ($l2 -match 'prod'){ $env = 'prod' } elseif ($l2 -match 'dev') { $env = 'dev'} elseif ($l2 -match 'sit') { $env = 'sit'} elseif ($l2 -match 'pat') { $env = 'pat'} ############ Hadoop Server match env starts here ##################### elseif ($l2 -match 'hadoop') { if ($l1 -match 'c01') { $env = 'prod' } elseif ($l1 -match 'c02') { $env = 'dev' } elseif ($l1 -match 'c03') { $env = 'pat' } elseif ($l1 -match 'c2#') { $env = 'sit' } elseif ($l1 -match 'c5#') { $env = 'sit' } } else { $env = 'undetermine'} $Final_Result += $cmdb_host+","+$env_frm_cmdb+","+$hname+","+$env+","+$fqdn+","+$l1+","+$l2+"`r`n" #Write-Host $Final_Result $output += $cmdb_host+","+$env_frm_cmdb+","+$hname+","+$env+","+$fqdn+","+$l1+","+$l2 } $output | Out-File -Encoding ascii -Append .\crowdstrike_host.txtGood luck!
Grtz,
Manfred de Laat
- beno_lowCopper Contributor
Thanks, Manfred. Yes, it helps to reduce the time from 8 hours down to 2 hours approximately.