Forum Discussion

beno_low's avatar
beno_low
Copper Contributor
Sep 08, 2020

Script Optimization

Hello Peeps,

 

I have a script that read in 2 files.

 

File A contains FDQN

File B contains HOSTNAME + ENV

 

The pseudo code is as follows:

 

For each line in File A

     Split HOST from FDQN

             For each line in File B

                    if match HOST to HOSTNAME, set env = ENV

 

Using the above logic, i have 2 loops, outerloop A and innerLoop B, the time taken is close to 8 hours, as the number of entries is close to 27000 in each file (27000 x 27000 loops).

How can i make this faster?

2 Replies

  • Manfred101's avatar
    Manfred101
    Iron Contributor

    beno_low 

    Hello beno_low,

    You didn’t attach a example input file, so I was not able to test the script below. But I think you can speedup your script al lot by using your RAM instead of reading and writing to disk in each loop. Try something like the code below. I moved the Get-Content $args[1] in line 25 and load that into the $cmdb_content var. The Out-file at line 94 is also moved outside your primairy loop. 

     

    WARNING: You are using $env in your script, this is not a best practice since Powershell is also using $Env to store enviroment variables! More info: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables?view=powershell-7

     

    ##############################################################################
    #   Filename : gen_host_env.ps1
    #   Usage : .\gen_host_env CS_host CMDB_host
    #   Both input files must be in text format
    #   Output : 
    
    
    
    $Final_Result = @()
    #$Header = 'CMDB_Host,CMDB_Env,CS_Host,CS_Env,CS_FQDN,L1_Domain,L2_Domain'+"`r`n"
    #Write-Host $Header
    'CMDB_Host,CMDB_Env,CS_Host,CS_Env,CS_FQDN,L1_Domain,L2_Domain' | Out-File -Encoding ascii -Append .\crowdstrike_host.txt
    
    $cmdb_content = Get-Content $args[1]
    $output = @()
    
    foreach ($fqdn in (Get-Content $args[0])){
    	
    	$comp = $fqdn.Split(".")
        $hname = $comp[0]
        $l1 = $comp[1]
        $l2 = $comp[2]
        $l3 = $comp[3]
        $l4 = $comp[4]
        $l5 = $comp[5]
        $l6 = $comp[6]
        
        foreach ($cmdb_entry in $cmdb_content) {
            $cmdb_split = $cmdb_entry.Split(",")
            $cmdb_host = $cmdb_split[0]
            $cmdb_env = $cmdb_split[1]
            #Write-Host $cmdb_host
    
            if ($hname -imatch $cmdb_host) {
                $env_frm_cmdb = $cmdb_env
                break
            }
            else {
                $cmdb_host = 'NO match'
                $env_frm_cmdb = 'undetermined'
            
            }
        } 
    
        #Write-Host $hname
        #Write-Host $l1
     
     #### First Part of FQDN match criteria hostname.prod|pat|dev|.xyz###########     
        if ($l1 -match 'prod'){
            $env = 'prod' }
        
        elseif ($l1 -match 'splunk'){
            $env = 'prod' }
    
        elseif ($l1 -match 'corp'){
            $env = 'prod' }
                    
        elseif ($l1 -match 'dev') {
            $env = 'dev'}
        
        elseif ($l1 -match 'englab') {
            $env = 'dev'}
    
        elseif ($l1 -match 'sit') {
            $env = 'sit'}
    
        elseif ($l1 -match 'pat') {
            $env = 'pat'}
      
      #### Second Part of FQDN match criteria hostname.abc.prod|pat|dev|.xyz###########  
        elseif ($l2 -match 'prod'){
            $env = 'prod' }
            
        elseif ($l2 -match 'dev') {
            $env = 'dev'}
    
        elseif ($l2 -match 'sit') {
            $env = 'sit'}
    
        elseif ($l2 -match 'pat') {
            $env = 'pat'}
    
    ############ Hadoop Server match env starts here #####################
        elseif ($l2 -match 'hadoop') {
            if ($l1 -match 'c01') { $env = 'prod' }
            elseif ($l1 -match 'c02') { $env = 'dev' }
            elseif ($l1 -match 'c03') { $env = 'pat' }
            elseif ($l1 -match 'c2#') { $env = 'sit' }
            elseif ($l1 -match 'c5#') { $env = 'sit' }
        }
    
        else {
            $env = 'undetermine'}
    
        $Final_Result += $cmdb_host+","+$env_frm_cmdb+","+$hname+","+$env+","+$fqdn+","+$l1+","+$l2+"`r`n"
        #Write-Host $Final_Result
        $output += $cmdb_host+","+$env_frm_cmdb+","+$hname+","+$env+","+$fqdn+","+$l1+","+$l2 
        }
    
    $output | Out-File -Encoding ascii -Append .\crowdstrike_host.txt

     

    Good luck!

    Grtz, 

    Manfred de Laat

    • beno_low's avatar
      beno_low
      Copper Contributor

      Manfred101 

       

      Thanks, Manfred. Yes, it helps to reduce the time from 8 hours down to 2 hours approximately. 

Resources