Forum Discussion
csv file comparison
Hi All,
I am comparing two csv files based on USERNAME column of both files, kindly see my script below,
Compare-Object $SourceFile $TargetFile -Property "USERNAME" -PassThru | Where-Object {$_.SideIndicator -eq "=>"
this is working perfectly fine from the past few years.
Till last week both files had around 70,000 records in each, so this comparison took 2 hours to complete,
Recently these file(source, target) records has grown like 8,00,000, now this comparison is taking around 48 hours and some times i am getting below error.
+ CategoryInfo : NotSpecified: (:) [Where-Object], OutOfMemoryException
+ FullyQualifiedErrorId : System.OutOfMemoryException,Microsoft.PowerShell.Commands.WhereObjectCommand
Kindly suggest is there any other way we can do this comparison, so that we can mitigate this exception and too much of time ??
Response would be appreciated.
I saw in old forum something related to increase the memory,
https://social.technet.microsoft.com/Forums/en-US/ff516976-78a4-49b0-97e8-f443702eaf50/powershell-script-failure-systemoutofmemory-exception?forum=scogeneral
execute the command:
Get-Item WSMan:\localhost\Shell\MaxMemoryPerShellMB
by default it is 2147483647
check the link bellow, it have a solution to increase it.
https://stackoverflow.com/questions/28503765/powershell-gives-get-content-exception-of-type-system-outofmemoryexception-w?rq=1
