Forum Discussion
array limitations?
Are there limitations to array sizes? I'm trying to get the line count of very large .txt files ( so far the largest file is 782MB) and also look for duplicates. The script works with small files a...
First thing I have noticed here is that you are running
Get-Content $sCARDHIST1twice, you would save time by running this into a variable once and then use that variable for your
$Content = Get-Content $sCARDHIST1
# get the count
$Content.count
# and then the duplicate rows
$Content | Group-Object ...
As the file numbers and sizes grows though you will encounter more and more performance problems, loading 782Mb of a text file into memory and then sorting each row will not be a fast process in PowerShell. Depending on row size that is a _lot_ of rows