For loop for all files in get-childitem one file at a time

[Marcus535]

Hi all

I have written a script that successfully convert and extract just a few of key/value pairs and write that to a. csv file. That part works nice and it could run for a small numbers of files but I now need to scale this to handle 4000-6000 files at 1 to 500kb sizes. Powershell ISE yams when I try because I believe it takes all files into memmory and then process it. I just want one file being processed at a time so my out folder starts growing instantly.  Without for loop on line three it works for a few files but 20 files seems to much, and I need to process alot of files. This script seems to work in somesort but allways writes to same filename file size growing and the becomes 0 makes me think it writes one loop at a time but as I said allways to same filename and loops never stop.

As you probably can see on the code I'm pretty new to powershell and scripting is just my leisure.

$array = @()
$files = Get-ChildItem -Path C:\test\tu\*.json
for ($i=0;$i -lt $files.count ;$i +1){
foreach ($f in $files){
$outname = (Split-Path -$files -Leaf).split(".")[0]+".csv"
$response = Get-Content $files -Raw | ConvertFrom-Json
$entity = $response.entity
foreach($i in $entity){
foreach ($i in $entity){
$array += New-Object psobject -Property @{'datum' = $i.trip_update.trip.start_date; 'entity_id' = $i.id; 'trip_id' = $i.trip_update.trip.trip_id; 'vehicle_id' = $i.trip_update.vehicle.id}

$array |ConvertTo-Csv -NoTypeInformation |Out-File c:\test\out\$outname
}}}}

[#2]

Hi all

I come that far now that I arrange files in an array and the script now makes on file for each source file however the data seems to be appended filesizes increase by as much as the filesize would be 6-10kb. I hope anyone can help me out.

$array = @()
$files = @(Get-ChildItem -Path 'C:\test\tu\*.json')
for ($ii=0;$ii -lt $files.Count ;$ii ++){
foreach ($f in $files[$ii]){
$outname = (Split-Path -$f -Leaf).split(".")[0]+".csv"
$response = Get-Content $f -Raw | ConvertFrom-Json
$entity = $response.entity
foreach ($i in $entity){
$array += New-Object psobject -Property @{'datum' = $i.trip_update.trip.start_date; 'entity_id' = $i.id; 'trip_id' = $i.trip_update.trip.trip_id; 'vehicle_id' = $i.trip_update.vehicle.id}
}}
$array |ConvertTo-Csv -NoTypeInformation |Out-File c:\test\out\$outname
}