Possibility of Script that works with a load of data loop by loop - or from time to time

Hey there!

In this case if you are sure the connection time limit is 30 minutes, I recommend using something that measures time.
This would be my approach:

- Create a PowerShell runspace and instance to run the workload asynchronously.
- Start a System.Diagnostics.StopWatch
- Enter a while loop that checks the execution of the PowerShell instance, and times out after 30 minutes.

However, there is a chance of your script terminate the execution in the middle of an item processing, which in this case it's better to use item count.

Here's a sketch. Mind that this was not tested, and you need to adjust to your needs.

$bigList = @(<# Your 72,000 items list #>)

# Creating a Runspace Pool to hold our PowerShell 'threads'.
$pool = [runspacefactory]::CreateRunspacePool()
$pool.ApartmentState = 'STA'
$pool.Open()

# Create a synchronized hash table to exchange data between the current and new threads.
$messenger = [hashtable]::Synchronized(@{
Processed = [System.Collections.ArrayList]@()
Pending = $null
})

while ($messenger.Processed.Count -lt $bigList.Count) {

# Separate the items you already processed. You might have to adjust this according
# to the complexity of your object list.
$messenger.Pending = $bigList | Where-Object { $_ -notin $messenger.Processed }

# Creating the PowerShell instance.
$powerShell = [powershell]::Create()
$powerShell.RunspacePool = $pool
[void]$powerShell.AddScript({

param ($Messenger)

foreach ($item in $Messenger.Pending) {
<#
Your logic goes here. We used the messenger to retrieve the pending items to process.
After you finish processing each item, add it to the 'Processed' property from the
messenger:

$messenger.Processed.Add($item)
#>
}
})

# Passing the messenger as a parameter.
[void]$powerShell.AddParameter('Messenger', $messenger)

# Starting the instance asynchronously
$handle = $powerShell.BeginInvoke()


# Here is your main while loop. You can wait 'X' amount of time, or 'X' itens processed, while monitoring
# the messenger processed list. If you choose time, use a StopWatch.

$stopWatch = [System.Diagnostics.Stopwatch]::StartNew()
do {

# Check the instance execution.
if ($powerShell.InvocationStateInfo.State -ne 'Running') {
break
}

} while ($stopWatch.Elapsed.TotalMinutes -le 30)

# Dispose of the PowerShell instance. This is important to avoid resource leak.
# If you care about the outcome, assign a variable instead of [void]~.
[void]$powerShell.EndInvoke($handle)
$powerShell.Dispose()
}

# Clenup the pool as well
$pool.Dispose()