Bulk delete all the old jobs from the batch account

You need -MaxCount 0 to delete more than 1000 jobs. 

$jobsForDelete = Get-AzBatchJob -BatchContext $batchContext -MaxCount 0 | Where-Object {$_.CreationTime -lt "2024-05-01"}

And if you have a lot of jobs you probably want to remove in parallel:

$jobsForDelete | ForEach-Object -Parallel {

    Write-Host $_.Id

    Remove-AzBatchJob -Id $_.Id -BatchContext $using:batchContext -Force

} -ThrottleLimit 10