However, I would need to be a site collection administrator of all the site collections. And it sounds extreme.
How did you solve it in your tenant? Do you have a user who is an admin of all the site collections?
Should I grant access as a site coll admin (Set-SPOUser -site $SiteCollURL -LoginName $SiteCollectionAdmin -IsSiteCollectionAdmin $True) before checking Get-SPOExternalUser and then remove the user?
Also, there is another pitfall. When the permissions of an external user are deleted from a site collection, we can still see it with Get-SPOExternalUser because the user stays in the SharePoint UserInfo List.