Deleting a Team from within Teams client UI seems to leave the associated SharePoint group site

Further investigation shows for our E2 tenant, when you delete a Team via the Teams App, the O365 Group gets deleted, but the corresponding SharePoint site is given a LockState of "No Access", so even if you have permission to the site, it refuses to display in the browser. To see all these sites on your tenant, use the following PowerShell: 

get-SPOSite | Select URL, LockState | Where lockstate -eq 'NoAccess'

To unlock the site, so you can browse it again, use this:

set-SPOSite -identity <Site URL> -lockstate "unlock"

And if you want to then delete it:

Remove-SPOSite -identity <Site URL>

Hope that helps