I am struggling to understand how users are handled across Office365, Exchange, Teams, SharePoint, Planner etc. When a user leaves our organisation, in Exchange on-line I block their login, remove them from all groups, in their profile in the section Show in global address list I select No and convert their mailbox to a shared mailbox so the person taking over their role has access to their historical emails.
However, this is causing issues in Planner because the users who no longer work for our organisation are showing up, as non-members, and are able to be selected.
The only way to prevent them from being available seems to be to delete the user from Exchange. I have even tried finding their profile in SharePoint and deleting it but that had no effect either. Apart from privacy I really don't want those users to show up as a choosable option. Is there any way of removing them from the list without deleting their account (and so their emails) from our system?