Skype for Business chat within Outlook on the web showing old contacts

Just had a look at the very snappy sounding Skype for Business chat within Outlook on the web. It does show quite a few contacts which have left our organisation months ago and are not listed within my desktop version. How can these be removed or better how can we make sure they don't show in the first place? Thanks