Still getting notifications for channels I've turned them off for - how to disable?

I have the exact same problem. I turned off all notifications and yet I'm still getting notifications. This is for an alerts channel that I followed (back when you had to follow a channel to be notified).

All new posts - off

Channel mentions - off

I think it's a bug. Has anyone raised it with Microsoft support?