New Outlook notifications when the client is not launched

Recently, users started to receive toast notifications for new email when New Outlook wasn't running. When checking running processes, olk.exe was not running, however, Edge had a few pages loaded (none of them were Outlook for the Web). This is really annoying because users receive 2 notifications and the notification from New Outlook seems to arrive faster, resulting in users clicking on it and launching New Outlook. New Outlook is not set to auto-start and this behaviour is relatively recent.

Any ideas on what controls it and how to address it without having to disable notifications in New Outlook or Win 11? The idea is that users are testing the product, and need the notifications when using it.