I'm part of two Teams teams with different email addresses. I use the full Teams application for the first email, and when I get a new Chat message, a badge shows up on my Windows taskbar on the Teams icon to alert me.
For the second email, I created an Edge browser "app" (i.e. Settings / Apps / Install this site as an app), and while it puts a Teams icon on the taskbar, and essentially works like the full Teams application, I never get taskbar badges when new Chats pop up. Because of this, I have to keep opening the window every few minutes to see if anyone's trying to chat. :(
Is there a way to to get a badge to show? I can confirm that I've got all Windows, Teams, and Edge Notifications turned on as much as possible.