After installing a website as an app, the app is automatically launched in a new window as expected. However, when you pin it to the taskbar, the app icon (favicon) in the taskbar is changed to the edge icon. You have to unpin and then pin again to get the appicon/favicon back in the taskbar.
I'm also running into this problem. Small detail but this is one of my favorite features of the new browser outside of the favicon bug! I really like running some of my web pages as individual apps and not just a regular browser window.
If you open the Start menu and find your recently installed Edge app in the Apps list, by right-clicking and choosing the "Pin to start" option from this Start menu screen, the app will show the correct favicon on the taskbar even when it isn't open.
It seems to only change the favicon when you choose the option from inside the browser.
@CodeDJ thanks so much for this feedback. Yes, I repro this as well. We are investigating the issue currently. The easiest workaround is to right click to see the menu, then dismiss the menu without making any choices, then right click to see the menu again. This second time, you should see the expected icon and "Pin to Taskbar" should work.