Add app through Menu-> Apps->Install this site as an app
Now uninstall this site (doesn't matter through Edge or Apps and Features). The app successfully uninstalled from Edge
but remain in Start menu (Recently added)
and still exists in Apps and Feature list (here I've try this two times and it stuck consequently, so probably it would keep numbering (2)/(3)/(4) etc.)
In order to delete those entries from Apps and Features the only workaround I've found is deleting from registry.
@Joshua1605 thanks for reporting this one. We are working in this very area currently, so I guess I'm not surprised there are some bugs in there. I've made sure it's logged internally so we can take a look.