You gave me a bit inspiration and so I looked into this issue a bit... I'm not 100% sure it works in all cases and it also only works on Modern pages... but I explored the idea of using Search API to find out if web parts are being used in an environment, and it worked for me.
@Beau__Cameron Love the approach. For our internal deployment at MS, the path we were going down was to use App Insights to track where our web parts were being used, how often, etc. Our second release was going to also include errors, perf, etc. Some screens of what @darinious built for me: