MicrosoftVNJoe all the things you mentioned are customizable. I've just come through months of working with and tuning EOP for a large client whose MX records point to on-premises and who have a to-be-retired 3rd party spam gateway near-passively there, then Exchange on-premises, and then to EXO/EOP where the mailboxes are. It's actually two distinct/separate on-premises setups like that, which merge into one tenant (2 hybrids). If you can imagine, we had to make use of Enhanced Filtering for Connectors to make sure EOP's SPF tests would happen against the correct IP addresses. We also had to stop using on-premises transport rules to add an 'external email' disclaimer and let EXO so that instead, in order to preserve the DKIM body hash so EOP's DKIM tests wouldn't fail erroneously.
We've rolled out quarantine notifications to 40K+ mailboxes.
We got to a point where we had to take the time to submit many samples to MS support because we couldn't tell what else we could do to prevent some false positives. They were able to find some Machine Learning gone wrong which they corrected.
Apart from that one glitch, it truly does seem like everything works great. There are lots of false positives in EOP but I'd be willing to bet a huge portion of them are due to problems that should be corrected rather than EOP lowering it's guard. I think that is the point that Microsoft can't just outright say, but it is important for vendors to try and push the world to be better at not only their own SPF/DKIM/DMARC implementations, but also proper infrastructure design and configuration.
The setting for high confidence phish going to quarantine and not letting users release those is a smart default. Most phishing attacks include links that are not detectable by Safe Links as malicious initially (i.e., lay dormant for some time). Users are horrible at NOT getting phished. So it makes sense to protect against those emails by default and then let customers dial it back from there. If the opposite were the case, you'd have 10X as many people here telling MS they're not protecting their paying customers well enough. So it's a lose lose for MS, but they picked the correct side in being safer. If an important message is missed, it should at least be resendable. If it is a very important email then everything under the sun should have been done by both the sender and the recipient orgs to ensure deliverability.
That last point is the most important to consider. If certain emails are very important not to miss, those emails (the messages and attachments themselves) and the sending/receiving infrastructures had better be setup properly. If they're getting caught as high confidence phish, they are flawed in some way (or there's a glitch which MS Support should be able to address). The right thing to do is quarantine them.
