Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community

Standard Security Policy flagging too many emails as "Potential Phishing"

Iron Contributor

We decided to enable the Standard Security Policy for Defender on our Microsoft 365 tenant, and immediately noticed that it was quarantining way too many emails that it flagged as either Phishing or High Confidence Phishing (mostly automated notices from cloud services like Asana, Klaviyo, etc.).   These are emails that would easily be allowed through any other mail scanning firewall I've used in the past.   I'm now concerned about using Defender's "Standard Security Policy" level for Defender, for fear that it's going to have my users missing emails that should easily be passing through, because Defender moved them to Quarantine or Junk.    Is there a way to modify the aggressiveness levels for the Standard Security Policy?

9 Replies

Hey , thanks for your message!

 

Firstly, it's great to hear you're using presets, they are an awesome way of ensuring you're up to date with the latest security recommendations, I'm sorry to hear you're finding false positives (FPs) but we can for sure get to the bottom of it.

 

Secondly, have you tried doing admin submissions for the emails? this will do several things, it will tell us we got our verdict wrong, It will also let you know the results of our analysis. - you will also get the ability during the process to add the part of the message which caused it to be blocked (URL, Sender, Attachment, Spoof) to our Tenant Allow/Block List so it's overridden in the future while we work on getting the reason for the issue fixed.

 

It's also important to note that sometimes, emails are sent in a way which is non-compliant, so actually instead of overriding decisions, the correct course of action is to fix the underlying issue. - for the examples you have, could you let me know the detection technology which lead to the phish verdict and I may be able to help point you in the right direction. (we also document them all here at aka.ms/emailtech)

 

Thanks

 

Ben.

I've definitely been submitting them during release, so the system can try to learn from the email's fingerprint that they're valid.
Please also ensure these are admin submissions, not just user submissions :) - more details here.

https://learn.microsoft.com/en-us/microsoft-365/security/office-365-security/submissions-admin?view=...

@OneTechBeyond in addition to the suggestions from @Ben_Harris, can you share a bit about your mail flow design? Does your MX record point to EOP/MDO, or does it point to a third party service? If it points to EOP/MDO, is there any additional routing (e.g., out to a third party, to an on-prem service) after the message is received, but before they are delivered to mailboxes?

Hi Paul. No EOP/MDO. it's 100% native Microsoft's internal services.

@OneTechBeyond, thanks for that note, but admittedly I am a little confused. 
When you say "No EOP/MDO. it's 100% native Microsoft's internal services.", do you mean you are using EOP (Exchange Online Protection) & MDO (Microsoft Defender for Office 365) exclusively, or you're not using them at all? 

It sounds like you're only using EOP & MDO, in which case, follow Ben's recommendations - keep sharing the false positives with us (via admin submissions) and check out the https://aka.ms/emailtech article to get additional insight as to why we marked items as we did (perhaps something is marked due to a user's Blocked sender list, or the sender's service failed DMARC alignment). 

If worse comes to worst, while you cannot adjust the sensitivity of the preset policies, you can create your own policies and adjust as necessary. The Microsoft recommendations for EOP and Defender for Office 365 security settings aticle gives the settings in each policy which align to the default settings (which are pretty low, all things considered), the Standard preset policy, and the Strict preset policy. I'd recommend starting with mirroring the Standard preset policy settings and making adjustments as necessary (based on what you find from the emailtech article), but remember Ben's note that the preset policies are updated automatically to keep up with any changes in recommendations/best practices, but custom policies are not. You would need to review your custom policies occasionally to see how they compare to our recommendations using Configuration Analyzer.

Sorry, yes I meant that we’re using Defender for M365 and EOP with no external spam filtering/mx relaying services.
So these notices from relatively established service providers are ending up categorised as phish, hmm? That is not where I would expect a false positive problem to appear, unless the provider has generally acknowledged problems.

Are the notices sent using your domain, the providers' domains or a third-party domain? If they are breaking the SPF or DMARC policies published for their envelopes then you would see a lot of phishing detection.

Look for the Authentication-Results header in example cases for each domain. If these show problems then you have to sort out your published policies or kick the provider or third party accordingly.

Look further down the headers for the Forefront lines. Are they showing a high SCL or BCL? If so, possibly your provider is being irresponsible either with notices (and is being reported for spam) or has rubbish shared hosting, in which case anything could happen.

Remember that if legitimate spoofing is at the heart of your problem, you can add the domains in question to the relevant part of your anti-phishing policy. It is better and safer to sort out delivery problems properly, but this can be the only solution for some components such as the Mailbox Intelligence Agent.

Could you change phishing sensitivity? Yes, but I would not recommend it. If you really have to go that way, could all of these lost notices instead be sent to a small subset of addresses, or an discreet shared mailbox? If so, you can add a higher-priority anti-phishing policy that just applies to that subset, then drop the sensitivity on that.