Blog Post

Microsoft 365 Blog
9 MIN READ

Quickly Diagnose Issues with Email Threat Policies

alexhudish's avatar
alexhudish
Icon for Microsoft rankMicrosoft
Sep 17, 2024

The Microsoft 365 commercial support team resolves customer support cases and provides support to help you be successful and realize the full potential and value of your purchase. Our support services extend across the entire lifecycle and include pre-sales, onboarding and deployment, usage and management, accounts and billing, and break-fix support. We also spend a considerable amount of time working to improve the supportability of Microsoft 365 services to reduce the number of issues you experience as well as minimize the effort and time it takes to resolve your issues if they do occur. 

 

Today, we’re excited to share some insights on working with Exchange Online Protection (EOP) and Microsoft Defender for Office 365 (MDO). 

 

Exchange Online Protection (EOP) and Microsoft Defender for Office 365 (MDO) provide cumulative layers of email security that include multiple threat policies. Some organizations choose our quick, “set-and-forget” preset policies, while others choose to customize their email threat policies for different sets of users, groups, domains and business requirements.  

Determine your protection policy strategy. 

 

Which policies apply to which recipients?  

In Microsoft Customer Service and Support (CSS), we often hear from administrators who create custom policies and find it challenging to determine which threat policy applied to a user or a message, especially if the recipient is part of multiple groups or policies. Consistent and effective policy management can lower administrator overhead, confusion, and even security risks, (e.g. bad emails being delivered to users due to overrides, or good emails being blocked because of aggressive blocks.) With that, we’re happy to announce two new tools to help you diagnose policy issues quickly and efficiently! 

 

Introducing: Email Threat Policy Diagnostic for a Recipient 

Requirements: Network Message ID, Recipient address 

 

Run the Email Threat Policy Diagnostic as an administrator in any admin portal (Microsoft 365 Admin Center, Microsoft Defender XDR, Exchange Admin Center, Compliance portal, etc.).  

 

The quick link https://aka.ms/diagmdopolicy will 1) open the Microsoft 365 Admin Center and 2) prepopulate the Get Help field (“?”) with the diagnostic query. 

 

Provide a Network Message ID and a recipient address for the Email Threat Policy diagnostic to show which policies applied when the message was received, and what policies covered the recipient.  

 

Example 1: Testing Safe Links user exclusions 

Your organization has three Safe Links policies defined. Joe works in the Threat Intelligence department, which commonly requires access for testing malicious links from email messages in a virtual environment. You decide to exclude Joe from Custom and Built-in policies to skip Safe Links processing.  

Figure 1: Safe Links policies screen in the Microsoft Defender XDR portal

 

 

Figure 2: Safe Links policy details and exclusions in the Microsoft Defender XDR portal

Figure 3: Safe Links policy details and exclusions in the Microsoft Defender XDR portal

 

Upon further testing, Joe still sees Safe Links applied to email messages with malicious URLs. After you collect a Network Message ID from Joe’s last test message, run the Email Threat Policy diagnostic. In this example, we will use these two pieces of input: 

 
Network Message ID: 42715389-04ae-4577-d1a3-08dcbad6af8a 

Recipient email address: joe@contoso.com 

Figure 4: Email Threat Policy Diagnostic in Microsoft Defender XDR portal – input

 

Figure 5: Email Threat Policy Diagnostic in Microsoft Defender XDR portal – results.

 

From the results, you’ll learn that the Standard Preset Security policy applied to this message. This is because Standard and Strict preset security policies take precedence over any custom and built-in policies and apply to your entire organization. To learn more about policy order and processing, see https://aka.ms/mdoorder. 

 

Solution and validation: 

 

Since your organization requires a higher degree of customizations, you decide to turn the Standard Preset Security policy Off .

 

Now that you only have two policies remaining (Custom and Built-in), and Joe is excluded from both, new test messages go through bypassing Safe Links. 

 

Example 2: Testing why anti-malware policies fire on excluded attachments 

You have multiple malware filtering policies that block different file attachments. The Custom malware policy is your latest policy that blocks all media file types, such as .mov, .mp4 and .mp3 

Figure 6: Anti-malware policies screen in the Microsoft Defender XDR portal

 

Joe stopped getting voicemail messages. You know your voicemail provider uses an .mp3 file type and upon investigation, you find these messages are quarantined unexpectedly. You collect the Network Message ID and recipient address and run the Email Threat Policy diagnostic to verify which policy is applied to the message. 

 

Solution and validation: 

Since the custom policy was recently defined to block all media file types, you decide to modify the policy and remove .mp3 from the list of restricted file types. To confirm, you can run the diagnostics using the Network Message ID from the quarantine, provide Joe’s recipient address, and find out that the “Custom Malware policy” applies. 

 

Email Threat Policy Diagnostic in Microsoft Defender XDR portal – input and results.

 

Email Threat Policy Diagnostic in Microsoft Defender XDR portal – input and results.

 

 

 
Why Network Message ID (NMID)?  

A network message ID is a unique message ID value that persists across copies of the message that may be created due to bifurcation or distribution group expansion. Here’s what one looks like in message headers: 

 

X-MS-Exchange-Organization-Network-Message-Id: 185a3445-695c-464a-d44c-08dcb7d88102 

 

OR a different x-header that links to the same NMID value: 

 

X-MS-Office365-Filtering-Correlation-Id: 185a3445-695c-464a-d44c-08dcb7d88102 

 

Learn more about NMID. 

 

Notes:  

  1. When providing a recipient, use an Exchange Online (Microsoft 365) mailbox which received the message. If a message was sent to a group, trace the message to the individual recipient first, and then provide the recipient Network Message ID.  
  2. The diagnostic also works for outbound messages and similarly requires the Network Message ID and the recipient address. 
  3. In addition to threat policies applied to the message, this diagnostic can also be used to help you troubleshoot which inbound connector was used to receive the message. This information is available in extended message trace reports, but it is surfaced in the results for your quick reference, which is helpful if you’re using multiple connectors and inbound routing configurations.  

Tip: Other self-help diagnostics are available for Exchange Online, Outlook and Microsoft Defender for Office 365. While these diagnostics can't make any changes to your tenant without your consent, they offer insights into known issues and provide instructions to fix those issues quickly.  

 

Introducing: Threat Policy Checker PowerShell Script  
 
Requirements: No parameters are required to perform general inclusion logic checks. Provide a recipient address for the policies scoped to a particular user. 

 

Use the Threat Policy Checker Script to identify and resolve policy inconsistencies, and to ensure threat policies in your organization apply as intended. The script performs several checks to help you find inconsistencies in user membership and policy application without needing to provide a specific Network Message ID. If issues are found, the script provides guidance on how to resolve them. It can help with such questions as  

  • Are there confusing policies with conditions that lead to unexpected coverage or coverage gaps?  
  • Which threat policies apply to a recipient, or should have applied but did not? No actual detection or Network Message ID needed 
  • Which actions would be taken on an email for each policy matched?     

The script only runs in “Read” mode from Exchange Online and Microsoft Graph PowerShell. It does not modify any policies, and only provides actionable guidance for administrators for remediation. 

Quick link: https://aka.ms/mdopolicycheck  * 

 

Parameters and Use Cases 

 

MDOThreatPolicyChecker 

Run the script without any parameters to review all threat protection policies and to find inconsistencies with user inclusion and/or exclusion conditions. 

 

 

PS C:\Users\x\Desktop> .\MDOThreatPolicyChecker.ps1 
MDOThreatPolicyChecker.ps1 script version 24.08.02.1321 
Connected to EXO 
Session details 
Tenant Id: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa 
User: Joe@contoso.com 
No logical inconsistencies found! 

 

 

Script Output 1: 'No Logical inconsistencies found' message if the policies are configured correctly, and no further corrections are required. 

 

 

 

PS C:\Users\x\Desktop> .\MDOThreatPolicyChecker.ps1 

MDOThreatPolicyChecker.ps1 script version 24.08.02.1321  

Connected to EXO 

Session details 

Tenant Id: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa 

User: Joe@contoso.com  

Policy Custom antispam policy:  
        Type: Anti-spam Policy.  
        State: Enabled.  
        Issues:  
                -> Illogical inclusions of Users and Groups.  
                        The policy will only apply to Users who are also members of any Groups you have specified.  
                        This makes the Group inclusion redundant and confusing.  
                        Suggestion: use one or the other type of inclusion.

 

 

Script Output 2: Inconsistencies found in the antispam policy named 'Custom antispam policy', and consequent recommendations shown -- illogical inclusions as both users and groups are specified. This policy will only apply to the users who are also members of the specified group. 

 

-IncludeMDOPolicies 

Add the parameter -IncludeMDOPolicies to view Microsoft Defender for Office 365 Safe Links and Safe Attachments policies: 

 

 

PS C:\Users\x\OneDrive - Microsoft\Attachments\Desktop> .\MDOThreatPolicyChecker.ps1 -EmailAddress "Joe@contoso.com" -IncludeMDOPolicies 

MDOThreatPolicyChecker.ps1 script version 24.08.02.1321  

Connected to EXO  

Session details  

Tenant Id: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa  

User: Joe@contoso.com  

Connected to Graph  

Session details  

TenantID: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa  

Account: Joe@contoso.com  

Policies applied to Joe@contoso.com...  

Malware:  
        Name: Custom Malware Policy  
        Priority: 0  

Anti-phish:  
        Default policy  

Anti-spam:  
        Default policy  

Outbound Spam:  
        Default policy  

For both Safe Attachments and Safe Links:  
        Name: Standard Preset Security Policy  
        Priority: 0

 

 

Script Output 3: Parameters -EmailAddress and -IncludeMDOPoliciesEOP specified to validate Microsoft Defender for Office 365 Safe Attachments and Safe Links policies, on top of Exchange Online Protection policies. 

 

 

-ShowDetailedPolicies 

To see policy details, run the script with the -ShowDetailedPolicies parameter: 

 

 

PS C:\Users\x\Desktop> .\MDOThreatPolicyChecker.ps1 -EmailAddress "Joe@contoso.com" -IncludeMDOPolicies -ShowDetailedPolicies 

MDOThreatPolicyChecker.ps1 script version 24.08.02.1321  

Connected to EXO  

Session details  

Tenant Id: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa  

User: Joe@contoso.com  

Connected to Graph  

Session details  

TenantID: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa  

Account: Joe@contoso.com  

Policies applied to Joe@contoso.com...  

Malware:  
        Name: Custom Malware Policy  
        Priority: 0  

        Properties of the policy that are True, On, or not blank:  
                EnableFileFilter: True  
                FileTypeAction: Reject  
                FileTypes: ace apk app appx ani arj bat cab cmd com deb dex dll docm elf exe hta img iso jar jnlp kext lha lib library lnk lzh macho msc msi msix msp mst pif ppa ppam reg rev scf scr sct sys uif vb vbe vbs vxd wsc wsf wsh xll xz z txt  
                QuarantineTag: AdminOnlyAccessPolicy  
                RecommendedPolicyType: Custom  
                IsValid: True  
                Guid: ff6ba341-625a-4a0b-b32a-65e5625a6627  

Anti-phish:  
        Default policy  

        Properties of the policy that are True, On, or not blank:  
                Enabled: True  
                ImpersonationProtectionState: Automatic  
                EnableMailboxIntelligence: True  
                TargetedUserProtectionAction: NoAction  
                TargetedUserQuarantineTag: DefaultFullAccessPolicy  
                MailboxIntelligenceProtectionAction: NoAction  
                MailboxIntelligenceQuarantineTag: DefaultFullAccessPolicy  
                TargetedDomainProtectionAction: NoAction  
                TargetedDomainQuarantineTag: DefaultFullAccessPolicy  
                AuthenticationFailAction: MoveToJmf  
                SpoofQuarantineTag: DefaultFullAccessPolicy  
                EnableSpoofIntelligence: True  
                EnableViaTag: True  
                EnableUnauthenticatedSender: True  
                HonorDmarcPolicy: True  
                DmarcRejectAction: Reject  
                DmarcQuarantineAction: Quarantine  
                RecommendedPolicyType: Custom  
                IsValid: True  
                Guid: bf512d2b-bc3b-4843-a01c-433a02fd6bab  

Anti-spam:  
        Default policy  

        Properties of the policy that are True, On, or not blank:  
                QuarantineRetentionPeriod: 15  
                TestModeAction: None  
                MarkAsSpamEmptyMessages: Test  
                MarkAsSpamBulkMail: On  
                MarkAsSpamNdrBackscatter: On  
                IsDefault: True  
                HighConfidenceSpamAction: Quarantine  
                SpamAction: Quarantine  
                BulkThreshold: 7  
                ZapEnabled: True  
                InlineSafetyTipsEnabled: True  
                BulkSpamAction: MoveToJmf  
                PhishSpamAction: MoveToJmf  
                IntraOrgFilterState: Spam  
                HighConfidencePhishAction: Quarantine  
                RecommendedPolicyType: Custom  
                SpamQuarantineTag: Notification policy  
                HighConfidenceSpamQuarantineTag: Notification policy  
                PhishQuarantineTag: DefaultFullAccessPolicy  
                HighConfidencePhishQuarantineTag: AdminOnlyAccessPolicy  
                BulkQuarantineTag: DefaultFullAccessPolicy  
                IsValid: True  
                Guid: 191b78dc-9221-4a2c-b51c-208a186e931a  

Outbound Spam:  
        Default policy  

        Properties of the policy that are True, On, or not blank:  
                IsDefault: True  
                ConfigurationType: HostedOutboundSpamFilterPolicy  
                ActionWhenThresholdReached: BlockUser  
                RecommendedPolicyType: Custom  
                AutoForwardingMode: On  
                Guid: 5a6504d0-b3e8-4dda-8060-94e03f9813c6  
                IsValid: True  

For both Safe Attachments and Safe Links:  
        Name: Standard Preset Security Policy  
        Priority: 0  
        Preset policy settings are not configurable but documented here:  
                https://learn.microsoft.com/en-us/defender-office-365/recommended-settings-for-eop-and-office365#microsoft-defender-for-office-365-security

 

 

Script Output 4: Parameters -EmailAddress,-IncludeMDOPolicies, and -ShowDetailedPolicies list all EOP and MDO policies applied to a user and their full details. 

 

* Please read the disclaimer when running the script. The scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Please use GitHub to report issues to the developers. 

 
We hope these tools help you evaluate and diagnose issues related to the order and precedence of email protection policies better. Do you have questions or feedback about Microsoft Defender for Office 365? Engage with the community and Microsoft experts in theDefender for Office 365 forum. 

 

Important resources: 

Email Threat Policy Diagnostic 
Threat Policy Checker Script 
Get started with Microsoft Defender for Office 365 
Order and precedence of email protection 
Preset security policies  
Anti-spam message headers 
Message trace in the new EAC in Exchange Online (NMID) 
Self-help diagnostics for issues in Exchange Online and Outlook 

 

 Alex Hudish is a Senior Supportability Program Manager in the Customer Service & Support (CSS) Supportability Team focused on Security and Microsoft Defender for Office 365 

 

Ross_Parkel is a Senior Technical Support Escalation Engineer in Customer Service & Support (CSS) focused on Security and Microsoft Defender for Office 365.  

 

Mithun_Rathinam is a Senior Technical Support Escalation Engineer in Customer Service & Support (CSS) Beta Team focused on Security and Microsoft Defender for Office 365 

 

 

Marc Nivens is a Senior Technical Support Embedded Escalation Engineer on the Microsoft Defender for Office 365 Team. 

 

 

Updated Oct 15, 2024
Version 2.0
  • JonasBack's avatar
    JonasBack
    Steel Contributor

    This is all great, especially for those customers without MDO P2, but shouldn’t this be added to Threat Explorer which is the go-to tool for use troubleshooting email delivery issues? I can’t recall that you can tell which policies are applied there?

     

  • Thank you for your feedback! I’m from the MDO PM team, and we are aware of your request. We are actively discussing it internally. We appreciate your patience and encourage you to keep an eye on the Microsoft 365 Roadmap and the Microsoft Defender for Office 365 What’s New page for any updates and announcements.

  • chrislehr's avatar
    chrislehr
    Copper Contributor

    Agree with Jonas ^^ if this is needed, maybe develop it into threat explorer/RT detections.

     

    OTOH, I have the order and precedence article bookmarked and mostly committed to memory and this is a step in the right direction for many admins, hope you can get this into threat explorer or some sort of "policy whatif" tool like Conditional Access has.