Exchange anti-spam myths revealed
Published Nov 13 2009 03:43 PM 76.4K Views

In Microsoft CSS (Customer Service and Support) we deal with many anti-spam and email security questions. While the anti-spam features that come out of the box with Exchange 2007 provide a robust level of protection against unwanted garbage in your inbox, there is still a lot of confusion out there as to how all the parts work together. The purpose of this post is to dispel some misconceptions about the E2007 AS features (where applicable differences introduced in Exchange 2010 will be pointed out as well). I present you the top 6 SMTP anti-spam myths - revealed! (drum roll)

Myth 1: Creating a hub transport rule to set the SCL will affect the behavior of SCLDeleteThreshold and SCLRejectThreshold.

This myth applies particularly to the case of Hub server role with anti-spam agents installed. While it is fine to install the anti-spam agents on a hub transport server, expecting that a hub transport rule with "set the spam confidence level to value" action will influence the content filter delete/reject/quarantine is false.

  • This is a misconception due to where in the transport pipeline the content filter agent (which does the actual deleting/rejecting) fires. If we run Get-TransportPipeline we will see that the content filter agent fires at OnEndOfData (EOD) while the transport rules fire during the OnRouted stage.

Since the Transport Rule Agent fires after the Content Filter Agent (CFA) anything the rule action does will have no effect on CFA behavior.

Conversely while this would work on an Edge server anti-spam solution due to the location in the pipeline where the Edge rule fires this actually leads me to Myth 1b.

Myth 1b: Setting an edge rule to inspect SCL value that the content filter sets will work out of the box.

Unlike Hub Transport Rules Agent, Edge Transport Rules Agent fires BEFORE the content filter.

So if we leave the default settings with Edge rules firing first we have nothing to inspect or act on since the CFA has not gotten the message yet.

Your best bet for using rules that operate in conjunction with content filtering is to run content filtering on an Edge role and the Transport Rules on the Hub role.

Myth 2: You need the CFA to move items to junk mail folder.

Unlike SCLDeleteThreshold and SCLRjectThreshold, SCLJunkThreshold is a store setting. It is a combination of a hidden rule in the client's inbox when junk mail is enabled and the value of the SCLJunkThreshold attribute on the OrganizationConfig and/or mailbox object within Active Directory that determines and moves the message to the junk mail folder when over the junk threshold.

The only role the CFA plays here is stamping an SCL rating on the message which store then acts on. However, you can set the SCL rating through an Edge or Hub transport rule as well to get the same result.

Speaking of SCLJunkThreshold, this can be set in two locations: on the mailbox object via set-mailbox or on the OrganizationConfig object with Set-OrganizationConfig. I bring this up because it leads me to Myth 2b.

Myth 2b: Setting the SCLJunkThreshold on the OrganizationConfig object from an Edge server will affect junk mail actions.

Setting this will actually do nothing. Since the edge server by definition is not part of an exchange org, and since this setting is only utilized by the hidden inbox rule, changing this value will not do anything.

While we are on junk mail thresholds I really have to pause (no, really) for a misconception that is so big that it probably deserves its own separate blog post, but since I have got your attention, I will just give it a myth number instead. Without further ado, here is Myth 3.

Myth 3: Exchange/Outlook junk mail filtering is one single monolithic entity.

This is the single largest point of confusion for Exchange administrators, and requires further elaboration due to the complex interaction between Outlook and Exchange junk mail.

Outlook uses its own SmartScreen filter technology separate from Exchange junk mail screening to filter junk mail. While the two sometimes agree Outlook ignores any SCL which Exchange may set on a message and uses its own criteria to determine the "spaminess" of a message. This is the client side filter (It should be noted here that Outlook will honor an SCL of -1. The above only applies to any SCL ratings other than -1.).

Now here is where the confusion begins. The server side filter that Exchange store uses to deliver messages to junk mail has two parts. There is the setting on the mailbox and/or OrganizationConfig object to determine at what SCL value we decide to send a message to the junk e-mail folder. However, for the spam to actually be moved to the junk e-mail folder, a hidden inbox rule has to be enabled. This rule can be enabled in a number of ways. One way is to log into your inbox with Outlook in cached mode. Another is to turn on junk mail filtering in Outlook Web Access (by going to Options and then Junk E-Mail).

Now if you weren't confused already here is the super confusing part- Enabling the rule with the Outlook cached mode way also enables Outlook's client side filtering as well. Turning it on in OWA but not in Outlook (either never logging in with Outlook or going to Tools, Options, Junk E-Mail Options, and clicking the "No Automatic Filtering" radio button in the Outlook client) only turns on the server side Exchange filtering.

In fact while troubleshooting issues with junk mail here in CSS we usually make sure that the client side filter is turned off so we know which filter is not working. Starting with Outlook 2007 sp2 the InfoBar will tell you if Outlook client filtering moved the message or some other filter did the deed (Exchange, rule, 3rd party, etc). See http://support.microsoft.com/kb/968383 for an overview.

If you are not running Outlook 2007 sp2 an alternate way to check if Outlook is moving the message is to fire up MFCMapi, open the mailbox in question, browse to the Junk E-Mail folder, and then highlight the email of interest. Search the lower pane in the "Named Prop Name" tab for a property with the name of "0x859C=34204 = PidLidSpamOriginalFolder, dispidSpamOriginalFolder".

If this property is present then the Outlook client filter moved the message to junk mail.

Your best bet when deploying Outlook clients that will be used exclusively with Exchange is to not install the Outlook SmartScreen filter, as this feature complicates troubleshooting and will not be as effective as the Exchange server based Content Filter Agent.

Speaking of Junk Mail filter behavior we should take a brief stop at Myth 3b.

Myth 3b: Running Set-OWAVirtualDirectory -JunkEmailEnabled $True/False will turn on/off the server side filtering.

This is false. The above cmdlet will only influence the presence of the Junk E-Mail management option in OWA. It has no effect on the server side filter. We get asked all the time if there is a way to programmatically enable junk mail filter within OWA and while there are scripts out there on the Internet that do just that we do not directly support any of them. While these scripts may work now, they are always subject to break with each new RU/SP/version of Exchange. In Exchange 2010 we have a new set of cmdlets, Get-MailboxJunkEmailConfiguration/Set- MailboxJunkEmailConfiguration. These allow us to view the blocked senders and domains in addition to enabling the hidden junk mail rule on a particular mailbox.

Since we are on the topic of Edge servers lets go to myth 4 (ok we weren't talking about Edge servers but that last myth got me really confused and I didn't have a good segue handy)...

Myth 4: You need an Edge server to use safe list aggregation

Safe list aggregation is a feature of exchange 2007 anti-spam is where running the update-safelist cmdlet for a particular mailbox will update the msExchangeSafeSenderHash attribute on the mailbox object in Active Directory with the safe senders list from a user's mailbox. This feature is a useful way to prevent false positives from being generated by anti-spam agents.

A common misconception is that this requires an Edge server with an edge subscription. However, this feature will work on a hub transport server with the anti-spam agents installed. The CFA will read these values from AD much in the same way it would read them from ADAM if we were running on an edge server subscribed to the AD site.

Since we are on the subject of safe list aggregation, I would be remiss in not pointing out that some people add themselves to their outlook contact list. By default all outlook contacts are added to the Safe Senders list. You can see where I am going with this. Safe list aggregation will populate the mailbox's safe list hash in AD resulting in their own email address bypassing anti-spam checks. Since many spammers like to send messages spoofing the recipient address this would allow these types of spam to end up in the clients' mailbox. Because this value is hashed when placed into AD, there is no way to parse the information without logging into the user's mailbox. However In Exchange 2010 we have the Get-MailboxJunkEMailConfiguration/Set-MailboxEMailJunkMailConfiguration cmdlet come to rescue us again where we can view and configure this information from a remote power shell session.

Also it might be informative to note here that you no longer have to run the update-safelist cmdlet either manually or via a script in Exchange 2010 as it is now run automatically out of the box with no special configuration.

For more information on Exchange 2007 Safelist Aggregation can be found here: http://technet.microsoft.com/en-us/library/bb125168.aspx

Myth 5: Putting a server IP on the InternalSMTPServers list of the TransportConfig object AND IP Allow List is a good idea.

A bit of background before we tell you it's a bad idea (just a bit I promise).

Putting an IP on the IP allow list via Exchange Management Console or Add-IPAllowListEntry cmdlet allows any connection coming from that IP to bypass all anti-spam filtering except sender/recipient filter. It gives any message a SCl of -1 that originates from that IP.

The internal servers list is a list of IPs in your perimeter that you wish to bypass SenderID/IP BlockList provider/IP block list agents. It is NOT intended to bypass all AS filtering. InternalSMTPServers will cause the anti-spam analysis to be performed against the "Received" headers rather than the incoming protocol. In the case of a relay between the Internet and your first Exchange hop, this will prevent Exchange from seeing all email (spam and legitimate) as coming from the same source and blocking that source.

As you can see, these features are not the same. So, putting a server IP in both lists will not give you results. One of these is that we will still stamp an SCL rating on the message and not give the expected "-1" for a server on the allow list.

Below is a comparison of the anti-spam headers on a message of a server on the allow list and a message from a server on both:

Figure 1 above - Headers when only on IP Allow List

Figure 2 Header when on both IP Allow and InternalSMTPServers lists

As can be seen the results are completely different. For a great over view of the InternalSMTPServers attribute (as well as other aspects of the Anti-Spam agents) please take a look at this great post here: http://msexchangeteam.com/archive/2008/06/23/449070.aspx

Finally we arrive at our last myth.

Myth 6: Running Enable-AntispamUpdates forces you to download all windows updates to your Exchange server.

Enabling automatic anti-spam updates does use the windows automatic update API and does require that you opt in one time for windows update for the AS server. However, it does not require that you download all the other updates available on Windows Update like service packs and rollups etc.

In fact the anti-spam update does not use the schedule available in AU but instead uses its own download timeframe listed here: http://msexchangeteam.com/archive/2007/01/03/432050.aspx

When running the cmdlet from your edge server (or HT server with the AS agents installed) you can specify "RequestNotifyDownload" value for the -MicrosoftUpdate parameter. This maps to the "Notify me but don't automatically download or install them" radio button on the Automatic Updates client (for Windows 2008 this is worded as "Check for updates but let me choose whether to download and install them").

The above is only available from EMS and not the GUI in EMC.

Ok so that's it. I hope I was able to clear up some confusion on how Exchange anti-spam agents function.

Special thanks go to Scott Landry and Dave Forrest for their invaluable assistance with this post.

- Tom Kern

11 Comments
Not applicable
Thanks for the post.

Can you elaborate on how the actual SPAM heuristics work? I have been getting a lot of the exact same email spam being marked as SCL 0 or 1 for stuff that is clearly SPAM. I understand this may take you guys a couple of days to accurately update the spam definitions to catch this stuff, but I have been getting these for weeks now. Why does it take so long for you guys to start to catch these? Our EHS spam solution catches all of them. Does EHS use a different spam engine? EHS is Exhange Hosted Solutions. Hotmail also seems to do a better job catching SPAM than the Exchange 2007 build in SPAM engine.

Thanks

Lonwa
Not applicable
Tom,

Is there any info on how well the SPAM protection works?  We're interested in replacing our current SPAM solution with Edge, but we'd like to know how well it works.  Any statistics or success stories would help our cause.  The company I work for has 5K employees.

Teo
Not applicable
@Lonwa: Do you have an enterprise CAl or standard? With an enterprise CAL you should be getting  spam signature updates as needed
Not applicable
@Fido: Our AS agents work really well out of the box with no admin configuration or tweaking.
In the spirit of eating our own dogfood MSIT is one of the biggest consumers of our Edge AS solution and my inbox can attest to its effectiveness.
Not applicable
Could you please explain - what does Postfix do at the front side of Microsoft Corp mail?

host -t MX microsoft.com
microsoft.com mail is handled by 10 mail.global.frontbridge.com.
telnet mail.global.frontbridge.com 25
Trying 207.46.51.86...
Connected to mail.global.frontbridge.com.
H220 mail133-sin.bigfish.com ESMTP Postfix EGGS and Butter

Isn't MS Exchange + FF enough to secure Microsoft's mail?
This makes great dissonance among MS techologies followers and sceptics.

Can you answer?
Not applicable
Microsoft Forefront Online Protection (FOPE formerly known as EHS and Frontbridge before that) uses a variety of technologies built on Microsoft platforms, as well as existing investments from aquired companies.  At this time, Microsoft.com is in fact pointing to the FOPE service -- though there are other Microsoft properties that may or may not point directly to Edge servers at any given time -- as well as individuals whose email may bypass the service.  I do recommend investigating the service based offering -- it does use some of the same technologies, but also additional technologies that are not available with the Exchange-based filter alone.  In addition, there may be some significant cost & reliability benefits for hosting your MX records in the cloud.  For more information, the FF team blog is available here:
http://blogs.technet.com/forefront/
Not applicable
Hi Tom,

We do have the enterprise CAL, and get the updates daily, but that has not helped prevent the SPAM from being let in, and marked as an SCL of 0, 1 or 2, when its clearly no legitimate.

Thanks

Lonwa
Not applicable
Lonwa the EHS (now FOPE-Forefront Online Protection for Exchange) solution uses multiple AS technologies and services in addition to the filters which come installed with the Edge role. This may be why you are seeing different results.
Unfortunately false negatives are unique to each customer environment and it would be difficult for me to determine root cause based on the information I have.
A better forum would be the Technet anti spam newsgroups here-http://social.technet.microsoft.com/Forums/en-US/exchangesvrantivirusandantispam/threads

Or open a support case with Microsoft CSS.
Thanks
Not applicable
Great Post!

I have myself tried to explain these myths as they apply to Exchange 2003 in various articles on ExchangeInbox.com. Here are some links:

Enabling/Disabling the Junk Email Folder
http://www.exchangeinbox.com/article.aspx?i=27

Centralizing Junk Email Folder Administration
http://www.exchangeinbox.com/article.aspx?i=29

The above links amongst others discuss the BIG Myth #3 in the case of  Exchange 2003 IMF.

I also have to say that the Junk Folder enablement for Ex03 IMF and Ex07 Content Filter are two of the biggest problems Administrators report to me.

Get-MailboxJunkEmailConfiguration
Set- MailboxJunkEmailConfiguration will definitely render the Ex10 Content Filter much easier to manage.
Not applicable
I don't have Edge server.  If all I want is to have server-side moving spams into Junk Mail folder, do I still have to:
1. run the "install-AntispamAgents.ps1" command?
2. tell every user to open OWA in order to enable some junk mail option?
Not applicable
Hello Horus,

1. you need something to set an SCL that the junk filter can act on. Installing the anti-spam agents on the hub server is the easiest way to achieve this. Alternately you can set an SCL via a hub/edge rule as well but I recommend having the content filter agent do this.
2. You can turn on the server side rule in a number of ways-
a. turn it on via OWA
b. logging into outlook in cached mode automatically turns it on
c. log into outlook in online mode and go to "tools", "options", and "junk email".

Hope this helps.
Thanks
Version history
Last update:
‎Nov 13 2009 03:43 PM
Updated by: