In Microsoft CSS (Customer Service and Support) we deal with many anti-spam and email security questions. While the anti-spam features that come out of the box with Exchange 2007 provide a robust level of protection against unwanted garbage in your inbox, there is still a lot of confusion out there as to how all the parts work together. The purpose of this post is to dispel some misconceptions about the E2007 AS features (where applicable differences introduced in Exchange 2010 will be pointed out as well). I present you the top 6 SMTP anti-spam myths - revealed! (drum roll)
Myth 1: Creating a hub transport rule to set the SCL will affect the behavior of SCLDeleteThreshold and SCLRejectThreshold.
This myth applies particularly to the case of Hub server role with anti-spam agents installed. While it is fine to install the anti-spam agents on a hub transport server, expecting that a hub transport rule with "set the spam confidence level to value" action will influence the content filter delete/reject/quarantine is false.
- This is a misconception due to where in the transport pipeline the content filter agent (which does the actual deleting/rejecting) fires. If we run Get-TransportPipeline we will see that the content filter agent fires at OnEndOfData (EOD) while the transport rules fire during the OnRouted stage.
Since the Transport Rule Agent fires after the Content Filter Agent (CFA) anything the rule action does will have no effect on CFA behavior.
Conversely while this would work on an Edge server anti-spam solution due to the location in the pipeline where the Edge rule fires this actually leads me to Myth 1b.
Myth 1b: Setting an edge rule to inspect SCL value that the content filter sets will work out of the box.
Unlike Hub Transport Rules Agent, Edge Transport Rules Agent fires BEFORE the content filter.
So if we leave the default settings with Edge rules firing first we have nothing to inspect or act on since the CFA has not gotten the message yet.
Your best bet for using rules that operate in conjunction with content filtering is to run content filtering on an Edge role and the Transport Rules on the Hub role.
Myth 2: You need the CFA to move items to junk mail folder.
Unlike SCLDeleteThreshold and SCLRjectThreshold, SCLJunkThreshold is a store setting. It is a combination of a hidden rule in the client's inbox when junk mail is enabled and the value of the SCLJunkThreshold attribute on the OrganizationConfig and/or mailbox object within Active Directory that determines and moves the message to the junk mail folder when over the junk threshold.
The only role the CFA plays here is stamping an SCL rating on the message which store then acts on. However, you can set the SCL rating through an Edge or Hub transport rule as well to get the same result.
Speaking of SCLJunkThreshold, this can be set in two locations: on the mailbox object via set-mailbox or on the OrganizationConfig object with Set-OrganizationConfig. I bring this up because it leads me to Myth 2b.
Myth 2b: Setting the SCLJunkThreshold on the OrganizationConfig object from an Edge server will affect junk mail actions.
Setting this will actually do nothing. Since the edge server by definition is not part of an exchange org, and since this setting is only utilized by the hidden inbox rule, changing this value will not do anything.
While we are on junk mail thresholds I really have to pause (no, really) for a misconception that is so big that it probably deserves its own separate blog post, but since I have got your attention, I will just give it a myth number instead. Without further ado, here is Myth 3.
Myth 3: Exchange/Outlook junk mail filtering is one single monolithic entity.
This is the single largest point of confusion for Exchange administrators, and requires further elaboration due to the complex interaction between Outlook and Exchange junk mail.
Outlook uses its own SmartScreen filter technology separate from Exchange junk mail screening to filter junk mail. While the two sometimes agree Outlook ignores any SCL which Exchange may set on a message and uses its own criteria to determine the "spaminess" of a message. This is the client side filter (It should be noted here that Outlook will honor an SCL of -1. The above only applies to any SCL ratings other than -1.).
Now here is where the confusion begins. The server side filter that Exchange store uses to deliver messages to junk mail has two parts. There is the setting on the mailbox and/or OrganizationConfig object to determine at what SCL value we decide to send a message to the junk e-mail folder. However, for the spam to actually be moved to the junk e-mail folder, a hidden inbox rule has to be enabled. This rule can be enabled in a number of ways. One way is to log into your inbox with Outlook in cached mode. Another is to turn on junk mail filtering in Outlook Web Access (by going to Options and then Junk E-Mail).
Now if you weren't confused already here is the super confusing part- Enabling the rule with the Outlook cached mode way also enables Outlook's client side filtering as well. Turning it on in OWA but not in Outlook (either never logging in with Outlook or going to Tools, Options, Junk E-Mail Options, and clicking the "No Automatic Filtering" radio button in the Outlook client) only turns on the server side Exchange filtering.
In fact while troubleshooting issues with junk mail here in CSS we usually make sure that the client side filter is turned off so we know which filter is not working. Starting with Outlook 2007 sp2 the InfoBar will tell you if Outlook client filtering moved the message or some other filter did the deed (Exchange, rule, 3rd party, etc). See http://support.microsoft.com/kb/968383 for an overview.
If you are not running Outlook 2007 sp2 an alternate way to check if Outlook is moving the message is to fire up MFCMapi, open the mailbox in question, browse to the Junk E-Mail folder, and then highlight the email of interest. Search the lower pane in the "Named Prop Name" tab for a property with the name of "0x859C=34204 = PidLidSpamOriginalFolder, dispidSpamOriginalFolder".
If this property is present then the Outlook client filter moved the message to junk mail.
Your best bet when deploying Outlook clients that will be used exclusively with Exchange is to not install the Outlook SmartScreen filter, as this feature complicates troubleshooting and will not be as effective as the Exchange server based Content Filter Agent.
Speaking of Junk Mail filter behavior we should take a brief stop at Myth 3b.
Myth 3b: Running Set-OWAVirtualDirectory -JunkEmailEnabled $True/False will turn on/off the server side filtering.
This is false. The above cmdlet will only influence the presence of the Junk E-Mail management option in OWA. It has no effect on the server side filter. We get asked all the time if there is a way to programmatically enable junk mail filter within OWA and while there are scripts out there on the Internet that do just that we do not directly support any of them. While these scripts may work now, they are always subject to break with each new RU/SP/version of Exchange. In Exchange 2010 we have a new set of cmdlets, Get-MailboxJunkEmailConfiguration/Set- MailboxJunkEmailConfiguration. These allow us to view the blocked senders and domains in addition to enabling the hidden junk mail rule on a particular mailbox.
Since we are on the topic of Edge servers lets go to myth 4 (ok we weren't talking about Edge servers but that last myth got me really confused and I didn't have a good segue handy)...
Myth 4: You need an Edge server to use safe list aggregation
Safe list aggregation is a feature of exchange 2007 anti-spam is where running the update-safelist cmdlet for a particular mailbox will update the msExchangeSafeSenderHash attribute on the mailbox object in Active Directory with the safe senders list from a user's mailbox. This feature is a useful way to prevent false positives from being generated by anti-spam agents.
A common misconception is that this requires an Edge server with an edge subscription. However, this feature will work on a hub transport server with the anti-spam agents installed. The CFA will read these values from AD much in the same way it would read them from ADAM if we were running on an edge server subscribed to the AD site.
Since we are on the subject of safe list aggregation, I would be remiss in not pointing out that some people add themselves to their outlook contact list. By default all outlook contacts are added to the Safe Senders list. You can see where I am going with this. Safe list aggregation will populate the mailbox's safe list hash in AD resulting in their own email address bypassing anti-spam checks. Since many spammers like to send messages spoofing the recipient address this would allow these types of spam to end up in the clients' mailbox. Because this value is hashed when placed into AD, there is no way to parse the information without logging into the user's mailbox. However In Exchange 2010 we have the Get-MailboxJunkEMailConfiguration/Set-MailboxEMailJunkMailConfiguration cmdlet come to rescue us again where we can view and configure this information from a remote power shell session.
Also it might be informative to note here that you no longer have to run the update-safelist cmdlet either manually or via a script in Exchange 2010 as it is now run automatically out of the box with no special configuration.
For more information on Exchange 2007 Safelist Aggregation can be found here: http://technet.microsoft.com/en-us/library/bb125168.aspx
Myth 5: Putting a server IP on the InternalSMTPServers list of the TransportConfig object AND IP Allow List is a good idea.
A bit of background before we tell you it's a bad idea (just a bit I promise).
Putting an IP on the IP allow list via Exchange Management Console or Add-IPAllowListEntry cmdlet allows any connection coming from that IP to bypass all anti-spam filtering except sender/recipient filter. It gives any message a SCl of -1 that originates from that IP.
The internal servers list is a list of IPs in your perimeter that you wish to bypass SenderID/IP BlockList provider/IP block list agents. It is NOT intended to bypass all AS filtering. InternalSMTPServers will cause the anti-spam analysis to be performed against the "Received" headers rather than the incoming protocol. In the case of a relay between the Internet and your first Exchange hop, this will prevent Exchange from seeing all email (spam and legitimate) as coming from the same source and blocking that source.
As you can see, these features are not the same. So, putting a server IP in both lists will not give you results. One of these is that we will still stamp an SCL rating on the message and not give the expected "-1" for a server on the allow list.
Below is a comparison of the anti-spam headers on a message of a server on the allow list and a message from a server on both:
Figure 1 above - Headers when only on IP Allow List
Figure 2 Header when on both IP Allow and InternalSMTPServers lists
As can be seen the results are completely different. For a great over view of the InternalSMTPServers attribute (as well as other aspects of the Anti-Spam agents) please take a look at this great post here: http://msexchangeteam.com/archive/2008/06/23/449070.aspx
Finally we arrive at our last myth.
Myth 6: Running Enable-AntispamUpdates forces you to download all windows updates to your Exchange server.
Enabling automatic anti-spam updates does use the windows automatic update API and does require that you opt in one time for windows update for the AS server. However, it does not require that you download all the other updates available on Windows Update like service packs and rollups etc.
In fact the anti-spam update does not use the schedule available in AU but instead uses its own download timeframe listed here: http://msexchangeteam.com/archive/2007/01/03/432050.aspx
When running the cmdlet from your edge server (or HT server with the AS agents installed) you can specify "RequestNotifyDownload" value for the -MicrosoftUpdate parameter. This maps to the "Notify me but don't automatically download or install them" radio button on the Automatic Updates client (for Windows 2008 this is worded as "Check for updates but let me choose whether to download and install them").
The above is only available from EMS and not the GUI in EMC.
Ok so that's it. I hope I was able to clear up some confusion on how Exchange anti-spam agents function.
Special thanks go to Scott Landry and Dave Forrest for their invaluable assistance with this post.
- Tom Kern
You Had Me at EHLO.