Introduction
Usually there is one question that is always asked when dealing with mail-enabled public folders: How does the message get delivered to a replica?
To answer that question, let's take a walk down memory lane.
Exchange 5.5
In Exchange 5.5 we used to store replica information in the directory (aka "home server" field on public folders). This allowed us to determine a replica to route to immediately upon categorization of the message.
This did have its faults though. You had to manage each public folder and its home server designation (i.e. a single point of failure). In addition, because we did store that information in the directory, DS/IS Consistency Check needed to be executed routinely to clean up the directory or to re-home the replica if the site containing it was no longer present.
Exchange 2000 / 2003
In Exchange 2000 / 2003 we moved away from storing replica information in the directory and instead just stored it within the public folder hierarchy. Now the only thing stored in the Active Directory is a proxy object for the public folder that basically tells us the email address and that it's a public folder proxy object. So as a result of these changes we had to change the way transport dealt with SMTP messages destined for public folders.
The basic process is (for in-depth information, take a look at http://msexchangeteam.com/archive/2004/09/10/228114.aspx):
1. SMTP mail addressed to Public Folder comes into transport.
2. Transport categorizes the message and obtains a list of all PF stores for that hierarchy. We then pick a store and route the message to it:
a. If a PF store is on the same server, we use that.
b. If there are no local PF stores, then we'll use a PF store in the same routing group.
c. If there are no local PF stores in the routing group, then we use the first entry in the list from msExchOwningPFTreeBL entry.
This list always has the most recently added public folder stores to the organization at the top (and as a public folder store is created when a new server is installed, normally this means the most recently installed servers will be at the top of the list). What could happen is we pick a server that has just been added to the org and may not have the full public folder hierarchy yet (and hence not know what to do with the message). To prevent this in Exchange 2003, we don't select any stores younger than two days, unless there is no alternative. See http://support.microsoft.com/kb/328870/en-us for more information on use of PF store Age in routing decisions.
3. If there is a replica on this store, then the message is simply delivered. If not, the store returns the location of the closest replica and hands the message back to transport – which then sends it on to the store which actually holds the replica.
Exchange 2007 RTM
In Exchange 2007 we made some fundamental changes to the way transport works (see http://technet.microsoft.com/en-us/library/aa998825.aspx for more information):
- No more routing groups. Instead we will use the Active Directory topology for determining the least cost route.
- Direct Connections are preferred. When possible the source Hub Transport server will establish a direct TCP connection with the destination Hub Transport server that resides in the Active Directory site where the mailbox or replica resides. No more passing it along a chain of Exchange servers if we can avoid it.
So how does that change Public Folder mail routing? Well let's take a look at the stages.
Routing Table Calculation
When the Microsoft Exchange Transport service starts on a Hub Transport Server (or when a change notification occurs, or after 6 hours), it calculates a set of routing tables that is based on the snapshot of information that is retrieved from Active Directory. The routing component refers to the routing tables to determine how to route messages to recipients. When configuration changes are made, the routing tables are rebuilt. The new routing tables are used to route new incoming messages. Messages in remote delivery queues are also rerouted if the routing component determines that they are affected by the configuration changes. For more information on the routing table calculation, please see http://technet.microsoft.com/en-us/library/aa998825.aspx.
Public folder hierarchies are one item that is retrieved during the calculation of the routing tables. Specifically, to ensure that proper delivery can occur to the public folder replica, the "best" public folder store is chosen by reviewing the msExchOwningPFTreeBL directory entry (Note: The msExchOwningPFTreeBL list always has the most recently added public folder stores at the top) using the following criteria (at RTM):
1. First and foremost, if the list contains any legacy Exchange PF stores, they are removed if Exchange 2007 PF stores exist. Age of the store is not considered when building this list.
2. If the choice is between two (or more) Exchange 2007 PF stores, then the following criteria will be used:
a. Age Ranking (PF store less than 2 days old by default will not be considered, unless the store's age is less than the threshold or is unknown).
b. If there is a local PF store on the same physical machine, we will use that.
c. If there is not a local PF store, then we will choose a store within the local Active Directory Site. If there are multiple PF stores, the first server returned is chosen.
d. If there are no PF stores within the local Active Directory site, then we will choose a PF store in a remote Active Directory Site, sorted by lowest cost (if the choice is between two or more remote AD sites). If the result is a tie, the first server returned is chosen.
3. In the event that there are no Exchange 2007 PF stores, and if the choice is between two (or more) legacy Exchange PF stores, then the following criteria will be used:
a. Age Ranking (PF store less than 2 days old by default will not be considered, unless the store's age is less than the threshold or is unknown).
b. If there are multiple PF stores, then the first store is chosen.
Transport Steps
1. The Hub Transport server receives the message and categorization is performed. When the categorizer receives a message, it looks up the email address in the directory. If the email address resolves to a public folder directory entry it knows that it has to do something special.
2. The categorizer looks up the homeMDB attribute of the public folder's directory entry. This tells it which PF hierarchy the folder belongs to (usually there's only one PF hierarchy and that's the default "Public Folders" hierarchy associated with the default public stores that get created automatically).
3. Based on the routing table calculations performed by the Transport service, the "best" public folder hierarchy store ("best" TLH) is used to determine which public folder store contains a replica that can be used for delivery.
a. If the "best" TLH is located within the same Active Directory site as the Hub Transport server, then we proceed to Step 4.
b. If the "best" TLH is not located within the same AD site, we will route the message to a Hub Transport server in the destination AD site by using the routing table to determine the least cost route. We then proceed at Step 1 with the destination Hub Transport server, the message is categorized (Step 2) and the best TLH store is used to determine the public folder store that contains a replica (Step 3a).
c. If the "best" TLH is located in an routing group on a legacy Exchange server, then we will route the message to the legacy Exchange server, and the legacy Exchange server will handle determination of replica based on the steps outlined in http://msexchangeteam.com/archive/2004/09/10/228114.aspx.
4. Instead of transferring the message via SMTP to the hierarchy store for replica lookup, the Hub Transport server will establish a connection to the store service on the mailbox server role that contains the "best" TLH store. The store is then queried to determine if content for the folder (referenced by legacyExchangeDN) is available (IsContentAvailable), which results in a response containing the list of servers hosting the folder if the content is not available locally (store override). The list of servers hosting the folder replica is listed in the same order provided in client folder referrals and the top entry is chosen by transport. This referral tells us to which replica we should route the message, aka, the "best" replica.
5. The Hub Transport server then uses the routing table to determine the least cost route for the server that contains the "best" replica and routes the message accordingly (see http://technet.microsoft.com/en-us/library/aa998825.aspx for more information on how least cost routing occurs).
6. The message gets delivered to the public folder store.
Exchange 2007 SP1
In Exchange 2007 SP1 we are altering the behavior slightly. In RTM, we prefer an Exchange 2007 PF store over a legacy Exchange PF store, regardless of age.
That's right in RTM, we do not consider the age of the public folder store in that case. What does that mean? Well that means we may end up choosing a PF store that doesn't contain the entire hierarchy (as a result of just being created and not having received or processed the hierarchy replication messages). If we choose a PF store that doesn't have the hierarchy we cannot perform a lookup of the replica, and thus we'll NDR the message.
That's not the best situation. So in SP1 the decision tree will be:
- Age Ranking - PF stores less than 2 days old by default will not be considered, unless the all of the stores are less than the threshold or the age is unknown.
- Proximity - local server is preferred over local AD site over remote AD site over remote RG.
- Cost - if the choice is between two remote AD site replicas.
- If still a tie, the first server in the replica list returned by AD is chosen.
Hierarchy Lookup Failure Scenarios
If the Hub Transport Server cannot contact the "best" TLH store, then the message will queue on the Hub Transport Server. What are the scenarios here? Well once we choose a "best" TLH store we will stick with it unless one of the following happens:
- The TLH store can be contacted and we deliver locally in case of a local replica on the TLH store, or we obtain a referral for replica delivery.
- If the Transport service gets restarted; though this could still result in the same TLH store being chosen.
- A different TLH store will only be chosen if an underlying topology change has occurred (routing table recalculated as a result of that change), and a new "best" TLH store is found during route recalculation).
- Message times out and NDRs.
General Queuing Scenarios
So you might be thinking, besides the hierarchy lookup failure scenarios, where might a SMTP based message (and at this point it doesn't even really matter that it is destined to a public folder) queue?
So for the purposes of this discussion, let's discuss each based off of the following diagram (please click on the thumbnail to see the full size):
Routing the Message to a Hub Transport Server
- If the SMTP message destined to a public folder comes from an external source (e.g. the Internet SMTP server), and that server cannot contact the Edge Transport server, then the message will queue on the source SMTP server.
- This can be mitigated by having multiple ingress SMTP points into the organization that are physically separate, and by having an appropriate DNS infrastructure in place.
- In addition, each physical site should have multiple Edge Transport servers, so that the server itself is not a single point of failure (again with the appropriate DNS infrastructure in place).
- If the SMTP message destined to a public folder is successfully delivered to the Edge Transport server, but the Edge Transport server cannot contact a Hub Transport server, the message will queue at the source (the Edge transport Server).
- This can be mitigated by having multiple Hub Transport servers within the AD site since the Edge Transport server (configured via EdgeSync) will balance traffic across the Hub Transport servers for that AD Site. In the event that a Hub Transport server is unresponsive, the Edge Transport server will attempt delivery to another Hub Transport server within that AD Site. If all Hub Transport servers are unresponsive, then the message will queue on the Edge Transport server.
- If the SMTP message destined to a public folder comes from an internal client (i.e. one that submits messages to the store service like Outlook, OWA, or EAS), then if the mailbox submission service cannot contact the Hub Transport server, the message will queue on the Mailbox server (in the information store).
- This can be mitigated by having multiple Hub Transport servers within the AD site. The mailbox store will prefer the Hub Transport role installed locally if there is one on the box, and if that is unavailable, it will load balance traffic via round-robin algorithm across the Hub Transport servers within the same AD site so that it can deliver the message. If all Hub Transport servers are down within the AD site, the message will be queued on the Mailbox server until a Hub Transport server becomes available. Mailbox servers cannot communicate with Hub Transport servers that are not members of its AD site.
Routing the Message to its Destination
Once the Hub Transport server receives the message it will perform the steps outlined in the "Exchange 2007 RTM" section.
There are two potential scenarios that could result in replica delivery queuing and they depend on the location of the replica:
1. Once the server hosting the best folder replica is chosen, if the Hub Transport server cannot contact the mailbox server that contains the PF replica (intra-AD site), then it will queue on the Hub Transport server. In this scenario, we will queue until the store service is responsive.
2. Once the best replica is chosen, if the Hub Transport server cannot contact the destination Hub Transport Server (inter-AD site), then it will queue on a Hub Transport server. There are a few things with this scenario:
a. For one, each AD site that has a Mailbox server, should consider having multiple Hub Transport servers so that the Hub Transport server does not become a single point of failure.
b. In the scenario, where all Hub Transport servers are unresponsive (say because of network outage), the source Hub Transport server will not be able to establish a direct connection with the destination Hub Transport servers. So in this case, we will perform a back-off and get it as close as possible to the destination AD site (re-route, i.e. choosing another least cost route, would only occur after a site topology change).
Conclusion
We hope the above information helps explain how the routing process works for mail delivery to mail-enabled public folder replicas.