Load Balancing in Exchange 2016

Like Exchange 2013, Exchange 2016 does not require session affinity at the load balancing layer.

To understand this statement better, and see how this impacts your designs, we need to look at how MBX2016 functions. From a protocol perspective, the following will happen:

1. A client resolves the namespace to a load balanced virtual IP address.
2. The load balancer assigns the session to a MBX server in the load balanced pool.
3. The Client Access services located on the MBX server authenticates the request and performs a service discovery by accessing Active Directory to retrieve the following information:
   1. Mailbox version (for this discussion, we will assume an Exchange 2016 mailbox)
   2. Mailbox location information (e.g., database information, ExternalURL values, etc.)
4. The Client Access services located on the MBX server makes a decision on whether to proxy the request or redirect the request to another MBX infrastructure (within the same forest).
5. The Client Access services located on the MBX server queries an Active Manager instance that is responsible for the database to determine which Mailbox server is hosting the active copy.
6. The Client Access services located on the MBX server proxies the request to the Mailbox server hosting the active copy.

Step 5 is the fundamental change that enables the removal of session affinity at the load balancer. For a given protocol session, the Client Access services located on the Mailbox server now maintains a 1:1 relationship with the Mailbox server hosting the user’s data. In the event that the active database copy is moved to a different Mailbox server, MBX closes the sessions to the previous server and establishes sessions to the new server. This means that all sessions, regardless of their origination point (i.e., MBX servers in the load balanced array), end up at the same place, the Mailbox server hosting the active database copy.This is vastly different in releases prior to Exchange 2013 – for example, in Exchange 2010, if all requests from a specific client did not go to the same endpoint, the user experience was negatively affected.

The protocol used in step 6 depends on the protocol used to connect to MBX. If the client leverages the HTTP protocol, then the protocol used between Mailbox servers is HTTP (secured via SSL using a self-signed certificate). If the protocol leveraged by the client is IMAP or POP, then the protocol used between the Mailbox servers is IMAP or POP.

Telephony requests are unique, however. Instead of proxying the request at step 6, MBX will redirect the request to the Mailbox server hosting the active copy of the user’s database, as the telephony devices support redirection and need to establish their SIP and RTP sessions directly with the Unified Messaging components on the Mailbox server.

Figure 1: Exchange 2016 Client Access Protocol Architecture

However, there is a concern with this architectural change. Since session affinity is not used by the load balancer, this means that the load balancer has no knowledge of the target URL or request content.All the load balancer uses is layer 4 information, the IP address and the protocol/port (TCP 443):

Figure 2: Layer 4 Load Balancing

The load balancer can use a variety of means to select the target server from the load balanced pool, such as, round-robin (each inbound connection goes to the next target server in the circular list) or least-connection (load balancer sends each new connection to the server that has the fewest established connections at that time).

Health Probe Checking

Unfortunately, this lack of knowledge around target URL (or the content of the request), introduces complexities around health probes.

Exchange 2016 includes a built-in monitoring solution, known as Managed Availability. Managed Availability includes an offline responder. When the offline responder is invoked, the affected protocol (or server) is removed from service. To ensure that load balancers do not route traffic to a Mailbox server that Managed Availability has marked as offline, load balancer health probes must be configured to check <virtualdirectory>/healthcheck.htm (e.g., https://mail.contoso.com/owa/healthcheck.htm). Note that healthcheck.htmdoes not actually exist within the virtual directories; it is generated in-memory based on the component state of the protocol in question.

If the load balancer health probe receives a 200 status response, then the protocol is up; if the load balancer receives a different status code, then Managed Availability has marked that protocol instance down on the Mailbox server. As a result, the load balancer should also consider that end point down and remove the Mailbox server from the applicable load balancing pool.

Administrators can also manually take a protocol offline for maintenance, thereby removing it from the applicable load balancing pool. For example, to take the OWA proxy protocol on a Mailbox server out of rotation, you would execute the following command:

Set-ServerComponentState <Mailbox Server> -Component OwaProxy –Requestor Maintenance –State Inactive

For more information on server component states, see the article Server Component States in Exchange 2013.

What if the load balancer health probe did not monitor healthcheck.htm?

If the load balancer did not utilize the healthcheck.htm in its health probe, then the load balancer would have no knowledge of Managed Availability’s removal of (or adding back) a server from the applicable load balancing pool. The end result is that the load balancer would have one view of the world, while Managed Availability would have another view of the world. In this situation, the load balancer could direct requests to a Mailbox server that Managed Availability has marked down, which would result in a negative (or broken) user experience. This is why the recommendation exists to utilize healthcheck.htm in the load balancing health probes.

Namespace and Affinity Scenarios

Now that we understand how health checks are performed, let’s look at four scenarios:

1. Single Namespace / Layer 4 (No Session Affinity)
2. Single Namespace / Layer 7 (No Session Affinity)
3. Single Namespace / Session Affinity
4. Multiple Namespaces / No Session Affinity

Single Namespace / Layer 4 (No Session Affinity)

In this scenario, a single namespace is deployed for all the HTTP protocol clients (mail.contoso.com). The load balancer is operating at layer 4 and is not maintaining session affinity. The load balancer is also configured to check the health of the target Mailbox servers in the load balancing pool; however, because this is a layer 4 solution, the load balancer is configured to check the health of only a single virtual directory (as it cannot distinguish OWA requests from RPC requests). Administrators will have to choose which virtual directory they want to target for the health probe; you will want to choose a virtual directory that is heavily used.For example, if the majority of your users utilize OWA, then targeting the OWA virtual directory in the health probe is appropriate.

Figure 3: Single Namespace with No Session Affinity

As long as the OWA health probe response is healthy, the load balancer will keep the target MBX in the load balancing pool. However, if the OWA health probe fails for any reason, then the load balancer will remove the target MBX from the load balancing pool for all requests associated with that particular namespace. In other words, in this example, health from the perspective of the load balancer, is per-server, not per-protocol, for the given namespace.This means that if the health probe fails, all client requests for that namespace will have to be directed to another server, regardless of protocol.

Figure 4: Single Namespace with No Session Affinity - Health Probe Failure

Single Namespace / Layer 7 (No Session Affinity)

In this scenario, a single namespace is deployed for all the HTTP protocol clients (mail.contoso.com). The load balancer is configured to utilize layer 7, meaning SSL termination occurs and the load balancer knows the target URL. The load balancer is also configured to check the health of the target Mailbox servers in the load balancing pool; in this MBXe, a health probe is configured on each virtual directory.

As long as the OWA health probe response is healthy, the load balancer will keep the target MBX in the OWA load balancing pool. However, if the OWA health probe fails for any reason, then the load balancer will remove the target MBX from the load balancing pool for OWA requests. In other words, in this example, health is per-protocol; this means that if the health probe fails, only the affected client protocol will have to be directed to another server.

Figure 5: Single Namespace with Layer 7 (No Session Affinity) - Health Probe Failure

A single namespace utilizing layer 7 without session affinity is the recommended namespace and load balancer configuration for Exchange 2016.

Single Namespace / Layer 7 with Session Affinity

this scenario, a single namespace is deployed for all the HTTP protocol clients (mail.contoso.com). The load balancer is configured to maintain session affinity (layer 7), meaning SSL termination occurs and the load balancer knows the target URL. The load balancer is also configured to check the health of the target Mailbox servers in the load balancing pool; in this MBXe, the health probe is configured on each virtual directory.

As long as the OWA health probe response is healthy, the load balancer will keep the target MBX in the OWA load balancing pool. However, if the OWA health probe fails for any reason, then the load balancer will remove the target MBX from the load balancing pool for OWA requests. In other words, in this example, health is per-protocol; this means that if the health probe fails, only the affected client protocol will have to be directed to another server.

Figure 6: Single Namespace with Layer 7 with Session Affinity - Health Probe Failure

Note: By having session affinity enabled, the load balancer’s capacity and utilization are decreased because processing is used to maintain more involved affinity options, such as cookie-based load balancing or Secure Sockets Layer (SSL) session-ID. Check with your vendor on the impacts session affinity will have in your load balancing scalability.

Multiple Namespaces / No Session Affinity

This scenario combines the best of both worlds – provides per-protocol health checking, while not requiring complex load balancing logic.

In this scenario, a unique namespace is deployed for each HTTP protocol client; for example:

Figure 7: Multiple Namespaces with No Session Affinity

Note: As seen in the picture depicted above, ECP is provided its own namespace. ECP and OWA are intimately tied together, and thus, ECP does not necessarily require its own namespace. However, ECP does have its own application pool, is the endpoint for the Exchange Administration Center, and used by Outlook clients for certain configuration items. Therefore, you may want to provide a unique namespace for ECP.

The load balancer is configured to not maintain session affinity (layer 4). The load balancer is also configured to check the health of the target Mailbox servers in the load balancing pool; in this case, the health probes are effectively configured to target the health of each virtual directory, as each virtual directory is defined with a unique namespace, and while the load balancer still has no idea what the URL is being accessed, the result is as if it does know.

As long as the OWA health probe response is healthy, the load balancer will keep the target MBX in the OWA load balancing pool. However, if the OWA health probe fails for any reason, then the load balancer will remove the target MBX from the load balancing pool for OWA requests. In other words, in this example, health is per-protocol; this means that if the health probe fails, only the affected client protocol will have to be directed to another server.

Figure 8: Multiple Namespaces with No Session Affinity - Health Probe Failure

The downside to this approach is that it introduces additional namespaces, additional VIPs (one per namespace), and increases the number of names added as subject alternative names on the certificate, which can be costly depending on your certificate provider. But, this does not introduce extra complexity to the end user – the only URL the user needs to know is the OWA URL. ActiveSync, Outlook, and Exchange Web Services clients will utilize Autodiscover to determine the correct URL.

Exchange Scenario Summary

The following table identifies the benefits and concerns with each approach:

Office Online Server Load Balancing

Exchange Server 2016 leverages Office Online Server to provide the rich document preview and editing capabilities for OWA. While this was a necessary change to ensure a homogenous experience across the Office Server suite, this does introduce additional complexity for environments that don’t have Office Online Server.

As discussed in the architecture and namespace planning articles, the Office Online Server infrastructure requires unique namespaces. The load balancer is configured to maintain layer 7 with session affinity (using cookie-based persistence) for each Office Online Server namespace, meaning SSL termination occurs and the load balancer knows the target URL.This ensures the client is always directed to the same Office Online Server while the user is utilizing the document collaboration capabilities within OWA.

Conclusion

Exchange 2016 introduces significant flexibility in your namespace and load balancing architecture. With load balancing, the decision ultimately comes down to balancing functionality vs. simplicity. The simplest solution lacks session affinity management and per-protocol health checking, but provides the capability to deploy a single namespace. At the other end of the spectrum, you can utilize session affinity management, per-protocol health checking with a single namespace, but at the cost of increased complexity. Or you could balance the functionality and simplicity spectrums, and deploy a load balancing solution that doesn’t leverage session affinity, but provides per-protocol health checking at the expense of requiring a unique namespace per protocol.

Ross Smith IV
Principal Program Manager
Office 365 Customer Experience