The case of constant OAB generation and full OAB downloads
Published Nov 25 2019 09:35 AM 20K Views

Introduction

The thing that prompted publishing of this blog post is a recent on-premises customer support case. We figured it is a good case study to help illustrate the OAB (Offline Address Book) generation process and how to troubleshoot it if a need arises.

The situation was that Outlook users were always getting full download of the OAB, whether downloaded manually or automatically, which was causing a lot of network traffic on the Exchange Server and was causing unwelcome performance overhead.

Previous blog posts on OAB in Exchange Server 2013 and Managing OAB in Exchange Server 2013 explain what an OAB is and how to manage it. In this article, we will dig a little deeper in to the OAB generation process, the files that are involved, what happens in case of a failover, and how that was linked to this specific scenario (Or was it? Read on to find out!)

OAB files on the server and the client

Before we get into troubleshooting details, we want to go over how OAB files that are generated on the server map to OAB files that are downloaded by the Outlook client.

In Exchange Server 2013 onwards, the mailbox server hosting the OAB Generation mailbox is the one responsible to generate the OAB. OAB files are generated and stored under %ExchangeInstallPath%\ClientAccess\OAB\ in a folder named with a <GUID>. This GUID1 is the same one that’s on the OAB object.

Example:

OAB1.jpg

 

Inside this GUID1 folder, you will find files ending with extensions LZX, FLT and one file called OAB.XML. The LZX files contain the actual OAB, delta changes, language and OS specific changes. These are compressed files stored in the following location of the server that hosts the OAB generation mailbox server: “%ExchangeInstallPath%\ClientAccess\OAB\<guid>\”. Each LZX file has a randomly generated GUID2 associated with it. The OAB.XML file is called a Manifest.

Example:

OAB2.jpg

 

OAB files downloaded by the Outlook client are stored under %LOCALAPPDATA%\Microsoft\Outlook\<GUID>. The GUID folder on the client has the same name as that of GUID2 observed on the names of the LZX. The GUID2 on the OAB generation file plays a crucial part in determining if full or incremental download is needed. Any time this GUID2 changes, the result will be a full OAB download on clients.

The GUID2 is maintained even during database failover, in an interesting way. Let’s see how!

OAB generation workflow

Let’s now see how the OAB generation workflow happens.

These are the OAB properties:

 

Params:
Name                           : Region1oab
Guid                             : 72a6e712-6e36-4fe2-b338-9096e29ba1cf
GeneratingMailbox   : contoso.com/Users/contosoOAB

 

As an example, let’s look at the output from the LDP tool, after the Region1oab is created:

 

Expanding base 'CN=Region1oab,CN=Offline Address Lists,CN=Address Lists Container,CN=Contoso,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=contoso,DC=com'...
Getting 1 entries:
Dn: CN=Region1oab,CN=Offline Address Lists,CN=Address Lists Container,CN=Contoso,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=contoso,DC=com
cn: Region1oab;
 
msExchOABFlags: 0;
msExchOABFolder: ;
msExchOABGeneratingMailboxLink: CN=contosoOAB,CN=Users,DC=contoso,DC=com;

 

So, what really happens under the hood? Let us have a look at the OABGenerator log (stored on the server under %ExchangeInstallPath%\Logging\OABGeneratorLog), which shows the generation flow:

 

OABGenerator invoked from Update-OAB on Server:E15A for on-demand generation of OAB 72a6e712-6e36-4fe2-b338-9096e29ba1cf for Org  DatabaseGuid=7bc2e8fa-9b1b-4836-9133-5b966666b466, MailboxGuid=225bb99d-0fcd-4ec1-8f08-a79680d65607"
OABGeneratorAssistant.GetOABsFromAD() returned 1 oab(s)
Logged Event Id 279145. Params: \Region1oab||CN=Region1oab,CN=Offline Address Lists,CN=Address Lists Container,CN=Contoso,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=contoso,DC=com||72a6e712-6e36-4fe2-b338-9096e29ba1cf"
OABGeneratorAssistant.BeginProcessingOAB: finish
OABGenerator.PrepareFilesForOABGeneration: start
OABGenerator.CreateOABDirectory: created new directory C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\OAB\72a6e712-6e36-4fe2-b338-9096e29ba1cf

 

This then results in the following directory on the server:

OAB3.jpg

 

At this step, if one logs into the OAB Generation mailbox (using MFCMAPI), it will look like the following  (there is no OAB folder, because the generation is not complete):

OAB4.jpg

 

Continuing with reading of the log:

 

OABGenerator.TryGetPreviousVersionOfFile: there is no usable previous manifest
OABGenerator.BeginGeneratingAddressListFiles:Creating new guid: 097c87cb-b8aa-40d4-a3ce-9548ed82a84d-
OABGenerator.FinishGeneratingAddressListFiles: finish

 

Now you can see the OAB folder got generated inside the mailbox, and inside the folder we have a mailitem of type IPM.FileSet with the subject as the GUID of the OAB:

OAB5.jpg

 

The file inside:

OAB6.jpg

 

Following along in the log:

 

OABGenerator.Publish: start
OABGenerator.PublishFileToDistribPoint: published file: C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\OAB\72a6e712-6e36-4fe2-b338-9096e29ba1cf\097c87cb-b8aa-40d4-a3ce-9548ed82a84d-

 

Now the manifest file is getting created:

 

OABGenerator.WriteManifest: writing to manifest file C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\OAB\72a6e712-6e36-4fe2-b338-9096e29ba1cf\oab.xml, version ((097c87cb-b8aa-40d4-a3ce-9548ed82a84d,1)):\n\rAddressList[0]=Id=097c87cb-b8aa-40d4-a3ce-9548ed82a84d, DN:/, Name:\Default Global Address List, file count:101\r\n"

 

Once OAB files are generated on the disk, they are uploaded to the OAB gen mailbox:

 

OABGenerator.UploadFilesToMailbox: selected OAB file C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\OAB\72a6e712-6e36-4fe2-b338-9096e29ba1cf\oab.xml to upload to the mailbox

 

Once that’s successful, a confirmation is written to the log:

 

Uploaded file 'C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\OAB\72a6e712-6e36-4fe2-b338-9096e29ba1cf\oab.xml' to mailbox"

 

Finishing the process:

 

ABGeneratorAssistant.FinishProcessingOAB: start
Logged Event Id 279146
Function  OABGeneratorAssistant.FinishProcessingOAB: finish

 

Here is the output from LDP, after OAB is generated:

 

msExchConfigurationXML: <OfflineAddressBookConfigXML><ManifestVersion><AddressLists AddressListId="097c87cb-b8aa-40d4-a3ce-9548ed82a84d" Sequence="1" /></ManifestVersion><LastFailedTime p2:nil="true" xmlns:p2="http://www.w3.org/2001/XMLSchema-instance" /><LastGeneratingData><MailboxGuid>225bb99d-0fcd-4ec1-8f08-a79680d65607</MailboxGuid><DatabaseGuid>7bc2e8fa-9b1b-4836-9133-5b966666b466</DatabaseGuid><ServerFqdn>E15A.contoso.com</ServerFqdn></LastGeneratingData></OfflineAddressBookConfigXML>;

 


You might ask yourself: why upload those OAB files to the OAB generation mailbox at all? The answer is that this is how we maintain the continuity of OAB generation GUID2 in case of database switch/failover to another node.

So, what happens if the database fails over to another node?

At first client request for OAB download, the OABAssistant logs on to the OAB generation mailbox and then extracts the LZX files to %ExchangeInstallPath%\ClientAccess\OAB\, based on the manifest.information in the manifest file (OAB.xml).

To summarize a few points here:

  • OAB files generated on the disk are uploaded to the OAB generation mailbox
  • The files from the OAB generation mailbox are used if the database hosting OAB generation mailbox is switched/failed over.

Therefore, if the OAB manifest files cannot be (for any reason) copied from OAB generation mailbox, the OAB generation is triggered again, causing the reset of GUID2.

Troubleshooting

In our support ticket, whenever a failover happened, we observed the LZX files were stamped with a new GUID2, triggering the full OAB download on the clients.

Now that we know the OAB generation process, we can see that a new GUID at every failover can be caused by the manifest file not getting uploaded to the OAB generation mailbox.

To solve the mystery of missing OAB manifest file in the OAB generation mailbox, we checked our old friend the OABGeneratorLog and found that the manifest file was not getting uploaded to the OAB generation mailbox:

 

18352550            0             Error      12:54:14.865             13448    25652               ManagedStore.Common               ExceptionHandler             "Exception: ExExceptionMaxSubmissionExceeded Text: Microsoft.Exchange.Protocols.MAPI.ExExceptionMaxSubmissionExceeded: ErrorCode: MaxSubmissionExceeded, LID: 42792 - Message over size limitation

 

Hence, whenever there was an OAB request, the OAB generation would create a new GUID.

So, why was the Manifest file not uploaded? During the troubleshooting, we figured out that a setting called MaxSendSize on the OABGen mailbox was set to Unlimited. Well, it is interesting to note that the literal meaning of “unlimited” doesn’t apply here. This defaults to the transport config settings for the MaxSendSize, which was in this case set to 75 MB. But the size of the manifest file was 100 MB, therefore, over this limit.

This error was also reported in the OABGeneratorLog:

 

18352549            0             Error      12:54:14.862             13448    25652               ManagedStore.Mapi        Quota    Message 00000000-0000-0000-0000-000000000000[0]-0 over size limitation, current size=107762607, limitation=78643200

 

The solution here was to set the MaxSendSize on OABGen mailbox to a higher limit, to accommodate the size of the manifest file.

Hope the information here was useful in understanding and troubleshooting OAB generation and download issues!

Naveen Vasudevan

3 Comments
Version history
Last update:
‎Nov 25 2019 09:35 AM
Updated by: