Blog Post

Exchange Team Blog
8 MIN READ

How Exchange Online automatically cares for your public folder mailboxes (via AutoSplit)

The_Exchange_Team's avatar
Jan 11, 2021

When your public folder content migration to Exchange Online is complete or you create public folders for the very first time, you do not have to worry about managing many aspects of public folders. As you previously read, Modern public folders starting with Exchange Server 2013 are stored within a new mailbox type in the mailbox database. Our on-premises customers have to create public folder mailboxes, monitor their usage, create new public folder mailboxes when necessary, and split content to different public folder mailboxes as their content grows over time. In Exchange Online we automatically perform the public folder mailbox management so you may focus your time managing the actual public folders and their content. This is what we refer to as ‘AutoSplit’. If we were to peek behind the Exchange Online curtain, we would see two automated processes always running to make this happen:

  • Automatic public folder moves based on public folder mailbox quota usage
  • Automatic public folder mailbox creation based on active hierarchy connection count

Let’s go through each one of them, shall we?

Automatic public folder moves based on public folder mailbox quota usage

This process actively monitors your public folder mailbox quota usage. The goal is to ensure you do not inadvertently overfill a public folder mailbox and stop it from being able to accept new content for any public folder within it.

How does it work?

We have a time-based assistant (known as PublicFolderSplitProcessor) actively monitoring the public folder mailbox quota usage. The assistant scans each public folder mailbox every hour to ensure we do not inadvertently overfill a public folder mailbox and stop it from being able to accept new content for any public folder within it.

Public folder mailbox’s quota for any new mailboxes has been set to the following (regardless of a mailbox plan or policy which may apply to user mailboxes):

  • ProhibitSendQuota = 99GB
  • ProhibitSendReceiveQuota = 100GB

For public folder mailboxes to undergo a split, the size should exceed the following percentage of the ProhibitSendQuota:

  • Primary mailbox = 40%
  • Secondary mailboxes = 80%

During the split, public folders from source PF mailbox (the one that is reaching its size limit) are moved to either a new target PF mailbox (which is created automatically by the system) or an existing PF mailbox that still has space to accommodate new content.

 

In the above example from tenant audit logs, the system created a new public folder mailbox “AutoSplit_GUID” and initiated a new public folder move request for public folder data to move to the newly created target AutoSplit mailbox.

When creating a new mailbox, Exchange online will always exclude it from serving hierarchy since a newly created public folder mailbox would not have the folder hierarchy synced yet.

If the total mailbox count is > 90% of the PublicFolderMailboxCountQuota (count > 900, with current Exchange Online limits), the system randomly chooses an existing mailbox (if any exist) with < 50% of the quota used where there is no moveRequest (by AutoSplit) to/from this mailbox. Else, we proceed to AutoSplit a mailbox with name: AutoSplit_<random guid (without “–“ in between)> until PublicFolderMailboxCountQuota ”1000” has been reached.

Naming of the new target mailbox

The system created new public folder mailbox has the following naming convention. The guid has some information regarding the source mailbox from which the split happened.

In this example, public folder mailbox “PFMBX” had crossed the split threshold; the Exchange guid of this source mailbox is 3a40f5da-0e0a-42f3-910f-1849580ff25d, the first part of the guid is always shared with the name of the target AutoSplit mailboxes.

If we had to find out the AutoSplit targets of a source mailbox, we can use following command in EXO PowerShell:

Get-Mailbox -PublicFolder “AutoSplit_3a40f5da*”

Example:

In a single move, a maximum of 20GB (in total size) or 2000 (+2000 dumpster) folders are moved (whichever is lower in size). There can be multiple moves to the same target mailbox if the target mailbox’s size is still lower than 30 GB, else the system would choose a new target mailbox. Public folders are chosen with consideration of their subtree structure, so there is no guarantee that 20GB or 2000 folders will be moved every time.

To help illustrate this, here is an example; let's say we have 4 PFs in the public folder mailbox where AutoSplit process is taking place:

\PF1 = 13 GB
\PF2 = 1 GB
\PF3 = 3 GB
\PF4 = 4 GB

Here, \PF1,\PF2 & \PF3 are going to be chosen as they are up to the limit of 20GB and were chosen while considering folder hierarchy.

In another scenario, let's say we have 4 PFs in the public folder mailbox where AutoSplit process is taking place:

\PF1 = 13 GB
\PF1\PF4 = 7 GB (PF4 is a subfolder of PF1 and the size of PF4 content is 7 GB)
\PF2 = 1 GB
\PF3 = 3 GB

Here, \PF1 and \PF1\PF4 are going to be chosen as they are up to the limit of 20GB and were chosen with consideration of subtree structure.

Note that the AutoSplit process will drain the source mailbox entirely, moving the contents completely to other mailboxes.

For example: 80GB public folder mailbox might copy the contents inside to 4 or more new target AutoSplit mailboxes (providing we are not above 900 mailboxes total). The source mailbox will eventually become empty once the moved folders are deleted after the item retention. Then our empty mailbox removal processor may remove it, if it is not serving hierarchy and no new public folder was added to it.

Automatic public folder mailbox creation based on active hierarchy connection count

This is the second automated process to help maintain the most optimal user experience accessing public folders in Exchange Online. Exchange Online will actively monitor how many hierarchy connections are being spread across all of your public folder mailboxes. If this value goes over a pre-determined number (2000), we will automatically create a new public folder mailbox. Creating the additional public folder mailbox will reduce the number of hierarchy connections accessing each public folder mailbox by spreading user connections across a larger number of public folder mailboxes. If you have a small amount of public folder content in Exchange Online, yet you have an extremely large number of active users, then you may see the system create additional public folder mailboxes regardless of your public folder content size.

In the above example, public folder mailbox “PFMBX-005” was created by the service to overcome the measured high load of user connections to existing PF mailboxes serving hierarchy.

The public folder mailboxes created by the system for balancing hierarchy connections will be named “PublicFolderMailbox_(Date)”.

In this example, public folder mailbox “PublicFolderMailbox_2020_05_25” was created by the service on the date of 25th May 2020 to address the user connection load.

Daily auto-generated public folder mailboxes

If there is a user traffic spike, or if you have a tenant with recently migrated public folders, the system might create multiple hierarchy sync mailboxes. We do not create all mailboxes that might be needed in a single day; the system creates these mailboxes once per day, to avoid provisioning lots of auto-generated mailboxes together. The creation would stop once we have enough mailboxes to serve hierarchy to users.

In this example, multiple public folder mailboxes with naming convention “PublicFolderMailbox_Date” were created by the service to overcome the user spike on 25th May 2020 till 12th June 2020 to maintain the optimal user experience accessing public folders in Exchange Online.

Common issues with AutoSplit

The following are some of the scenarios under which the AutoSplit process may encounter issues and also tips on how to deal with these scenarios.

Giant folder and auto-split:

A single public folder occupying more than 40-80% space of the public folder mailbox could be called a “Giant folder”. The AutoSplit process is not helpful in giant public folder scenarios, since it can only distribute the folders into multiple mailboxes. If a single folder taking up all the space, the problem (the folder size) remains the same wherever that folder goes. Remember, a single folder cannot span multiple public folder mailboxes.

When there is only a single resident giant folder+dumpster residing in a mailbox, and total mailbox size has exceeded split threshold (size/quota>=80%), system checks to see if the ratio of the (resident giant folder size/total mailbox size) exceeds 99% and will split only if it is less. This is to ensure that large amounts of mailbox data which is under move retention does not cause the mailbox to hit quota, and cause a data loss for the giant folder.

Action for tenant administrators:

In such a scenario, tenant admin must advise affected public folder owners to either delete or move items from that giant folder to reduce the size of that public folder to an appropriate size. A general recommendation is that individual public folder size shouldn’t exceed 20GB.

MRS related issues causing AutoSplit failure

AutoSplit uses Mailbox Replication Service (MRS) framework to move the public folders from source PF mailbox (mailbox that is getting full) to target PF mailbox (new empty PF mailbox). If MRS issues are transient in nature, the AutoSplit will be delayed but will complete.

Tenant administrators can track the MRS related failures using Get-PublicFolderMailboxDiagnostics command under AutoSplitInfo section.

In the following example, MRF:0 reads Move Requested Failed 0 times (other example could be like MRF: 2 means Move Request Failed 2 times), and can be seen in get-publicfoldermailboxdiagnostics under AutoSplitInfo section:

When something fails in AutoSplit, we handle the exception and increment the bucket counter with the hope that AutoSplit auto recovers in the next attempt. When these bucket counts hit certain thresholds, the AutoSplit state becomes Halted. This is temporarily stopping AutoSplit attempts, as they failed too many times.

Action for tenant administrators:

In such cases, please raise a support request with Microsoft and include Get-PublicFolderMailboxDiagnostics output for the affected public folder mailbox.

Moveditemretention and AutoSplit

We have seen some support tickets where users might complain to Admins that they are not able to post new items to a public folder or send an email to it (folder was mail-enabled). We found the ProhibitSendQuota was reached and by checking publicfoldermailboxdiagnostics, also saw that ReducedStateProgressState “auto-split process status” was SplitCompleted and ReducedStateTargetMailboxWhenCreatedUTC “date of creation of target public folder mailbox where the data of source public folder mailbox was split to it” was recent.

Action for tenant administrators:

First, find out PF mailbox that is at full quota:

$PFmbx = Get-Mailbox -PublicFolder
$PFmbx | ft Name,Prohibit*Quota*

The public folder mailbox PFMBX1 where ProhibitQuota values were modified to smaller values 

$PFmbxstats = $PFmbx | Get-MailboxStatistics
$PFmbxstats | fl DisplayName,TotalItemSize

PFMBX1 total item size has exceeded ProhibitSendQuota value

If TotalItemSize is less than a quota, there is no further action since AutoSplit completed successfully and data is moving. Otherwise, proceed to next step.

Then, Check if MovedItemRetention is keeping the mailbox full. Even though AutoSplit completed successfully, you might reduce DefaultPublicFolderMovedItemRetention to be 1 day and then invoke mailbox assistant to process the mailbox.

Run:

Set-OrganizationConfig -DefaultPublicFolderMovedItemRetention 1.00:00:00

Update-PublicFolderMailbox <AffectedPFMailbox>

After a few hours, check again:

$PFmbxstats = $PFmbx | Get-MailboxStatistics
$PFmbxstats | fl DisplayName,TotalItemSize

PFMBX1 total item size has been reduced

Finally, check if this TotalItemSize has now reduced. If the size is reduced, then the issue is fixed, and you may set the MovedItemRetention back to default value of 7.00:00:00. Otherwise, please raise a support request with us for further investigation.

Public folder content not accessible during AutoSplit

During the move, the source mailbox will continue to serve the content; however, as PublicFolderMoves are auto-complete in nature, the cut-over (copying the delta, updating content mailbox info on primary, source and target, directory updates in case of mail enabled public folders, etc.) may cause temporary service interruptions. The content access will be resumed as soon as the move completes.

In closing

We hope this has been a useful view into processes that we have that will help you manage your public folder content! In most cases, we hope that you never have to do anything with this. But if you see some public folder mailboxes that you do not recognize, this post hopefully explains why!

Special thanks to the public folder experts who reviewed and contributed to this post: Bhalchandra Atre, Dheeraj Ram, Deepak Sadan, Nino Bilic.

Hazem Embaby

Updated Jan 11, 2021
Version 2.0