Migration to Modern Public Folders - Notes from the Field

In this blog post I wanted to go through the documented TechNet process for migrating legacy public folders to Exchange 2013/2016, expanded with real world guidance gained from field support. I have been involved in a number of legacy public folder data migrations to Exchange 2013/2016 and I wanted to pass along some of the lessons learned along the way. Please note: in case of discrepancies in various CMDlets used, TechNet article is the guidance you should follow.

Note for Exchange 2007 customers reading this guidance - Exchange 2010 is the only version of Exchange explicitly mentioned in the TechNet article as this version of the guidance focuses only on Exchange 2016, and Exchange 2007 is not supported in coexistence with it. The Exchange 2010 guidance applies equally to Exchange 2007 unless otherwise noted.

Part 0: Verify public folder replication

This step is not included in the TechNet article, but it is vital to ensure that the data set being migrated is complete. If your public folder infrastructure only consists of one public folder database, you may proceed to Step 1, downloading the migration scripts. If, on the other hand, your public folders are replicated among two or more databases, you must first ensure that public folder replication between your databases is working properly. While it is ideal to have a single database that contains all of the folder replicas (for both reliability and speed of migration), it is not a requirement.

1. First, download the script attached to this blog post.

2. The output of the script is an HTML page showing several metrics, most importantly the listing of each public folder replica and the replication status of each folder expressed as a percentage. Hovering over the percentage will pop up the size and item count of a specific folder replica. 3. It is important to note that we do not use the data stored in the legacy Non_IPM_Subtree folders, such as OWAScratchPad, SCHEDULE+ FREE BUSY or OFFLINE ADDRESS BOOK in the new Exchange 2013/2016 Modern Public Folders. Only concern yourself with making sure you have one public folder database with a complete set of the user data folders.

Troubleshooting public folder replication

This section is intended to cover the most common issue related to replication, not an exhaustive reference for the task. Bill Long's Public Folder Replication Troubleshooting blog series is the first, best option for tracking down a difficult replication issue. If you do not find a solution here or in Bill's blog posts, please contact Microsoft support and open a case to resolve your issue. Not all folders have a replica in every database - in this case, use the AddReplicaToPFRecursive.ps1 script in the built in Scripts directory of your Exchange installation. Using the -Server and -ServerToAdd switches allows you to be specific about which is the source and target database in the process, so be prepared to run the script from Server1 to Server2 and vice versa to fully replicate two public folder databases. Depending on the amount of data to be replicated, it may take some time for the target database to converge. Rerun the Get-PublicFolderReplication script at intervals to verify progress.

Part 1: Download the migration scripts

1. Download all scripts and supporting files from Public Folders Migration Scripts.
2. Save the scripts to the local computer on which you’ll be running PowerShell. For example, C:\PFScripts. Make sure all scripts are saved in the same location.

Notes from the field:

Yes, the migration scripts are included in the scripts directory of the Exchange installation directory (don't forget you can use CD $Exscripts to easily change to that directory in the Exchange Management Shell), but it is recommended that you download them fresh from the link above to ensure you get the latest version.

Part 2: Prepare the Exchange 2010 server and public folders for the migration

Perform all steps in this section in the Exchange Management Shell on your Exchange 2010 server. 1. Open the Exchange Management Shell on your Exchange 2010 server. 2. For verification purposes at the end of migration, run the following commands to take snapshots of your current public folder deployment:

1. Run the following command to take a snapshot of the original source folder structure. Get-PublicFolder -Recurse | Export-CliXML C:\PFMigration\Legacy_PFStructure.xml
2. Run the following command to take a snapshot of public folder statistics such as item count, size, and owner. Get-PublicFolderStatistics | Export-CliXML C:\PFMigration\Legacy_PFStatistics.xml
3. Run the following command to take a snapshot of the permissions. Get-PublicFolder -Recurse | Get-PublicFolderClientPermission | Select-Object Identity,User -ExpandProperty AccessRights | Export-CliXML C:\PFMigration\Legacy_PFPerms.xml Save the information from the preceding commands for comparison purposes after your migration is complete.

Notes from the field:

I've seen customers use some form of file difference comparison utility to great effect here. The files generated with the previous three commands will be compared to the same files generated after the migration process is completed to verify consistency. Thus far, I've not had a customer suffer any data loss or experience permissions issues post migration. 3. If the name of a public folder contains a backslash (\), the public folders will be created in the parent public folder when migration occurs. Before you migrate, you need to rename any public folders that have a backslash in the name if you don't want this to happen.

1. To locate public folders that have a backslash in the name, run the following command. Get-PublicFolderStatistics -ResultSize Unlimited | Where {$_.Name -like "*\*"} | Format-List Name, Identity
2. If any public folders are returned, you can rename them by running the following command. Set-PublicFolder -Identity <public folder identity> -Name <new public folder name>

Part 3: Generate the CSV files

1. Open the Exchange Management Shell on your Exchange 2010 server. 2. Run the following command to create a file that maps the folder name to the folder size for each public folder you want to migrate. You'll need to specify an accessible network share where the CSV file created by the following command is run, and you'll need to specify the FQDN of your Exchange 2010 server. This command needs to be run by a local administrator and will create a CSV file that contains two columns: FolderName and FolderSize. The values for the FolderSize column will be displayed in bytes. For example, \PublicFolder01,10000. .\Export-PublicFolderStatistics.ps1 <Folder to size map path> <FQDN of source server> Example: .\Export-PublicFolderStatistics.ps1 "\\FileServer\Share\FolderSize.csv" "EX2010.corp.contoso.com" Here is a sample of what the output file will look like:

Notes from the field:

Nothing to fear on this one. It's purely informational output in the form of a .csv file. This output file becomes the input file for the next script. Feel free to run this script as needed before the migration process is actually started to get a feel for the script and to parse the output. 3. Run the following command to create the public folder-to-mailbox mapping file. This file is used to calculate the correct number of public folder mailboxes on the Exchange 2016 Mailbox server. You'll need to specify the following parameters:

• Maximum mailbox size in bytes This is the maximum size you want to set for the new public folder mailboxes. When specifying this setting, be sure to allow for expansion so the public folder mailbox has room to grow. In the command below, the value 20000000000 is used to represent 20 GB.
• Folder to size map path This is the file path of the CSV file you created when running the previous command. For example, \\FileServer\Share\FolderSize.csv.
• Folder to mailbox map path This is the file name and path of the folder-to-mailbox CSV file that you’ll create with this step. If you specify only the file name, the file will be generated in the current Windows PowerShell directory on the local computer. .\PublicFolderToMailboxMapGenerator.ps1 <Maximum mailbox size in bytes> <Folder to size map path> <Folder to mailbox map path> Example: .\PublicFolderToMailboxMapGenerator.ps1 20000000000 "\\FileServer\Share\FolderSize.csv" "\\FileServer\Share\PFMailboxes.csv"

Notes from the field:

This part deserves substantial additional explanation. First, let's talk about the size and number of public folder mailboxes to be provisioned in Exchange 2016. Using an example of a 20 GB maximum PF mailbox size (100 GB is the current maximum - see Limits for Public Folders), let's look at preparing to migrate 100 GB of public folder data. Following our guidance to only fill the public folder mailboxes to 50% of their capacity at migration, we would need 10 public folder mailboxes created to host the data. This example would modify the example command above to: .\PublicFolderToMailboxMapGenerator.ps1 10737418240 "\\FileServer\Share\FolderSize.csv" "\\FileServer\Share\PFMailboxes.csv" Finally, the output file may be manipulated prior to future steps in order to name the public folder mailboxes appropriately.

Note: in larger modern public folder installations, it's recommended for the hierarchy mailbox (the first PF mailbox to be created that has the writable copy of the folder hierarchy) to be isolated onto a database and server that hosts no other PF mailboxes. That hierarchy mailbox should be configured not to answer hierarchy requests from users in order to dedicate it to the task of maintaining the hierarchy for the environment.

Here is a sample of what the initial output file will look like: Here is a modified file with the folder structure expanded and the PF mailbox names changed: While I have your attention on the screenshot of the public folder mapping file, let me advise that on one of my engagements, there were at least a dozen or so formatting errors that halted the beginning of the public folder migration (step 5 in the process). From what I could see, Exchange 2013/2016 processes the entire mapping file before starting the move of data to ensure that it is formatted correctly because the errors that exited the command to start the migration occurred very quickly, and some of the errors were hundreds of lines into the mapping file. The tedious aspect of encountering mapping file errors like this is that the command to start the migration exits immediately and must be rerun to find the next error. After restarting the command a dozen or so times, I was quite pleased when the command finally returned that the migration was queued. The errors I encountered were not difficult to remedy. When the command errors out, it tells you which line of the file the error is located in. My recommendation is to have a utility on hand that displays line numbers (the venerable Notepad++ comes to mind) to make it easy to navigate the file. The types of errors I saw were either missing double quotes or too many double quotation marks defining the public folder mailbox or the path of the folder. As you can see in the screenshot above, the format is "PFMailbox","Total path of the folder". Look the affected line over and ensure it is correctly punctuated.

Part 4: Create the public folder mailboxes in Exchange 2016

Run the following command to create the target public folder mailboxes. The script will create a target mailbox for each mailbox in the .csv file that you generated previously in Step 3, by running the PublicFoldertoMailboxMapGenerator.ps1 script. .\Create-PublicFolderMailboxesForMigration.ps1 -FolderMappingCsv PFMailboxes.csv -EstimatedNumberOfConcurrentUsers:<estimate> Mapping.csv is the file generated by the PublicFoldertoMailboxMapGenerator.ps1 script in Step 3. The estimated number of simultaneous user connections browsing a public folder hierarchy is usually less than the total number of users in an organization.

Notes from the field:

First comment here is to determine if you really need to bother with the script to create the public folder mailboxes. If your modern public folder mailbox count to be created is significant, the script is quite efficient, but if you are only going to be creating a few mailboxes, it may be simpler to run a few quick EMS commands to put the needed mailboxes in place. Working through the process of manually creating the mailboxes also allows us to discuss two specific switches you should know about - HoldForMigration and -IsExcludedFromServingHierarchy.

• Creating the first Public Folder Mailbox. In the command below, the HoldForMigration switch prevents any client or user, except for the Microsoft Exchange Mailbox Replication service (MRS) process, from logging on to a public folder mailbox.

New-Mailbox -Name PFMB-HIERARCHY -PublicFolder –HoldForMigration –IsExcludedFromServingHierarchy:$True This first Public Folder Mailbox contains the writable copy of the Public Folder Hierarchy. In this example command, the mailbox is named for that purpose, and the -IsExcludedFromServingHierarchy switch will be set to $True permanently. Remember, it is recommended to locate the hierarchy public folder mailbox in a database with no other public folder mailboxes, mounted on a server with no other active public folder mailboxes, particularly in larger public folder environments.

• Create the Public Folder Mailboxes that will host data (based on our example .csv file above):

New-Mailbox -Name PFMB-AMERICAS -PublicFolder –IsExcludedFromServingHierarchy:$True New-Mailbox -Name PFMB-APAC -PublicFolder –IsExcludedFromServingHierarchy:$True New-Mailbox -Name PFMB-EMEA -PublicFolder –IsExcludedFromServingHierarchy:$True The -IsExcludedFromServingHierarchy switch set to $True for these mailboxes will NOT be a permanent setting here as it is for the hierarchy mailbox. Creating a new public folder mailbox without initially excluding it from serving the hierarchy creates a situation where users may connect to it for hierarchy requests before the initial hierarchy synchronization has completed. It's easy to determine if a public folder mailbox is ready to serve the hierarchy with a short Exchange Management Shell command: Get-Mailbox -PublicFolder | fl name,*hierarchy* Note the IsHierarchyReady attribute of the newly created "PFMB3" compared to the other PF mailboxes in this output below. Once that attribute reports True, the IsExcludedFromServingHierarchy switch may be set to $False. Hierarchy synchronization will occur automatically in short order, though it is possible to call the synchronizer into action immediately: Get-Mailbox -PublicFolder | Update-PublicFolderMailbox -InvokeSynchronizer A bit more on the IsExcludedFromServingHierarchy attribute from TechNet (highlight mine):

"This parameter prevents users from accessing the public folder hierarchy on the specified public folder mailbox. For load-balancing purposes, users are equally distributed across public folder mailboxes by default. When this parameter is set on a public folder mailbox, that mailbox isn't included in this automatic load balancing and won't be accessed by users to retrieve the public folder hierarchy. However, if you set the DefaultPublicFolderMailbox property on a user mailbox to a specific public folder mailbox, the user will still access the specified public folder mailbox even if the IsExcludedFromServingHierarchy parameter is set for that public folder mailbox."

Based on this guidance, there will need to be enough public folder mailboxes provisioned to accommodate no more than an average of 2,000 concurrent users (see again the TechNet article Limits for public folders). This adds another dimension to the sizing guidelines. For example, you may well be able to place the public folder data set in as few as 5 public folder mailboxes, but if your system hosts 16,000 mailbox enabled users, a minimum of 8 public folder mailboxes would be necessary to support the user connections. Be sure to validate both size and user connection counts in your modern public folder design stage.

Part 5: Start the public folder migration

At this point, you're ready to start the public folder migration. The steps below will create and start the migration batch. Depending on the amount of data in your public folders and the speed of your network connections, this could take a few hours or several days. During this stage of the migration, users will still be able to access their public folders and content on your Exchange 2010 server. In "Part 6: Complete the public folder migration (downtime required)", you'll run another sync to catch up with any changes made in your public folders, and then finalize the migration.

1. Open the Exchange Management Shell on your Exchange 2016 server.
2. Run the following command to create the new public folder migration batch. Be sure to change the path to your public folder-to-mailbox mapping file. New-MigrationBatch -Name PFMigration -SourcePublicFolderDatabase (Get-PublicFolderDatabase -Server EX2010) -CSVData (Get-Content "\\FileServer\Share\PFMailboxes.csv" -Encoding Byte)
3. Start the migration by using the following command. Start-MigrationBatch PFMigration

The progress and completion of the migration can be viewed and managed in the EAC. Because the New-MigrationBatch cmdlet initiates a mailbox migration request for each public folder mailbox, you can view the status of these requests by using the mailbox migration page. You can get to the mailbox migration page and create migration reports that can be emailed to you by doing the following:

1. Open the EAC by browsing to the URL of your Exchange 2016 Mailbox server. For example, https://Ex2016/ECP.
2. Navigate to Mailbox > Migration.
3. Select the migration request that was just created, and then click View Details in the Details pane.

The Status column will show the initial batch status as Created. The status changes to Syncing during migration. When the migration request is complete, the status will be Synced. You can double-click a batch to view the status of individual mailboxes within the batch. Mailbox jobs begin with a status of Queued. When the job begins the status is Syncing, and once InitialSync is complete, the status will show Synced.

Notes from the field:

In the deprecated "Serial Migration" TechNet article, the following guidance was given concerning the data transfer rate (highlight mine):

You’ll know that the command started successfully when the migration request reaches a status of Queued or InProgress. Depending on how much data is contained in the public folders, this command can take a long time to complete. If migration isn’t being throttled due to the load on the destination server, the typical data copy rate can be 2 GB to 3 GB per hour.

In my experience, it is difficult to set any real estimate on the data transfer rate, and the later TechNet article detailing the "Batch Migration" process on which this blog post is based has changed the verbiage to "this may take several hours". The variables of network speed and available bandwidth combined with the quality of the hardware supporting both old and new versions of Exchange will have a significant impact on your data transfer rate at this stage. The number and size of your public folders will also affect the speed of the migration. One particular migration I was involved with saw roughly 70% of the total public folder data located in one folder (high resolution pictures). During the time that particular folder was being moved, the data transfer rate was closer to 20 GB per hour, but the data rate slowed closer to the original 2-3 GB per hour estimate once the final 30% of the data was being moved from the many, smaller remaining folders. Under normal circumstances, the migration of data will progress to the 95% synchronization mark and autosuspend. If you leave the migration in the suspended state for any length of time, the migration process will resynchronize to 95% at intervals in the same manner as moving a mailbox configured for manual completion. In one of my public folder migrations, the process progressed to 95%, but the reported state did not show the desired "AutoSuspended", but rather an error. Looking at the onscreen report, the move failed due to TooManyLargeItemsPermamentException and called out two items, one 12 MB in size, the other 14 MB (organization maximum message size was 10 MB). These two items were located and saved offline from the folders by the customer using Outlook. I modified the migration request to skip the large items (simply added -LargeItemLimit 2 to the command) and resumed it, and the status moved to the AutoSuspended state. More on the LargeItemLimit switch to allow the large items to be skipped (save the items out if you want to keep them!):

The LargeItemLimit parameter specifies the maximum number of large items that are allowed before the migration request fails. A large item is a message in the source mailbox that exceeds the maximum message size that's allowed in the target mailbox. If the target mailbox doesn't have a specifically configured maximum message size value, the organization-wide value is used. For more information about maximum message size values, see the following topics:

• Exchange 2016   Understanding message size limits
• Exchange Online   Exchange Online Limits

Valid input for the LargeItemLimit parameter is an integer or the value unlimited. The default value is 0, which means the migration request will fail if any large items are detected. If you are OK with leaving a few large items behind, you can set this parameter to a reasonable value (we recommend 10 or lower) so the migration request can proceed.

Part 6: Complete the public folder migration (downtime required)

Until this point in the migration, users have been able to access the legacy public folders. The next steps will disconnect users from the Exchange 2010 public folders and will lock the folders while the migration completes its final synchronization. Users won’t be able to access public folders during this process. Also, any mail sent to mail-enabled public folders will be queued and won’t be delivered until the public folder migration is complete. Before you can finalize the migration, you need to lock the public folders on the Exchange 2010 server to prevent any additional changes by doing the following:

1. Open the Exchange Management Shell on your Exchange 2010 server.
2. Run the following command to lock the legacy public folders for finalization. Set-OrganizationConfig -PublicFoldersLockedForMigration:$true

If your organization has multiple public folder databases, you’ll need to wait until public folder replication is complete to confirm that all public folder databases have picked up the PublicFoldersLockedForMigration flag and any pending changes users recently made to folders have replicated across the organization. This may take several hours.

Notes from the field:

When you have progressed to this step, there is one helpful tip I've found to speed up the process. After running the command Set-OrganizationConfig -PublicFoldersLockedForMigration:$true, restart the information store service on the legacy Exchange server involved in the migration. This will refresh the Store cache on that server that the public folders are locked for migration, allowing you to move immediately to the Step 7 command Complete-MigrationBatch PFMigration. Considering that all your mailboxes have been moved before you migrate public folders anyway, there should be nothing to stop you from recycling the information store service. If you are unable to do this, the command to complete the migration batch in step 7 will not proceed until the information store service on the legacy server picks up the change.

Part 7: Finalize the public folder migration (downtime required)

First, run the following cmdlet to change the Exchange 2016 deployment type to Remote: 1. Set-OrganizationConfig -PublicFoldersEnabled Remote Once that is done, you can complete the public folder migration by running the following command: 2. Complete-MigrationBatch PFMigration Or, in EAC, you can complete the migration by clicking Complete this migration batch. When you complete the migration, Exchange will perform a final synchronization between the Exchange 2010 server and Exchange 2016. If the final synchronization is successful, the public folders on the Exchange 2016 server will be unlocked and the status of the migration batch will change to Completing, and then Completed.

Notes from the field:

If you have advanced to this point, it's likely that the final 5% synchronization of data will finish smoothly and the process will reach the Completed status. The only real note here is that 5% is a relative number. If your legacy public folder installation is 17 GB as one of my customers was, that's only about 870 MB to finish syncing. If your legacy install has 1.5 TB of data, you still have multiple hours of sync time remaining.

Part 8: Test and unlock the public folders

After you finalize the public folder migration, you should run the following test to make sure that the migration was successful. This allows you to test the migrated public folder hierarchy before you switch to using Exchange 2016 public folders.

1. Open the Exchange Management Shell on your Exchange 2016 server.
2. Run the following command to assign some test mailboxes to use any newly migrated public folder mailbox as the default public folder mailbox. Set-Mailbox -Identity <Test User> -DefaultPublicFolderMailbox <Public Folder Mailbox Identity>
3. Open Outlook 2010 or later using the test user identified in the previous step, and then perform the following public folder tests:
   • View the hierarchy
   • Check permissions
   • Create and delete public folders
   • Post content to and delete content from a public folder
4. If everything looks okay, run the following command to unlock the public folders for all other users. Get-Mailbox -PublicFolder | Set-Mailbox -PublicFolder -IsExcludedFromServingHierarchy $false
5. On the Exchange 2010 server, run the following command to indicate that the public folder migration is complete. Set-OrganizationConfig -PublicFolderMigrationComplete:$true
6. After you've verified that the migration is complete, run the following command on the Exchange 2016 server. Set-OrganizationConfig -PublicFoldersEnabled Local

Notes from the field:

At this point, your Modern Public Folders are accessible to the users. Outlook will determine the updated location of the public folders via AutoDiscover, so it may be necessary to reopen Outlook in order to force an AutoDiscover query for the Public Folder list to show up and be accessible. Outlook performs an AutoDiscover query at startup and at regular intervals thereafter, so any open clients will eventually pick up the change under any circumstance. Hope this is helpful! I want to acknowledge the fantastic help I received from Bill Long for reviewing the replication script and general knowledge transfer, Brian Day for his generous time and explanations during one particular public folder critical situation, and Dhaval Hansaliya and Nino Bilic for their assistance reviewing this post to get it ready for publishing. Thank you!

Butch Waller

Premier Field Engineer