Want more control over Sent Items when using shared mailboxes?
Edit 6/15/2015: We are starting to make this feature available again in Office 365. Note that the new shared mailbox Sent Items behavior is disabled by default. If you want to enable it for your users, you can do so by using cmdlets mentioned below. Additionally, we are still on track to ship this to our on-premises customers in Exchange 2013 CU9, as mentioned before. Whether a mailbox is used by multiple users as a collaborative tool or a communication gateway to customers, retaining a record of emails sent from a shared mailbox remains an important business requirement. In Exchange 2010, there was a way to configure this behavior, but we did not have this feature starting with Exchange 2013. Our customers have told us that a shared mailbox should keep a copy of emails sent from the mailbox by all members of the mailbox in its own Sent Items folder. We have taken that feedback and decided to make some changes to how sent mails are handled for shared mailboxes. We are excited to announce that once this feature is enabled for you (see below), by default all shared mailboxes will retain a copy of emails sent from the mailbox. You will no longer have to figure out which mailbox member sent an email as the shared mailbox or on behalf of it. How does it work? Emails can be sent as the shared mailbox itself or on behalf of it by member(s) of the mailbox, assuming proper permissions have been granted. This feature is designed to retain a copy of an email sent from the shared mailbox in the Sent Items folder of the shared mailbox. The same behavior can be expected for emails sent on behalf of the shared mailbox, when configured to do so. A copy of the sent mail will also reside in the Sent Items folder of the member’s personal mailbox. Note: If the user has used the Outlook 2013 feature to change the folder that Sent Items are saved to, the messages will be copied to that folder instead of the user’s Sent Items folder. Users can reconfigure this by clicking the “Save Sent Items To” button on the Email Options tab. Administrators have control over this feature for either mails Sent As or Sent on Behalf of a shared mailbox. The table below summarizes where sent mails reside when members of a shared mailbox send mail from the shared mailbox. User Mailbox Shared Mailbox Sent Items Exchange 2010 Exchange 2010 Controlled through settings in KB2632409 Exchange 2010 Exchange 2013 (any version) Controlled through settings in KB2632409 Exchange 2013 CU9 (or newer) Office 365 Exchange 2010 The sent mail will be delivered to both the Inbox of the shared mailbox as an email attachment* and to the user mailbox' sent items Exchange 2013 CU9 (or newer) Office 365 Exchange 2013 CU9 (or newer) Office 365 The sent mail will be delivered to the Sent Items folder of shared mailbox and to the user mailbox' sent items *In a scenario where the user’s mailbox is on an Exchange 2013 CU9 server (or an Exchange 2013 without CU9 installed) and the shared mailbox is on an Exchange 2010 server, the shared mailbox will get an email message that looks like the following: This behavior will continue until the shared mailbox is moved to an Exchange 2013 CU9 server. Because this can happen even between Exchange 2013 servers (pre-CU9 and CU9), you might want to wait in turning the feature on for your on-premises deployment until you have completely rolled out Exchange 2013 CU9 to all of your servers. Who can use this feature? The feature is going to be available to all customers with shared mailboxes in Office 365 (phased rollout starting on 6/15/2015), as well as our on-premises customers (starting with Exchange 2013 CU9). Note: if after installation of Exchange 2013 CU9 in an on-premises environment you do not see the new CMDlet parameters, you should run /preparead explicitly with CU9 setup. How do I enable/disable this feature? In Office 365 and Exchange 2013 CU9, this feature is disabled by default. Enable the feature In Office 365, using the Admin portal, you can select your shared mailbox by going to Groups > Shared mailboxes first: Then Edit the Sent items copy status of the mailbox: Using PowerShell, for emails Sent As the shared mailbox: set-mailbox <mailbox name> -MessageCopyForSentAsEnabled $True For emails Sent On Behalf of the shared mailbox: set-mailbox <mailbox name> -MessageCopyForSendOnBehalfEnabled $True Disable the feature For messages Sent As the shared mailbox: set-mailbox <mailbox name> -MessageCopyForSentAsEnabled $False For emails Sent On Behalf of the shared mailbox: set-mailbox <mailbox name> -MessageCopyForSendOnBehalfEnabled $False What else do I need to know? The administrator for your organization has to create the shared mailbox and add you to it as an user, before you can use it. If you are an Office 365 Small Business administrator, see Create and use shared mailboxes. If you are an administrator for a different version of Office 365 or an on-premises Exchange administrator, see the TechNet article Create a Shared Mailbox. If you are currently using a client registry key to duplicate messages sent from a shared mailbox to a sent items folder of the sender, you should remember to remove client registry keys if you enable this feature server side. If you do not remove client registry keys, you could see duplicate messages in Sent Items folders (one created by client registry keys and one by the server). If one of the transport rules prevents messages larger than certain size and the message being sent is over that size, then the message will not be delivered to the its original recipient and will also not be copied to the sent items of the shared mailbox. Philip LohHow to Export and Import mailboxes to PST files in Exchange 2007 SP1
There might be times when an Exchange Administrator will need to export the contents of individual mailboxes to offline files in order to present specific users with a format that is easily portable and ready to consume using Outlook clients. To fulfill this need Exchange 2007 SP1 will have a new set of features to export and import mailboxes to and from PST files. As I know you will ask - yes, those PST files can be bigger than 2 GB, which was a limitation of Exmerge tool used for this purpose in previous versions of Exchange. Export/Import to PST Requirements In order to export or import mailboxes to PST files the following requirements must be met: Export/Import to PST must be run from a 32 bit client machine with Exchange Management Tools installed (Version Exchange 2007 SP1 or later). The 32bit requirement comes from a dependency with the Outlook client. Either Outlook 2003 or Outlook 2007 must be installed on the client machine. The user running the task must be an Exchange Organization Admin or an Exchange Server Admin on the server where the mailbox to export/import lives. Exporting mailboxes to PST files The most basic cmdlet to export a mailbox to a PST file is as follows: Export-Mailbox –Identity <mailboxUser> -PSTFolderPath <pathToSavePST> PSTFolderPath must be a full path pointing either to a directory or to a (.pst) file. If a directory is specified a PST file named after the mailbox alias will be used as the target of the export. Note that if the PST file already exists the contents of the mailbox will be merged into it. Example: After the cmdlet finishes execution, the .pst file will be ready in the specified location: To export multiple mailboxes to their respective .pst files at once you can pipe in the identities of those mailboxes to the export task. Notice that when bulk exporting the PSTFolderPath parameter must forcefully point to a directory since one .pst file will be created for each mailbox. Example: Get-Mailbox -Database 'MDB' | Export-Mailbox -PSTFolderPath D:\PSTs Importing mailboxes from PST files The process for importing mailbox contents from a PST file is quite similar: Import-Mailbox -Identity <mailboxUser> -PSTFolderPath <PSTFileLocation> Again, PSTFolderPath must be the full path to the directory where the .pst file lives or to the (.pst) file itself. In the case where PSTFolderPath points to a directory the cmdlet will try to match the mailbox alias with the name of an existing .pst file in the specified directory and import the content of that file. Example: Just as with the export to PST scenario, when bulk importing mailboxes the PSTFolderPath must forcefully point to a directory and the task logic will try to match mailboxes alias with the .pst file names under that location. If no match is found for a particular mailbox, that mailbox will be skipped. Example: Get-Mailbox -Database 'MDB' | Import-Mailbox -PSTFolderPath D:\PSTs Filtering content in Export/Import to PST When only specific content is desired in the PST file (or back into the mailbox) a common set of filters can be used to leave out the rest of the messages. Export/Import to PST support the following filters: Locale, StartDate, EndDate, ContentKeywords, SubjectKeywords, AttachmentFileNames, AllContentKeywords, SenderKeywords, and RecipientKeywords. Example: Import only those messages that were created between 1/1/06 and 12/1/06 and contain the word "review" in the subject and any of the words {"project","alpha"} in the body. Import-mailbox -Identity ricardr -PSTFolderPath D:\PSTs -StartDate 1/1/06 -EndDate 12/1/06 -SubjectKeywords:'review' -ContentKeywords:'project','alpha' Now, we realize that you would like to try this today, but please be patient! - Ricardo Rosales GuerreroExchange 2013/2016 Monitoring Mailboxes
Note 10/2022: This article also applies to Exchange Server 2019. Introduction Exchange Server 2013 introduced a new feature called Managed Availability, which is a built-in monitoring system with self-recovery capabilities. Managed Availability performs continuous tests (probes) that simulate end-user actions, to detect possible problems with Exchange components or their dependencies. If probes are failing, it performs gradual simple recovery actions to bring the affected component in healthy state. It uses special type of mailboxes, called Monitoring Mailboxes or health mailboxes, to simulate end-user kinds of tests. The life cycle of monitoring mailboxes is taken care entirely by Managed Availability components. In this post, we’ll see how Managed Availability takes care of monitoring mailboxes, what best practices to keep monitoring mailboxes happy are and some related troubleshooting. Monitoring Mailboxes Functioning of Managed Availability is implemented by Microsoft Exchange Health Manager Service running on each Exchange Server 2013 role. The Microsoft Exchange Health Manager Service is responsible for creating and maintaining monitoring mailboxes Let’s take a look at how the Health Manager creates them! How do we create monitoring mailboxes? The MS Exchange Health Manager Service runs in two processes, MSExchangeHMHost.exe and MSExchangeHMWorker.exe (let’s call it the HM worker). HM worker process, at the time of startup, checks for availability of monitoring mailboxes and creates monitoring mailboxes as needed. Starting with Exchange Server 2013 Cumulative Update 1, accounts for monitoring mailboxes are created under following container in the domain Exchange server resides: <ADdomain>\Microsoft Exchange System Objects\Monitoring Mailboxes Example: The logic HM worker process uses to detect and create monitoring mailboxes depends on Exchange Cumulative Update (CU) installed, Exchange role installed on the box and mailbox databases present. The Following logic was used to create monitoring mailboxes for Exchange Server 2013 servers between RTM to Cumulative Update 5: One monitoring mailbox per mailbox database copy, plus one for all of CAS servers. Here’s an example of monitoring mailboxes created on Exchange Server 2013 SP1 server that hosts both CAS and mailbox role: [PS] C:\>Get-Mailbox -Monitoring | ft displayname,whencreated DisplayName WhenCreated ----------- ----------- HealthMailboxb285a119be6649b3a89574f078e947f5 11/10/2014 9:07:29 AM HealthMailbox60d8a8d1285e41bfa5ce1ef1fb93d14e 11/10/2014 9:07:36 AM The display name of the monitoring mailbox created for database copy contained the GUID of the mailbox database for which it was created. Example: The display name of the monitoring mailbox created for CAS server contained the GUID of the Exchange server for which it was created. Example: The following logic is used to create monitoring mailbox for Exchange Server 2013 Cumulative Update 6 and onwards: One monitoring mailbox for each mailbox database copy hosted on mailbox role, plus ten monitoring mailboxes for each CAS server. The following naming convention is used for the display name of the monitoring mailbox created for a database: “HealthMailbox-“ + host name of server + “-“ + database name. At the startup, the HM worker checks for name of databases present on the server and then checks for presence of monitoring mailboxes with display name as explained above. If it doesn’t find a monitoring mailbox for a specific database, it creates a new monitoring mailbox. That means, if you rename DB1 to DB2, HM worker will create new monitoring mailbox for DB2 on next startup. And the following naming convention is used for display name of the monitoring mailbox created for a CAS server: “HealthMailbox-“ + host name of the CAS server + “-001” through “010.” We attempt to distribute the monitoring mailboxes created for CAS servers across the mailbox databases available. Let’s make this real: Imagine that you have only one Exchange server running Exchange 2013 CU6 or newer in the org (server is named EXCH1) that hosts both CAS and Mailbox role and has only one mailbox database, 11 monitoring mailboxes will be created. Example: The Health Manager Service will create more mailboxes according to the logic explained above as you go on adding more databases or server roles. Password resets HM Worker is responsible for maintaining the password for Monitoring mailboxes. HM worker uses a complex algorithm to generate password to be used for monitoring mailbox. The password for monitoring mailbox is reset under the following conditions: A new health mailbox is being created Each time HM Worker process starts and is not able to retrieve the existing password for the monitoring mailbox Any other scenario where HM Worker is not able to get hold of existing password for the monitoring mailbox Best practices Here are some best practices regarding management of user accounts associated with monitoring mailboxes as well as mailboxes themselves: Do not apply third party customized password policies to user accounts of monitoring mailboxes Exclude monitoring mailboxes from user account lockout policies Do not move the user accounts out from the Monitoring Mailboxes container Do not change the user account properties, like restricting password change etc. Do not change AD permission inheritance Since HM worker handles password resets for monitoring mailboxes, in a large environment, it is normal to see increased password reset traffic for monitoring mailbox accounts; note that doing one of the things above might increase the frequency of those resets Do not move the monitoring mailboxes between mailbox databases Do not apply mailbox quotas to monitoring mailboxes If applying a retention policy, ensure the data within the monitoring mailbox is retained for a minimum of 30 days before being deleted If you see a mailbox size increasing significantly for specific monitoring mailbox; you can simply disable the specific mailbox. The HM worker will create a new mailbox at next startup. Common tasks with monitoring mailboxes How to list monitoring mailboxes The Get-Mailbox command is provided with special parameter “-Monitoring” to list only the monitoring mailboxes. Here are some examples: To list all monitoring mailboxes present in the organization: Get-Mailbox –Monitoring To list monitoring mailboxes present on a database: Get-Mailbox –Monitoring -Database <database name> However, the mailboxes listed above may not be associated with the server on which database is hosted. As explained in creation logic, the name of Client Access Server or the mailbox database is important in searching for monitoring mailbox associated and in creation of monitoring mailbox. Use the following command to list the monitoring mailboxes associated with a specific server: Get-Mailbox -Monitoring | ?{$_.DisplayName -like "*-<servername>-*"} Example: Get-Mailbox -Monitoring | ?{$_.DisplayName -like "*-exch2-*"} | ft name,displayname,database Use the following command to list the monitoring mailbox associated with a specific database: Get-Mailbox -Monitoring | ?{$_.DisplayName -like "*-<name of database>"} Example: Get-Mailbox -Monitoring | ?{$_.DisplayName -like "*-db2exch2"} | ft name,displayname,database,servername Troubleshooting tips Here are some troubleshooting methods for monitoring mailboxes. How to re-create monitoring mailboxes (NOT considered regular maintenance!) The mailbox database removal doesn’t cleanup the AD user account associated with monitoring mailboxes. This can then result in orphaned AD user accounts. This happens because of deny permission inherited on Monitoring Mailboxes container. KB article 3046530has details on this, as well as the workaround to resolve it. If there are too many orphaned monitoring mailbox accounts, you may want to re-create them. Steps: 1) Make sure “Monitoring Mailboxes” container is present Open Active Directory Users & Computers Click on View and select “Advanced Features” The Browse to Microsoft Exchange System Objects Verify the presence of the “Monitoring Mailboxes” container. Example: If the Monitoring Mailboxes container is missing: Make sure you have Exchange Server 2013 CU1 or above installed. Perform PrepareAD with the Exchange Server 2013 version installed. 2) Stop the “Microsoft Exchange Server Health Manager” service on all Exchange Server 2013 servers. 3) Open Exchange Management Shell and use following command to disable existing health mailboxes: Get-Mailbox -Monitoring | Disable-Mailbox 4) Go back to Active Directory users & computers, right click on domain and search for “HealthMailbox” 5) Delete the health mailbox user accounts. 6) Wait for AD replication or force AD replication. 7) Start the “Microsoft Exchange Server Health Manager” on all Exchange Server 2013 servers. Bhalchandra Atre Updates 6/2/15: Updated best practices to include information on aging out data via retention policies.Managing OAB in Exchange Server 2013
The Exchange team blog article OAB in Exchange Server 2013 introduced the new Offline Address Book (OAB) generation and distribution architecture in Exchange Server 2013. Take a few moments to visit the article if you haven’t seen it yet or re-visit it for a quick refresher. The OAB management and administration is different in Exchange 2013 because of architecture changes. Additionally, the new Exchange Admin Center does not currently have options for managing OABs. This means that, at this time, you will need to use Exchange Management Shell for OAB-related tasks. This article takes you through commonly performed tasks in OAB administration and has a couple of real life scenarios to help understand the tasks better. Note: If you are in multi-forest Active Directory domain environment, make sure the Shell session has ViewEntireForest is enabled, else some of the commands in the article won’t return any output. Command to enable ViewEntireForest: Set-ADServerSettings -ViewEntireForest $true Creating a new OAB Creating a new OAB in Exchange 2013 no longer uses the -Server parameter. In order to create a new OAB, you should only specify the address lists to be required. The following example creates OAB for address list named “Global Address List FAB” New-OfflineAddressBook -Name OAB-FAB -AddressLists "Global Address List FAB" Identify the OAB generation server(s) The arbitration mailboxes in Exchange Server 2013 are assigned certain “Persisted capability” that defines the purpose/function of the arbitration mailbox. An arbitration mailbox with Persisted Capability “OrganizationCapabilityOABGen” is responsible for OAB generation. We will refer this mailbox as “Organization Mailbox” throughout the article. Exchange Server 2013 mailbox server hosting the Organization Mailbox will generate all OAB’s defined in the environment. For a non-DAG environment, use following command to identify the OAB Generation servers: Get-Mailbox -Arbitration | where {$_.PersistedCapabilities -like "*oab*"} | ft name,servername For a DAG environment, identifying OAB generation server(s) is a two-step process. Step1: Identify the mailbox database hosting organization mailbox with OAB Gen capability. Use the following command to list the arbitration mailboxes with persisted capability of OABGen and database on which this mailbox is hosted: Get-Mailbox -Arbitration | where {$_.PersistedCapabilities -like "*oab*"} | ft name,database Step2: Identify the mailbox server where the database hosting organization mailbox is mounted Use following command to identify active copy of mailbox database: Get-MailboxDatabaseCopyStatus db1 The server where database status is “mounted” is the current OAB generation server. Change the OAB generation server There are two methods of changing the OAB generation server. Move mailbox Move the organization mailbox to a mailbox database on a server intended to be designated as OAB Generation server. Example: DB1 is a single copy database present on the server Exch1 and hosts the organization mailbox. DB2 is mailbox database present on Exch2. The following command can be used to move the organization mailbox to DB2 and make Exch2 the OAB generation server. Get-Mailbox -Arbitration -database db1| where {$_.PersistedCapabilities –like “*oab*”} | New-MoveRequest -TargetDatabase db2 This method is more suited for environments that have single copy of mailbox database hosting the Organization Mailbox. Activate the mailbox database on another server This method is suited for environments that have multiple copies of the mailbox database hosting Organization Mailbox. Example: DB1 hosts the Organization Mailbox and has copies on servers Exch1 and Exch2. DB1 is currently active on Exch1. The following command can be used to activate DB1 on Exch2 and therefore make it the OAB generation server: Move-ActiveMailboxDatabase DB1 -ActivateOnServer Exch2 Note: Review guidelines mentioned in “Placement of Organization Mailbox” below before changing the OAB Generation server. Creating a new Organization Mailbox Administrators can create additional Organization Mailboxes for fault tolerance or for serving users in a geographically disbursed Exchange deployment. Creating a new Organization Mailbox is a two step process: Step1: Create a new arbitration mailbox New-Mailbox -Arbitration -Name "OAB Seattle" -Database DB2Seattle -UserPrincipalName oabs@contoso.com –DisplayName “OAB Mailbox for Seattle” Step2: Enable OABGen capability Set-Mailbox -Arbitration oabs -OABGen $true -MaxSendSize 1GB Note: Review guidelines mentioned in “Placement of Organization Mailbox” below before creating additional organization mailboxes. Changing the OAB Generation Schedule The OAB Generation till Exchange Server 2010 was based on a “Schedule” set on OAB properties. You might see a “Schedule” defined when viewing properties of Exchange 2013 OAB. But, the Exchange Server 2013 OAB generation does not take place according to the “Schedule” defined on OAB properties: Instead, Exchange Server 2013 OAB Generation takes place according to OABGeneratorWorkCycle and OABGeneratorWorkCycleCheckpoint properties configured at the Mailbox Server. Example: The values in the above screenshot mean OAB is generated once every day. Which Mailbox Server processed the OAB download request? The Exchange Server 2013 CAS role proxies the OAB download request to an appropriate Mailbox role server. The CAS role maintains log of each request it handles in the log files, present in folder %ExchangeInstallPath%\Logging\HttpProxy\OAB\ These log files are an excellent tool to identify which mailbox server the CAS chose to serve the request. Information of some important fields from the log file: Field Description UrlStem Useful to identify the OAB being downloaded and also if this was a full download or incremental download AuthenticatedUser Name of the User requesting the OAB AnchorMailbox DN of Organization Mailbox identified as the closest to serve the OAB request ServerHostName CAS Server Name handling the request HttpStatus Status code for Proxy action ProxyAction Action CAS Server performed for the request, it will be mostly “Proxy” for Exchange 2013 OAB TargetServer Name of Mailbox role server to which request was proxied The log file can be imported in Excel for better readability. Example: Forcing the OAB Generation The Exchange Server 2013 OAB generation can be forced to start immediately by two methods. Method 1: Update-OfflineAddresBook Below command will force OAB generation of an OAB named "Default Offline Address Book" across all organization mailboxes. Update-OfflineAddressBook "default offline address book" Note: This command initiates an RPC request to each mailbox server hosting an active organization mailbox. Method 2: Restart the Mailbox Assistant service. The Microsoft Exchange Mailbox Assistant service on Mailbox Role is responsible for generating OAB. Restarting this service generates all OAB’s defined in the environment on a specific mailbox server, if it’s hosting an active organization mailbox. Placement of Organization Mailbox Exchange Server 2013 CAS role proxies the OAB download request to a “nearest” mailbox server hosting an active Organization Mailbox. It can proxy the request in round robin fashion if it finds more than one organization mailbox active in same AD site. Prior to CU5, this will result in frequent full OAB downloads and is therefore, not recommended. Hence, current guidance is to plan organization mailbox placement such that you will have one organization mailbox active in an AD site. This applies to creating a new organization mailbox as well as to creating copies of mailbox database that hosts an organization mailbox. Prior to CU5, customers should only deploy a single OAB generation mailbox per Exchange organization to prevent users from accessing different OAB generation mailboxes and requiring a full OAB download. With CU5 and later, customers can assign OABs to specific OAB generation mailboxes and not have to worry about accidentally triggering full OAB downloads due to accessing different OAB generation mailboxes. For more information, please see the article, OAB Improvements in Exchange 2013 Cumulative Update 5. Scenarios The following scenarios discuss a real life situation to further explain the new OAB management methods. Scenario 1: Create a new Organization Mailbox Contoso has Exchange Server 2013 Mailbox & CAS role servers deployed at Dallas and Seattle sites. John, the Exchange Admin for Contoso, analyzes the http proxy log files on CAS servers and finds the OAB download request for Seattle users is going to Dallas servers. On further investigation, John finds he has just one Organization Mailbox present in Dallas, hence OAB download requests of all the users are going to Dallas server. He decides to create a new Organization Mailbox at Seattle site with following commands: Step1: Create a new Arbitration Mailbox New-Mailbox -Arbitration -Name "OAB Seattle" -Database DB2Seattle -UserPrincipalName oabs@contoso.com –DisplayName “OAB Mailbox for Seattle” Step2: Enable the Arbitration Mailbox with OABGen capability Set-Mailbox -Arbitration oabs -OABGen $true -MaxSendSize 1GB Scenario 2: Customize OAB Generation Schedule Ben is an administrator of Exchange 2013 deployment at Tail Spin Toys. The default OAB generation schedule does not suit them and they want to generate OAB approximately every fourth hour of the day. Ben will use following command to change properties of the mailbox servers that will be hosting the Organization Mailbox. Set-MailboxServer Exch1 -OABGeneratorWorkCycle 01.00:00:00 -OABGeneratorWorkCycleCheckpoint 04:00:00 After a couple of days, John analyzes Event ID 17002 in application log and makes sure the OAB is generated every four hours. Hopefully, you find this post useful! Let us know your feedback below! Bhalchandra Atre Updates 5/15/14: Updated Placement of Organization Mailbox section to align with current guidance.Ask the Perf Guy: Sizing Exchange 2016 Deployments
Uh oh. You are probably thinking to yourself, here comes another one of those incredibly long blog posts about sizing. Thankfully, it’s not. If you want to fully understand the sizing process for Exchange 2016, you are certainly welcome to read the previous post that I did for Exchange 2013, as the overall process is effectively the same with one major caveat. Since we have eliminated the CAS role in Exchange 2016, you must follow the process for multi-role deployment sizing. Overall, the inputs to our sizing formulas stay the same from Exchange 2013 to Exchange 2016. This means that our IOPS requirements, memory requirements, and all of the other values provided in the Exchange 2013 sizing guidance should continue to be used for Exchange 2016. We are changing one set of inputs, however. Processor requirements We are slightly increasing the processor requirements for Exchange 2016 (compared to Exchange 2013) as this is a very new release, and we are still learning how it performs in production. This slight increase in CPU provides some additional headroom for unanticipated issues, and may be changed in the future as we learn more from our internal deployments as well as customer feedback. The same SPECint_rate2006 baseline value described in the Exchange 2013 guidance should continue to be used (33.75 per-core). Messages sent or received per mailbox per day Mcycles per User, Active DB Copy or Standalone Mcycles per User, Passive DB Copy 50 2.99 0.70 100 5.97 1.40 150 8.96 2.10 200 11.94 2.80 250 14.93 3.50 300 17.91 4.20 350 20.90 4.90 400 23.88 5.60 450 26.87 6.30 500 29.85 7.00 These changes are reflected in v7.8 and later in the calculator. System scalability The previously released guidance on maximum recommended cores and maximum memory size for Exchange 2013 is generally applicable to Exchange 2016 in terms of background and general scalability issues, however we have increased the recommended maximum memory size for currently supported versions of Exchange 2016 to 192GB. We recommend not exceeding the following sizing characteristics for Exchange 2016 servers. Recommended Maximum Processor Core Count 24 Recommended Maximum Memory 192 GB Note: Version 9.1 and later of the Exchange Server Role Requirements Calculator aligns with this guidance. Summary If you are at all familiar with the process for sizing the last couple of Exchange releases, you will be very comfortable with Exchange 2016. As with any new release, you should plan to roll-out in stages and monitor the health and performance of the solution carefully. We do expect that our performance and scalability guidance for Exchange 2016 will evolve over the lifespan of the product so watch the Exchange team blog for future updates. Jeff Mealiffe Principal PM Manager Office 365 Customer ExperienceDatabase Maintenance in Exchange 2010
Over the last several months there has been significant chatter around what is background database maintenance and why is it important for Exchange 2010 databases. Hopefully this article will answer these questions. What maintenance tasks need to be performed against the database? The following tasks need to be routinely performed against Exchange databases: Database Compaction Database Defragmentation Database Checksumming Page Patching Page Zeroing Database Compaction The primary purpose of database compaction is to free up unused space within the database file (however, it should be noted that this does not return that unused space to the file system). The intention is to free up pages in the database by compacting records onto the fewest number of pages possible, thus reducing the amount of I/O necessary. The ESE database engine does this by taking the database metadata, which is the information within the database that describes tables in the database, and for each table, visiting each page in the table, and attempting to move records onto logically ordered pages. Maintaining a lean database file footprint is important for several reasons, including the following: Reducing the time associated with backing up the database file Maintaining a predictable database file size, which is important for server/storage sizing purposes. Prior to Exchange 2010, database compaction operations were performed during the online maintenance window. This process produced random IO as it walked the database and re-ordered records across pages. This process was literally too good in previous versions – by freeing up database pages and re-ordering the records, the pages were always in a random order. Coupled with the store schema architecture, this meant that any request to pull a set of data (like downloading items within a folder) always resulted in random IO. In Exchange 2010, database compaction was redesigned such that contiguity is preferred over space compaction. In addition, database compaction was moved out of the online maintenance window and is now a background process that runs continuously. Database Defragmentation Database defragmentation is new to Exchange 2010 and is also referred to as OLD v2 and B+ tree defragmentation. Its function is to compact as well as defragment (make sequential) database tables that have been marked/hinted as sequential. Database defragmentation is important to maintain efficient utilization of disk resources over time (make the IO more sequential as opposed to random) as well as to maintain the compactness of tables marked as sequential. You can think of the database defragmentation process as a monitor that watches other database page operations to determine if there is work to do. It monitors all tables for free pages, and if a table gets to a threshold where a significant high percentage of the total B+ Tree page count is free, it gives the free pages back to the root. It also works to maintain contiguity within a table set with sequential space hints (a table created with a known sequential usage pattern). If database defragmentation sees a scan/pre-read on a sequential table and the records are not stored on sequential pages within the table, the process will defrag that section of the table, by moving all of the impacted pages to a new extent in the B+ tree. You can use the performance counters (mentioned in the monitoring section) to see how little work database defragmentation performs once a steady state is reached. Database defragmentation is a background process that analyzes the database continuously as operations are performed, and then triggers asynchronous work when necessary. Database defragmentation is throttled under two scenarios: The max number of outstanding tasks This keeps database defragmentation from doing too much work the first pass if massive change has occurred in the database. A latency throttle of 100ms When the system is overloaded, database defragmentation will start punting defragmentation work. Punted work will get executed the next time the database goes through that same operational pattern. There's nothing that remembers what defragmentation work was punted and goes back and executes it once the system has more resources. Database Checksumming Database checksumming (also known as Online Database Scanning) is the process where the database is read in large chunks and each page is checksummed (checked for physical page corruption). Checksumming’s primary purpose is to detect physical corruption and lost flushes that may not be getting detected by transactional operations (stale pages). With Exchange 2007 RTM and all previous versions, checksumming operations happened during the backup process. This posed a problem for replicated databases, as the only copy to be checksummed was the copy being backed up. For the scenario where the passive copy was being backed up, this meant that the active copy was not being checksummed. So in Exchange 2007 SP1, we introduced a new optional online maintenance task, Online Maintenance Checksum (for more information, see Exchange 2007 SP1 ESE Changes – Part 2). In Exchange 2010, database scanning checksums the database and performs post Exchange 2010 Store crash operations. Space can be leaked due to crashes, and online database scanning finds and recovers lost space. Database checksum reads approximately 5 MB per second for each actively scanning database (both active and passive copies) using 256KB IOs. The I/O is 100 percent sequential. The system in Exchange 2010 is designed with the expectation that every database is fully scanned once every seven days. If the scan takes longer than seven days, an event is recorded in the Application Log : Event ID: 733 Event Type: Information Event Source: ESE Description: Information Store (15964) MDB01: Online Maintenance Database Checksumming background task is NOT finishing on time for database 'd:\mdb\mdb01.edb'. This pass started on 11/10/2011 and has been running for 604800 seconds (over 7 days) so far. If it takes longer than seven days to complete the scan on the active database copy, the following entry will be recorded in the Application Log once the scan has completed: Event ID: 735 Event Type: Information Event Source: ESE Description: Information Store (15964) MDB01 Database Maintenance has completed a full pass on database 'd:\mdb\mdb01.edb'. This pass started on 11/10/2011 and ran for a total of 777600 seconds. This database maintenance task exceeded the 7 day maintenance completion threshold. One or more of the following actions should be taken: increase the IO performance/throughput of the volume hosting the database, reduce the database size, and/or reduce non-database maintenance IO. In addition, an in-flight warning will also be recorded in the Application Log when it takes longer than 7 days to complete. In Exchange 2010, there are now two modes to run database checksumming on active database copies: Run in the background 24×7 This is the default behavior. It should be used for all databases, especially for databases that are larger than 1TB. Exchange scans the database no more than once per day. This read I/O is 100 percent sequential (which makes it easy on the disk) and equates to a scanning rate of about 5 megabytes (MB)/sec on most systems. The scanning process is single threaded and is throttled by IO latency. The higher the latency, the more database checksum slows down because it is waiting longer for the last batch to complete before issuing another batch scan of pages (8 pages are read at a time). Run in the scheduled mailbox database maintenance process When you select this option, database checksumming is the last task. You can configure how long it runs by changing the mailbox database maintenance schedule. This option should only be used with databases smaller than 1 terabyte (TB) in size, which require less time to complete a full scan. Regardless of the database size, our recommendation is to leverage the default behavior and not configure database checksum operations against the active database as a scheduled process (i.e., don’t configure it as a process within the online maintenance window). For passive database copies, database checksums occur during runtime, continuously operating in the background. Page Patching Page patching is the process where corrupt pages are replaced by healthy copies. As mentioned previously, corrupt page detection is a function of database checksumming (in addition, corrupt pages are also detected at run time when the page is stored in the database cache). Page patching works against highly available (HA) database copies. How a corrupt page is repaired depends on whether the HA database copy is active or passive. Page patching process On active database copies On passive database copies A corrupt page(s) is detected. A marker is written into the active log file. This marker indicates the corrupt page number and that page requires replacement. An entry is added to the page patch request list. The active log file is closed. The Replication service ships the log file to passive database copies. The Replication service on a target Mailbox server receives the shipped log file and inspects it. The Information Store on the target server replays the log file and replays up to marker, retrieves its healthy version of the page, invokes Replay Service callback and ships the page to the source Mailbox server. The source Mailbox server receives the healthy version of the page, confirms that there is an entry in the page patch request list, then writes the page to the log buffer, and correspondingly, the page is inserted into the database cache. The corresponding entry in the page patch request list is removed. At this point the database is considered patched (at some later point the checkpoint will advance and the database cache will be flushed and the corrupt page on disk will be overwritten). Any other copy of this page (received from another passive copy) will be silently dropped, because there is no corresponding entry in the page patch request list. On the Mailbox server where the corrupt page(s) is detected, log replay is paused for the affected database copy. The replication service coordinates with the Mailbox server that is hosting the active database copy and retrieves the corrupted page(s) and the required log range from the active copy’s database header. The Mailbox server updates the database header for the affected database copy, inserting the new required log range. The Mailbox server notifies the Mailbox server hosting the active database copy which log files it requires. The Mailbox server receives the required log files and inspects them. The Mailbox server injects the healthy versions of the database pages it retrieved from the active database copy. The pages are written to the log buffer, and correspondingly, the page is inserted into the database cache. The Mailbox server resumes log replay. Page Zeroing Database Page Zeroing is the process where deleted pages in the database are written over with a pattern (zeroed) as a security measure, which makes discovering the data much more difficult. With Exchange 2007 RTM and all previous versions, page zeroing operations happened during the streaming backup process. In addition since they occurred during the streaming backup process they were not a logged operation (e.g., page zeroing did not result in the generation of log files). This posed a problem for replicated databases, as the passive copies never had its pages zeroed, and the active copies would only have it pages zeroed if you performed a streaming backup. So in Exchange 2007 SP1, we introduced a new optional online maintenance task, Zero Database Pages during Checksum (for more information, see Exchange 2007 SP1 ESE Changes – Part 2). When enabled this task would zero out pages during the Online Maintenance Window, logging the changes, which would be replicated to the passive copies. With the Exchange 2007 SP1 implementation, there is significant lag between when a page is deleted to when it is zeroed as a result of the zeroing process occurring during a scheduled maintenance window. So in Exchange 2010 SP1, the page zeroing task is now a runtime event that operates continuously, zeroing out pages typically at transaction time when a hard delete occurs. In addition, database pages can also be scrubbed during the online checksum process. The pages targeted in this case are: Deleted records which couldn’t be scrubbed during runtime due to dropped tasks (if the system is too overloaded) or because Store crashed before the tasks got to scrub the data; Deleted tables and secondary indices. When these get deleted, we don’t actively scrub their contents, so online checksum detects that these pages don’t belong to any valid object anymore and scrubs them. For more information on page zeroing in Exchange 2010, see Understanding Exchange 2010 Page Zeroing. Why aren’t these tasks simply performed during a scheduled maintenance window? Requiring a scheduled maintenance window for page zeroing, database defragmentation, database compaction, and online checksum operations poses significant problems, including the following: Having scheduled maintenance operations makes it very difficult to manage 24x7 datacenters which host mailboxes from various time zones and have little or no time for a scheduled maintenance window. Database compaction in prior versions of Exchange had no throttling mechanisms and since the IO is predominantly random, it can lead to poor user experience. Exchange 2010 Mailbox databases deployed on lower tier storage (e.g., 7.2K SATA/SAS) have a reduced effective IO bandwidth available to ESE to perform maintenance window tasks. This is an issue because it means that IO latencies will increase during the maintenance window, thus preventing the maintenance activities to complete within a desired period of time. The use of JBOD provides an additional challenge to the database in terms of data verification. With RAID storage, it's common for an array controller to background scan a given disk group, locating and re-assigning bad blocks. A bad block (aka sector) is a block on a disk that cannot be used due to permanent damage (e.g. physical damage inflicted on the disk particles). It's also common for an array controller to read the alternate mirrored disk if a bad block was detected on the initial read request. The array controller will subsequently mark the bad block as “bad” and write the data to a new block. All of this occurs without the application knowing, perhaps with just a slight increase in the disk read latency. Without RAID or an array controller, both of these bad block detection and remediation methods are no longer available. Without RAID, it's up to the application (ESE) to detect bad blocks and remediate (i.e., database checksumming). Larger databases on larger disks require longer maintenance periods to maintain database sequentiality/compactness. Due to the aforementioned issues, it was critical in Exchange 2010 that the database maintenance tasks be moved out of a scheduled process and be performed during runtime continuously in the background. Won’t these background tasks impact my end users? We’ve designed these background tasks such that they're automatically throttled based on activity occurring against the database. In addition, our sizing guidance around message profiles takes these maintenance tasks into account. How can I monitor the effectiveness of these background maintenance tasks? In previous versions of Exchange, events in the Application Log would be used to monitor things like online defragmentation. In Exchange 2010, there are no longer any events recorded for the defragmentation and compaction maintenance tasks. However, you can use performance counters to track the background maintenance tasks under the MSExchange Database ==> Instances object: Counter Description Database Maintenance Duration The number of seconds that have passed since the maintenance started for this database. If the value is 0, maintenance has been finished for the day. Database Maintenance Pages Bad Checksums The number of non-correctable page checksums encountered during a database maintenance pass Defragmentation Tasks The count of background database defragmentation tasks that are currently executing Defragmentation Tasks Completed/Sec The rate of background database defragmentation tasks that are being completed You'll find the following page zeroing counters under the MSExchange Database object: Counter Description Database Maintenance Pages Zeroed Indicates the number of pages zeroed by the database engine since the performance counter was invoked Database Maintenance Pages Zeroed/sec Indicates the rate at which pages are zeroed by the database engine How can I check whitepace in a database? You will need to dismount the database and use ESEUTIL /MS to check the available whitespace in a database. For an example, see http://technet.microsoft.com/en-us/library/aa996139(v=EXCHG.65).aspx (note that you have to multiply the number of pages by 32K). Note that there is a status property available on databases within Exchange 2010, but it should not be used to determine the amount of total whitespace available within the database: Get-MailboxDatabase MDB1 -Status | FL AvailableNewMailboxSpace AvailableNewMailboxSpace tells you is how much space is available in the root tree of the database. It does not factor in the free pages within mailbox tables, index tables, etc. It is not representative of the white space within the database. How can I reclaim the whitespace? Naturally, after seeing the available whitespace in the database, the question that always ensues is – how can I reclaim the whitespace? Many assume the answer is to perform an offline defragmentation of the database using ESEUTIL. However, that's not our recommendation. When you perform an offline defragmentation you create an entirely brand new database and the operations performed to create this new database are not logged in transaction logs. The new database also has a new database signature, which means that you invalidate the database copies associated with this database. In the event that you do encounter a database that has significant whitespace and you don't expect that normal operations will reclaim it, our recommendation is: Create a new database and associated database copies. Move all mailboxes to the new database. Delete the original database and its associated database copies. A terminology confusion Much of the confusion lies in the term background database maintenance. Collectively, all of the aforementioned tasks make up background database maintenance. However, the Shell, EMC , and JetStress all refer to database checksumming as background database maintenance, and that's what you're configuring when you enable or disable it using these tools. Figure 1: Enabling background database maintenance for a database using EMC Enabling background database maintenance using the Shell: Set-MailboxDatabase -Identity MDB1 -BackgroundDatabaseMaintenance $true Figure 2: Running background database maintenance as part of a JetStress test My storage vendor has recommended I disable Database Checksumming as a background maintenance task, what should I do? Database checksumming can become an IO tax burden if the storage is not designed correctly (even though it's sequential) as it performs 256K read IOs and generates roughly 5MB/s per database. As part of our storage guidance, we recommend you configure your storage array stripe size (the size of stripes written to each disk in an array; also referred to as block size) to be 256KB or larger. It's also important to test your storage with JetStress and ensure that the database checksum operation is included in the test pass. In the end, if a JetStress execution fails due to database checksumming, you have a few options: Don’t use striping Use RAID-1 pairs or JBOD (which may require architectural changes) and get the most benefit from sequential IO patterns available in Exchange 2010. Schedule it Configure database checksumming to not be a background process, but a scheduled process. When we implemented database checksum as a background process, we understood that some storage arrays would be so optimized for random IO (or had bandwidth limitations) that they wouldn't handle the sequential read IO well. That's why we built it so it could be turned off (which moves the checksum operation to the maintenance window). If you do this, we do recommend smaller database sizes. Also keep in mind that the passive copies will still perform database checksum as a background process, so you still need to account for this throughput in our storage architecture. For more information on this subject see Jetstress 2010 and Background Database Maintenance. Use different storage or improve the capabilities of the storage Choose storage which is capable of meeting Exchange best practices (256KB+ stripe size). Conclusion The architectural changes to the database engine in Exchange Server 2010 dramatically improve its performance and robustness, but change the behavior of database maintenance tasks from previous versions. Hopefully this article helps your understanding of what is background database maintenance in Exchange 2010. Ross Smith IV Principal Program Manager Exchange Customer ExperienceExchange 2010 Cross-Forest Mailbox Moves
EDIT 8/12/2010: Added a note about the necessity to manually enable MSProxy in remote forest. We are seeing some trends where quite a few customers are migrating mailboxes to a new Exchange organization, in a different Active Directory (AD) forest. This blog post is aimed at helping to explain the fundamentals of what is required to move mailboxes across forests so that you can be prepared with the correct data, make better plans, and successfully perform a migration without encountering painful problems. The blog post doesn't cover how to setup and configure shared address space or Free/busy. After reading this blog post, you should have better understanding of: How to plan your migration by understanding your current forest configuration and your desired configuration. Different ways for you to synchronize user data between different AD forests. Networking and Administrator permissions required to perform a successful cross-forest mailbox move. The trends we are seeing currently show that companies are having more trouble understanding the different scenarios than performing the migration. There are several scenarios here, and Microsoft has tools, documentation, and scripts to assist in each one of them. There are many reasons companies choose to have multiple forests or maybe find themselves with multiple forests, requiring cross-forest moves of users and mailboxes. For instance: Companies that merge, are bought out, or have absorbed another company in some manner. Companies who want to start fresh and leave a lot of legacy issues behind. Companies that have subsidiaries; segment their environment by Department, Geography, or for Security considerations. The common Active Directory topologies that are supported in Exchange 2010 are as follows: Single forest, single Active Directory site Single forest, multiple Active Directory sites Multiple forest, multiple Active Directory sites Exchange deployment topologies vary due to organizational size and business complexity. Variations may include Single Forest, Resource Forest, Hybrid Forest, and Cross Forest topology. For purposes of discussion the following forest definitions will be used going forward: Forest Name Active Directory user object status Mailbox Status Exchange Forest Enabled User Object Mailbox Enabled Account Forest Enabled User Object No mailbox enabled objects Resource Forest Disabled User Object (linked to a separate enabled user object in an Account Forest) Mailbox Enabled Hybrid Forest Both 1.) AD Enabled Mailbox Enabled 2.) AD Disabled Mailbox Enabled Both mailbox enabled and disabled objects Most of the Cross-Org Move Mailbox scenarios are closely related to the Active Directory Forests involved in the migration. There are 3 major scenarios to be considered: 1. Move from Exchange Forest A to Exchange Forest B. This means that the user is a security principal in forest A and after he is moved to forest B, he is a security principal in forest B as well. This may be a hybrid-forest scenario, typical during inter-forest migrations, because the user is security principal in both. Hybrid is when there are both enabled and disabled users in the same forest. 2. Move from Account Forest to Exchange Resource Forest. Company is splitting Exchange off to its own forest. Maybe due to outsourcing it, complex business organization, or desire to de-couple the Exchange org (e.g. messaging services) from the other infrastructure. 3. Move from Exchange Resource Forest to Account Forest. This is the reverse of #2. Company is bringing Exchange back into the same forest for simplicity, to better integrate with OCS (though they are not required to be in the same forest), or collapsing/consolidating previously separate Exchange orgs into one user forest. Cross-forest is when all users from the same organization are only contacts or mail enabled user objects in the other forest. This is not referenced as a common scenario because it's usually in place between two separate legal entities and there would not be much movement (e.g. migrations) between them. Active Directory Forest Configuration examples: Below are some AD forest configuration examples. The forest scenarios don't necessarily imply there is a "move" or migration going on, some are long-term configurations. Resource Forest A Resource Forest scenario is a deployment that has at least one Exchange Resource Forest that hosts user mailboxes (but not active user accounts or enabled user accounts) and at least one other forest that hosts the AD user accounts. In other words, Exchange is installed into an AD forest which is separate from the "user account" AD forest. A one-way forest trust where the resource forest trusts the account forest is created. Each mailbox in the Exchange forest must have a corresponding user in the account forest, which is granted access to logon to the mailbox. This is referred to as a "Linked Mailbox". The user objects in the Exchange forest are never logged onto by an end user and are disabled. Hybrid Forest Typically this scenario is maintained initially for co-existence while migrating and decommissioning a forest. It is different from a typical cross-forest scenario because there may be both enabled and disabled users in both forests for the same organization. In some cases, an organization may actually need to maintain the Hybrid Forest scenario over the long-term. While this is a supported scenario, it comes with additional complexity that must be addressed: Mastering User and Exchange attributes occurs on both sides. A tool such as Forefront Identity Manager (FIM), is needed to maintain consistent data on both sides, including the GAL. Free/Busy and Public Folder access requires additional configuration, tools, and in some cases maintaining an Exchange 2007 server. (Please note that the IOREPL tool isn't currently supported with Exchange 2010 as a target server and in fact follows the Exchange 2003 Product support life cycle.) Free/Busy, over the long-term will be best managed using the new Federation services (Microsoft Federation Gateway) For more information refer to Understanding Federation Cross-forest Both forests contain mailboxes and user accounts and contacts. This type of configuration has user accounts always enabled and mailbox enabled, with a corresponding contact in the other forest. The following diagram depicts how different objects are represented in the corresponding forest: For more information on forests related to Cross Org migrations, refer to http://msexchangeteam.com/archive/2006/11/02/430289.aspx Three Migration paths you need to choose from: Depending on the current topology you have employed, you may find yourself planning to move users into the new forest and then following with moving their mailboxes as well. There are essentially three ways of planning to move your resources: A customized deployment in which you write ILM rules extension code to create the target Mail Enabled User (MEU). You should already have a custom ILM deployment for cross forest GALSync. Microsoft Identity Lifecycle Manager Service Pack 1 Feature Pack 1 (ILM 2007 SP1 FP1) GALSync Management Agent (MA) doesn't include several attributes now required in Exchange 2010, most importantly, msExchMailboxGUID. The out of the box GALSync MA cannot be used since it creates contact object instead of user object required for Online Mailbox Move. The ILM sample code demonstrates how to sync source mailbox as Mail Enabled Users (MEU). Note: Customers using "out of the box" GALSync MA may probably not know how to customize ILM. Use Prepare-MoveRequest.ps1 script to create the target MEU. It is important to note that the PrepareMoveRequest script works in conjunction with "out of the box" Exchange GALSync MA for ILM (or FIM). This means the script has built-in logic to convert target Mail Enabled Contact (MEC) created by ILM GALSync MA into the required MEU. Use Prepare-MoveRequest.ps1 script and then use ADMT to migrate the other attributes on the user object. Important Note: Our recommendation on working with ADMT is to rely on the PrepareMoveRequest script to create the local user object for mailbox move, and then use ADMT to migrate SIDHistory and password and merge this into the MEU created by PrepareMoveRequest.ps1 script. The point of doing ILM or the script first is to ensure the MEUs are all created with the correct msExch* attributes. This also ensures the following benefits: A correct GAL immediately for co-existence (short or long-term) Permissions for delegates and mailbox access are preserved during the move using the msExchMailboxGUID attribute. Since this is populated on the target object with PrepareMoveRequest.ps1 the permissions will be maintained in the cross-forest move. At this point it doesn't matter if ADMT is used to migrate/merge the user objects all at once or in "batches" of user objects. ADMT can be controlled better to ensure only merging of SIDhistory and certain other mandatory attributes if it's not already populated. Running ADMT first, without ensuring exclusions on msExch* attributes, can cause corrupted objects which the script cannot correctly convert with the -UseLocalObject switch. Important Note: When SP1 ships, we will support running ADMT first and then the PrepareMoveRequest script later. ILM and PrepareMoveRequest Scenarios broken-down: There are basically 5 steps involved with moving a mailbox across a forest in Exchange 2010. They are: Preparing Active Directory, Network Prerequisites, Administrator Permissions, Moving Mailboxes and Clean-up. Each of these steps is series of smaller steps that need to be taken in order to move a mailbox from one Exchange forest to and Exchange 2010 forest. The first step in Cross Forest mailbox moves is preparing Active Directory. In the target forest a mail enabled user account must be created with certain attributes. The method used for creating the target account and setting the mandatory attributes is up to the organization administrator. ADMT and ILM can be used to synchronize/pull over the attributes from the source forest. Exchange Provisioning using ILM 2007 If you deployed ILM for cross-forest global address list (GAL) synchronization, the recommended approach to creating the mail-enabled user is to use ILM 2007 Service Pack 1 (SP1) Feature Pack 1 (FP1) or Forefront Identity Manager 2010 (FIM) GALSync MA. We've created sample code that you can use to learn how to customize ILM to synchronize the source mailbox user and target mail user. For more information, including how to download the sample code, refer to this link. To deploy Exchange 2010 in a cross-forest topology, you must first install Exchange 2010 in the new forest. Then, provision the mail-enabled users representing the source mailboxes so that Exchange 2010 can move the mailbox and migrated users can see all addresses. Deployment steps: Deploy Exchange 2010 in a cross-forest topology with ILM 2007 FP1 SP1. Import and install the ILM sample code from Prepare Mailboxes for Cross-Forest Moves Using Sample Code. Note: The main purpose of the sample code is to encourage customers to customize, or add more functions to the sample code. The sample code is very basic and it only copies very basic attributes. Customers who rely on this sample code may find many attributes missing. Configure the Mail-Enabled User provisioning Management Agents for each forest. This allows the mailboxes in the source forest to be created as MEU in the target forest and ensure a common GAL. Create an SMTP Send connector in each forest and configure SMTP namespace sharing (http://technet.microsoft.com/en-us/library/bb676395.aspx). In each forest, enable the Availability service so that users in each forest can view free/busy data about users in the other forest. For more information, see Managing the Availability Service. Note: The Availability service is supported only for Outlook 2007 clients and newer. If Outlook 2003 clients still exist in one of the forests, the only solution will be to deploy Exchange 2007 first in the Exchange 2010 organization (because adding it late is not possible if Exchange 2010 is deployed first) and implement the IOREPL tool to replicate Free/Busy system public folders to the Exchange 2007 server. The Free/Busy system public folder replicas can then be replicated using PF replication to your Exchange 2010 server. IOREPL will not replicate a public/system folder directly to an Exchange 2010 server. For more information review: Exchange Provisioning using ILM 2007 and FIM 2010 http://technet.microsoft.com/en-us/magazine/ff472471.aspx Prepare-MoveRequest.ps1 It may be difficult for some customers to synchronize the prerequisite attributes for performing mailbox moves without using ILM. You may have some other solution in place that does not synchronize the required attributes, and does not allow customization. Small companies may not have a solution at all and simply wish to transition users from an existing forest (that is set to be obsolete) to a new, clean Exchange 2010 forest. To solve this problem, the PrepareMoveRequest script has been written to prepare the AD target object and synchronize the required attributes for cross-forest moves to work. The script creates the target MEU if necessary, or synchronizes an existing MEU when possible. The PrepareMoveRequest script prepares Exchange 2003, Exchange 2007, and Exchange 2010 mailbox users for migration to an Exchange 2010 forest. For more information about using the sample script, refer to the following link. The PrepareMoveRequest script supports 2 scenarios: Creating a brand new user in the local forest where the MBX will be moved to. A local recipient, either a MEU or MEC already exists, created by an external agent such as ILM - If the local forest object is a mail contact, the script will convert the mail contact to a mail user while persisting the contact's existing exchange-related attributes. If the local forest object is a MEU, the script will reuse this mail user and stamp the essential attributes on the local mail user object. The administrator must specify the -UserLocalObject switch in order to tell the script to use this scenario. Note: The scenario that the script doesn't support is that some external process created a local user object and relies on the script to copy all the attributes and links from the remote MBX to the local user. This is the ADMT scenario described after this scenario. In order to run New-MoveRequest cmdlet to move a mailbox from an Exchange 2003/2007/2010 source forest to an Exchange2010 target forest, the target forest must contain a valid MEU account with the set of AD attributes described in this section. These attributes are synchronized by the PrepareMoveRequest script. There are certain mandatory attributes that should be present on the target mail user for New-MoveRequest to run properly. These attributes are always set by the PrepareMoveRequest script, either as they are taken from the source MBX, or as determined by the script. The attributes are listed here http://technet.microsoft.com/en-us/library/ee861103.aspx. Process Overview: Run PrepareMoveRequest script first and then ADMT Prepare MEU To create the target mail enabled user account in an Exchange 2010 forest from the source mailbox enabled account in the source Exchange forest, the PrepareMoveRequest script must be executed in the target Exchange 2010 forest. The script pulls the mailbox enabled account attributes from the source forest. The script can be used to provision one target MEU account at a time, but can also take data that is passed by pipeline as input to provision MEUs in bulk. Since PrepareMoveRequest script relies on Update-Recipient task that exists only in Exchange Management Shell, all the below commands need to be run in Exchange Management Shell. Running in PowerShell will only result in error. Run the below commands in the target forest $Local = Get-Credential Input the target forest's Administrator Credentials in "Domain\User" and Password format. Note: The account used should have permissions to call Update-Recipient which is available only to Exchange Enterprise Admin. $Remote = Get-Credential Input the Source forest's Administrator Credentials in "Domain\User" and Password format. Note: Since the PrepareMoveRequest script will also update the source object's proxyAddresses to include the target object's legacyDN as X500 address, the account used to run this command should have Read and Write access for the source forest. Run the PrepareMoveRequest script in the target forest [PS] C:\>.\Prepare-MoveRequest.Ps1 -Identity "DN of a user from SourceForest" -RemoteForestDomainController "FQDN of Source DC" -RemoteForestCredential $Remote -LocalForestDomainController "FQDN of Target Forest DC" -LocalForestCredential $Local -TargetMailUserOU "Distinguished name of OU in TargetForest" -UseLocalObject Note 1: You can use the -Verbose flag to check which attributes have been set if you want to get a detailed list of the attributes that were touched. Note 2: You can use the -UseLocalObject parameter here. If the local matching object is found, then the local object will be used. Note: If the local matching object is found and UseLocalObject is not defined, the script will throw an error. If the local object doesn't exist, even if UseLocalObject is specified, the script will still create a new one. If you are sure that you didn't prepare local object before, you could remove this parameter to ensure accidental overriding. In the target forest, we get a new disabled mail-enabled user AD object created with some of the following Exchange attributes: legacyExchangeDN, mail, mailnickname, msExchmailboxGuid, proxyAddresses, X500, targetAddress, userAccountControl, userprincipalName SIDHistory is empty. This is expected because Exchange doesn't migrate SIDs. At this point all of the required attributes to perform a mailbox move have been synced into the target forest. Run ADMT in the target forest. Note: Currently the Active Directory Migration Tool (ADMT) v3.1 is not supported on Windows 2008 R2 Servers. If you plan to use ADMT v3.1, it must be installed on Windows 2008 server. Check the results in the target forest: The user should now have SIDHistory matching the objectSid of the source object (all other attributes are left untouched) Gotchas running ADMT first and then PrepareMoveRequest script: Currently, several customers are running ADMT first and then running the PrepareMoveRequest script. When a user is created via ADMT, the PrepareMoveRequest script doesn't work since there are no proxyAddresses for the script to match the source forest user with the target forest user. The recommended approach is to copy at least 1 proxy address using ADMT. However, if you use the -UseLocalObject parameter, the script will only copy the 3 mandatory parameters (msExchMailboxGUID, msExchArchiveGUID, msExchArchiveName). This is not very useful. Customers can simply copy these 3 themselves. Important Note: In SP1, we are adding the OverwriteLocalObject parameter. This is designed for the ADMT case. ADMT can copy the SIDhistory, password, and proxyAddresses, and the PrepareMoveRequest script can sync the other email attributes. In this case, it will copy attributes from source to target, so it's the opposite of UseLocalObject. ADMT and Exchange Attributes ADMT transfers Exchange attributes (e.g. homeMDB, homeMTA, showInAddressBook, msExch*) which make the target user look like a legacy mailbox in the target domain. This leaves the target account in an invalid state (e.g. homeMDB still points to the old forest) which is unexpected for the PrepareMoveRequest.ps1 script. To prevent this, Exchange attributes are excluded from ADMT. The PrepareMoveRequest.ps1 script can identify and match existing accounts in the target forest based on their SMTP address (proxyAddresses attribute). Note: It can also do this based on the MasterAccountSid, but this is only populated for accounts in a resource forest scenario. More precisely, the script will use the existing target accounts if the following are true: The target account has a value in proxyAddresses which matches one of the proxyAddresses of the source account. The target account is a mail enabled user i.e. you can retrieve it with the Get-Recipient command. For this to succeed, it needs to have mail attributes like 'mail', 'targetAddress' etc. You need to specify the -UseLocalObject parameter in the script If all these are true, the script will copy further attributes needed (especially msExchMailboxGUID) to the target account so that the move request can process the accounts. By default, ADMT 3.1 does NOT migrate "mail", "msExchMailboxGuid" and "proxyAddresses" attributes because of security reasons. This is documented in the below article under "System attribute exclusion list" Managing Users, Groups, and User Profiles http://technet.microsoft.com/en-us/library/cc974331(WS.10).aspx Important Note: When running ADMT second after ILM due to both forests having the same schema (attributes), unexpected Exchange attributes are brought over. This can cause issues. HomeMDB for example is brought over and causes the MEU to look like a legacy mailbox, and is unusable. To resolve the problem of ADMT being run first, and leaving the user in an invalid state for the PrepareMoveRequest.ps1 script, you can create the following VB script/ADMT COM object model to exclude all Exchange attributes from being migrated by ADMT. Set O = CreateObject("ADMT.Migration"). o.SystemPropertiesToExclude = " HomeMDB,HomeMTA,showInAddressBook,msExchHomeServerName, mail, proxyAddresses, msExch*" This allows update-recipient to find the target object and match it with the source account and merge the two together. For more information, refer to the below article: You will find that several custom attributes are missing when you use ADMT to migrate users between two forests http://support.microsoft.com/kb/937537 Network Prerequisites When mailboxes are moved from one Exchange 2010 forest to another Exchange 2010 forest, the process is handled through Exchange 2010 Client Access Servers using the MRSProxy service. The only port required to be open between the forests for MRSProxy to use HTTPS traffic is port 443. This works even if the source mailboxes are on 2003 or 2007 MBX servers as long as an Exchange 2010 CAS server exists in both organizations. Note: The whole forest doesn't need to be Exchange 2010 in order to use the MRSProxy. If there is at least one Exchange 2010 CAS in the forest (with access to the Mailbox Servers and AD), it can be used as the MRS Proxy for moves from a mostly Exchange 2003 or Exchange 2007 forest. This can be called the "Remote" scenario (or the "MRSProxy" scenario). By default, MRSProxy is disabled. To start MRSProxy on the Client Access server in the remote forest, you must modify the Client Access server's Web.config file. For more information refer to http://technet.microsoft.com/en-us/library/ee732395.aspx. If CAS servers are behind the NLB, you should do this on all servers that can take the load. If the mailbox is being moved from legacy Exchange forest then the mailbox replication service will need to have the same TCP ports open that is needed for a normal local mailbox move. Listed are the TCP ports that are needed for a local mailbox move. These ports will be needed to be open both ways for mailboxes to be moved. Note: This is more of the "Remote Legacy" scenario, but it can be used between two Exchange 2010 forests as well as between one Exchange 2010 forest and one Exchange 2003/2007 forest. TABLE: Port Protocol 808 (TCP) Mailbox Replication Service uses to communicate 53 (TCP) DNS 135 (TCP) RPC End Point 389 (TCP) LDAP 3268 LDAP 1024 > (TCP) if mailbox store is not statically configured then 1024 higher ports need to be open 88 (TCP) Kerberos 445 (TCP) Microsoft-DS Service 443 (TCP) Mailbox Replication Proxy service uses port 443 to communicate with other Exchange 2010 client access server via HTTPS. Also it is necessary for servers in both forests to successfully perform name resolution using DNS. For cross forest mailbox moves via the MRSProxy service, the source and target servers use certificates to encrypt the HTTPS traffic. The CAS Servers in the source and target forests must have installed a valid certificate that has been issued by a trusted certificate authority recognized by the server in the opposite forest. Administrator Permissions In order to move mailboxes across different Exchange forests the account used to initiate the move request in the target forest and the account used to access the mailbox and directory in the source forest must have the proper permissions. The permissions that are needed for the account in the source forest depend on the type of move. Remote The account must have the privileges made available by membership in the Recipient Administrators group. Remote Legacy The migration account must have the following permissions. Exchange Server Administrators role Exchange Recipient Administrators role Destination Forest Permissions In the target Exchange 2010 organization the account used to create and manage the move request must be a member of the Organization Management or Recipient Management role groups, or have the following RBAC roles assigned either directly or through group membership: Move Mailboxes role Mail Recipients role Mail Recipient Creation role Only the Move Mailbox role is required to have access to the New-MoveRequest command. However, the Mail Recipients and Mail Recipient Creation roles may also be required to creating and managing target accounts in preparation for mailbox moves. Moving the mailbox There are two methods to move a mailbox across forests using Exchange 2010. The method used depends on the type of cross forest move. Both Remote and Remote Legacy cross forest moves can be performed from the Exchange Management Shell, but only Remote moves can be performed from the Exchange Management Console. Exchange Management Console To create a new move request for a cross forest move using Exchange Management Console (EMC), the console must have a session open to both the target and source forests at the same time using the feature Add Exchange Forest. This makes it possible to maintain a connection to an Exchange 2010 server in the source forest, and an Exchange 2010 server in the target forest. With a connection to servers in both source and target organizations via the EMC, you will be able to identify a mailbox that is to be moved from the source forest, while initiating the move request on an Exchange 2010 server in the target forest. To initiate a cross forest move with the Exchange Management Console, navigate to the Mailboxes folder in the Recipient Configuration node of the source forest, select the mailbox(es) to be moved, and then select New Remote Move Request. This starts the New Remote Move Request. Exchange Management Shell To initiate a cross forest mailbox move in the Exchange Management Shell a New-MoveRequest command must be issued with Remote* parameters. Move requests issued without Remote* parameters are local moves within the same Exchange forest. The New-MoveRequest cmdlet requires certain attributes to be synchronized between the source MBX account and the target MEU account in order for the mailbox move to succeed. This is described in the previous steps. In the target domain, perform the move request by running the below cmdlet New-MoveRequest -Identity "Distinguished name of User in Target Forest" -RemoteLegacy -TargetDatabase "E2K10 Mailbox Database Name" -RemoteGlobalCatalog "FQDN of Source DC" -RemoteCredential $Remote -TargetDeliveryDomain "Target domain name" After the move completes, the proxyAddresses and targetAddress attributes should have changed in the target forest. If the accounts are disabled in the target forest, enable it, set a password and log into OWA and test. After Online Mailbox Move (OMM), the source object is changed from MBX to MEU and target object is changed from MEU to MBX For more information on performing cross forest moves in Exchange 2010, refer to Managing Move Requests Clean-up When the MRS completes the moving of mailbox data from the source forest to the destination forest it mailbox enables the target user account. If the user account is disabled it leaves the account disabled. The MRS mailbox disables the source account, and converts it into a MEU account with a target address that refers to the primary SMTP address of the target mailbox account. The New-MoveRequest takes the TargetDeliveryDomain parameter. This is what determines which targetAddress to stamp. MRS checks the list of proxyAddresses for one (not necessarily the primary SMTP) that matches the FQDN specified in the TargetDeliveryDomain. The MRS will stamp this address as the targetAddress on the MEU. We moved away from using the primary SMTP address because there is a need to maintain the primary STMP when moving mailboxes cross-forest since this is part of a user's identity. When the primary SMTP address is the same on both forests, mail flow becomes more difficult. If the source account is to be retired and removed from the source forest, the administrator must plan for this manual operation outside of the mailbox move operation. What's coming in Exchange 2010 SP1 As mentioned earlier, SP1, will include the PrepareMoveRequest script as part of the install. Additionally, we are fixing a couple of issues with that script: Requiring separate local and remote credentials to run the script. LegacyExchangeDN not set on the new user object after converting local contact to local user. When specifying TargetMailUserOU, we will only search OUs (instead of other object class). Common Issues The most common issues related to PrepareMoveRequest script are listed below. These are not relevant if you have deployed the customized ILM, or if you have already run PrepareMoveRequest. Not able to match source forest user with target forest user. This is mainly due to the fact the script relies on proxyAddresses to match objects, so the target forest user needs to have at least 1 proxy address that matches the source Inadequate AD permission to delete/add recipient objects. The script manipulates AD directly and invokes the Update-Recipient cmdlet at the end, so you need to have the appropriate permission to change AD and call Update-Recipient. Another thing you can check is whether the TargetMailUserOU is set correctly. The current script does not have good support for users created by ADMT. The updated PrepareMoveRequest script in SP1 will support a new parameter "OverwriteLocalObject" for users created by ADMT and it will copy attributes from the source forest user to the target user. "UseLocalObject" - This is the script logic where we assume ILM has already created the target forest MEC or MEU, and you want to keep the target forest attributes. So the script will convert the target forest MEU or MEC to the required MEU for MBX move. Finally, a few words of thanks: I had the privilege of working with several SMEs from the Product Group, Consulting, UE and Support who helped me visualize, plan, develop and complete this blog. I would like to call out Ian Liu (Program Manager) who was instrumental in sharing his vision and being accessible at all times while writing this blog. I also want to thank Daniel Talbot for his expertise on this subject and his many contributions to the blog. Other contributors that I'd like to express gratitude are Andrew Ehrensing and Huangjian Guo. Thanks to Ying Zhang, Ramon Infante, Jeff Kizner, Kweku Ako-Adjei , Ben Winzenz, Kristi Simmons, Bill Haenlin, Laura La Fleur, Nino Bilic, Jonathan Runyon and Ayla Kol for their review and feedback. Last but not the least, I'd like to thank William Rall for his innumerable thorough reviews and feedback that helped shape this blog. - Nagesh MahadevSingle Item Recovery in Exchange Server 2010
Introduction Users often delete data from their mailboxes that they later want recovered. Recovery of these deleted items is the most common reason for IT admins to recover data from traditional point-in-time backups today. With previous versions of Exchange Server, administrators implemented two solutions to enable single item recovery, dumpster and restores. Both had their issues, unfortunately. Exchange 2010 aims to reduce the complexity and administrative costs associated with single item recovery. Terminology The following definitions may be useful for understanding the content within this article: Store Soft Delete - this is when an item has been deleted from the Deleted Items folder and placed within dumpster. Store Hard Delete - This is when an item has been marked for purging out of the store. Hard Delete via Outlook (Shift-Delete) - This is when the end user performs a shift-delete on an item. This results in the item bypassing the deleted items folder and being directly placed in dumpster. Single Item Recovery in Previous Versions of Exchange The end user single item recovery functionality was enabled through the store via the store dumpster. Administrators configured the dumpster setting on a per database or per mailbox basis. The default deleted item retention window in Exchange 2003 is 7 days, while with Exchange 2007 the default is 14 days. The end user recovery process worked typically like this (see figure 1): User receives message. User deletes the message. The message is moved to the Deleted Items folder. The user empties the deleted items folder (or an automated process like Managed Folders or Records Management deletes items out of the deleted items folder on a scheduled basis). The user then realizes he requires the previously deleted item. Using Outlook the user accesses Recover Deleted Items. Assuming the deletion timestamp of the deleted item is still within the deleted item retention period, the user has two choices: The user can recover the item. The recovered item is placed back in the Deleted Items folder. The user can purge the item from recover deleted items. This results in the deletion of the message permanently from the user's mailbox. Figure 1. Dumpster in Previous Versions If the item's deletion timestamp is beyond the deleted item retention period, then the item is not available for end user recovery. Instead, the user must call Help Desk and request recovery of the item. This involves: The user knowing when he deleted the item. The Exchange administrator locates a backup that contains the item in question. The Exchange administrator successfully restores the database to a recovery storage group. The Exchange Administrator extracts the deleted item and provides it back to the user. This of course assumes that there is a policy that allows for single item restoration and that this process isn't a really low priority for the Exchange administrator. If the backup is invalid, or if there is no backup for the time period in question, then the deleted data is unrecoverable. The other issue that needs highlighting is the fact that in previous versions of Exchange there is no way to prevent the end user from purging the data from the Recover Deleted Items view. This poses a significant legal risk for organizations that must ensure compliance requirements are met. Dumpster Changes In previous releases of Exchange Server, dumpster is essentially a view stored per folder. Items in the dumpster (henceforth known as Dumpster 1.0) stay in the folder where they were soft-deleted (shift-delete or delete from Deleted Items) and are stamped with the ptagDeletedOnFlag flag. These items are special-cased in the store to be excluded from normal Outlook views and quotas. In addition, data with this flag cannot be searched or indexed. Note: Users can perform a shift-delete and cause a message to bypass the Deleted Items folder and go straight to dumpster. Prior to Outlook 2007, the Recover Deleted Items tool, by default, only exposed the dumpster view for the Deleted Items folder. By setting the DumpsterAlwaysOn registry key (http://support.microsoft.com/kb/886205) you can enable the Recover Deleted Items tool for all folders in the mailbox and thus expose dumpster data for any folder in the Exchange 2003 or Exchange 2007 mailbox. One of the key architectural changes in Exchange 2010 is to truly enable a litigation hold experience for customers that have legal compliance requirements. As a result, Exchange 2010 must meet these requirements: Exchange has to ensure that dumpster data moves with the mailbox. Dumpster data must be indexed and discoverable. Dumpster must have a quota. Exchange has to prevent purging of data from dumpster. Exchange has to track edits of certain content. Dumpster should be per mailbox and not per folder. In order to facilitate these requirements, Dumpster was re-architected. Unlike Dumpster 1.0, Dumpster 2.0 is no longer simply a view. Dumpster in Exchange 2010 is implemented as a folder called the Recoverable Items and is located within the Non-IPM subtree of the user's mailbox (note that this is a hidden section of the mailbox and is not exposed to the end user through any client interface). The folder has three sub-folders: Deletions Versions Purges The Deletions folder replaces the ptagDeletedOnFlag view that was displayed when a user accessed the Recover Deleted Items tool. When a user soft deletes or performs an Outlook hard delete against an item, the item is moved to the Recoverable Items\Deletions folder. When the user accesses Outlook/OWA Recover Deleted Items, the RPC Client Access service translates the request and returns the Recoverable Items\Deletions folder view. The Versions and Purges folders will be covered in the Single Item Recovery in Exchange 2010 section. By architecting Dumpster to be a folder, three of the requirements are immediately met: Dumpster data is now indexed and discoverable. Dumpster data can now be moved with the mailbox. Dumpster data is now stored on a per-mailbox basis rather than a per folder basis. From an end-user perspective, this means that deleted data is now easier to recover because Recover Deleted Items tool will now expose deleted data across the entire mailbox. To ensure that Denial of Service attacks by placing large quantities of data into dumpster are prevented, Dumpster 2.0 has configurable quota settings. These settings can be configured per database and per mailbox: RecoverableItemsWarningQuota - Sets the soft limit that defaults to 20 GB. When the Recoverable Items folder reaches that size, the Exchange administrator will be notified via an event log alert. This alert should fire at the time the mailbox reaches the limit and once daily afterward. By default, the mailbox is not configured with this property, meaning that database-level limits are used. At this limit items will begin to be deleted from the dumpster using the first-in / first-out (FIFO) principle - essentially, the oldest items in the dumpster are deleted first. For example, consider a mailbox that has a dumpster that is 20GB in size and the user deletes an additional 50MB worth of data. In this case, the oldest 50MB worth of data is deleted to make room for the new 50MB worth of data. RecoverableItemsQuota - Sets the hard limit that defaults to 30 GB. At that limit, soft deletes will fail. The Exchange administrator will be notified via an event log alert. This alert should fire at the time the mailbox reaches the limit and once daily afterward. By default, the mailbox is not configured with this property, meaning that database-level limits are used. Note: Exchange 2010 includes capability for each mailbox to also maintain an archive mailbox as well. There is a dumpster for both the primary mailbox and the archive mailbox. Data deleted in the primary mailbox is placed in the primary mailbox dumpster, while data deleted in the archive mailbox is placed in the archive mailbox dumpster. Single Item Recovery in Exchange 2010 But how does Exchange 2010 meet the other two requirements, ensuring data is not either accidentally or maliciously purged and that versions are tracked? Exchange 2010 now includes two mechanisms to meet those requirements: Short-term preservation of data Long-term preservation of data Short-Term Preservation of Data Exchange 2010 includes the ability to ensure that data within the mailbox is preserved for a period of time. This feature can be enabled enabled on a per mailbox basis by running the following cmdlet: Set-Mailbox -SingleItemRecoveryEnabled $true Note: When enabling this feature, you will be notified that it could take up to 60 minutes for single item recovery to take effect. This is an approximation due to Active Directory replication. Be sure to evaluate your Active Directory replication topology with respect to administering recipients to understanding how Active Directory replication may impact changes in your environment. The time period by which the deleted data is maintained is based on the deleted item retention window. The default time period is 14 days in Exchange 2010 and is configurable per database or per mailbox. The following cmdlets let you alter this behavior: For the mailbox database: Set-MailboxDatabase -DeletedItemRetention For the mailbox: Set-Mailbox -RetainDeletedItemsFor Note: Regardless, of whether you have Single Item Recovery enabled, calendar items are maintained in the Recoverable Items folder structure for 120 days. Long-term data preservation via litigation hold will disable the expiration of the items. At this point you may be thinking, how is this any different than in previous versions of Exchange? With short-term preservation deleted items will still be moved into the Recoverable Items folder structure. However, the data cannot be purged until deletion timestamp is past the deleted item retention window. Even if the end user attempts to purge the data, the data is retained. Consider this example by a malicious user: User sends or receives a message that is legally incriminating. User deletes the message. The message is moved to the Deleted Items folder. The user empties the deleted items folder. The user accesses the Recover Deleted Items functionality in Outlook or OWA. The user then selects the item and deletes the item. At this point the user believes he has removed the incriminating evidence. And this is a key strength in this work flow as the end user's actions are not interrupted or prevented; in other words, the end user's work flow is not impaired with single item recovery enabled. However, the message was not purged from the mailbox store. Instead the message was moved from the Recoverable Items\Deletions folder to the Recoverable Items\Purges folder. All store hard-deleted items end up in this folder when single item recovery is enabled. The Recoverable Items\Purges folder is not visible to the end user, meaning that they do not see data retained in this folder in the Recover Deleted Items tool. When the message deletion timestamp has exceeded the deleted item retention window, Records Management will purge the item. See Figure 2 for a visual representation of this behavior. Figure 2. Dumpster 2.0 Not only does short term preservation prevent purging of data before the deleted item retention window has expired, but it also enables versioning functionality. Essentially when an item is changed, a copy-on-write is performed to preserve the original version of the item. The original item is placed in the Recoverable Items\Versions folder. This folder is not exposed to the end user. What triggers a copy-on-write? For messages and posts (IPM.Note* and IPM.Post*), copy-on-write will capture changes in the subject, body, attachments, senders/recipients, and sent/received dates. For other types of items, copy-on-write will occur for any change to the item except for moves between folders and read/unread status changes.\ Drafts will be exempt from copy-on-write to prevent excessive copies when drafts are auto-saved. The data stored in the Recoverable Items\Versions folder is indexed and discoverable by compliance officers. Long-Term Preservation of Data Customers sometimes require mechanisms by which data is maintained for longer periods of time, say indefinitely. This is typically due to litigation hold that occurs when the organization is undergoing a lawsuit. With Exchange 2010, litigation hold can be enabled via the Exchange Control Panel or by setting the property LitigationHoldEnabled via the Set-Mailbox cmdlet. Note: When enabling this feature, you will be notified that it could take up to 60 minutes for single item recovery to take effect. This is an approximation due to Active Directory replication. Be sure to evaluate your Active Directory replication topology with respect to administering recipients to understanding how Active Directory replication may impact changes in your environment. When litigation hold is enabled, records management purging of dumpster data ceases. Consider the following four cases: When the end user deletes an item from the "Deleted Items" folder or shift-deletes Outlook/OWA from any folder (soft delete), the item is moved to Recoverable Items\Deletions folder, where it can be recovered in the Outlook /OWA "Recover Deleted Items" view. If the end user purges data from the "Recover Deleted Items" view (hard delete from the Recoverable Items\Deletions folder), the item will be moved to the Recoverable Items\Purges folder. When an end user modifies an item, a copy of the original item is placed in the Recoverable Items\Versions folder based on the criteria discussed in the Short-Term Preservation of Data section. Also, when litigation hold is enabled, the FIFO deletion at warning limit is ignored . When a user's Recoverable Items folder exceeds the warning quota for recoverable items (as specified by the RecoverableItemsWarningQuota parameter), an event is logged in the Application event log of the Mailbox server. When the folder exceeds the quota for recoverable items (as specified by the RecoverableItemsQuota parameter), users won't be able to empty the Deleted Items folder or permanently delete mailbox items. Also copy-on-write won't be able to create copies of modified items. Therefore, it's critical that you monitor the Recoverable Items quotas for mailbox users placed on litigation hold. Recovering Purged Data Data that is stored in the Recoverable Items\Purges folder is not accessible or discoverable by the end user. However the data is indexed and discoverable for those that have the proper access role in the Exchange organization role. Role Based Access Control (RBAC) provides the Discovery Management role to allow secure search access to non-technical personnel, without providing elevated privileges to make any operational changes to Exchange Server configuration. Compliance officers or other Exchange administrators with the Discovery Management role can leverage the Exchange Control Panel (ECP) to perform discovery searches using an easy-to-use search interface. When a single item recovery request is received by help desk, the following actions can be taken: Either Help Desk or a member of the Discovery Management role utilizes the Exchange Control Panel to target a search against the user's mailbox to locate the data in question. The framework to perform the search allows the administrator to use Advanced Query Syntax. The recovered data is then extracted and placed in the Discovery Mailbox in a folder that bears the user's name and the date/time that the search was performed. The administrator then opens the Discovery Mailbox via OWA or Outlook and verifies that the content recovered is the content requested by the end user. The administrator then leverages the Exchange Management Shell to perform an export-mailbox against the discovery mailbox targeting the end user's mailbox. The data is then placed back into the end user's mailbox. Help Desk can close the ticket. Backups and Single Item Recovery Naturally after understanding the features included in Exchange 2010, a logical follow up question is "Do I still need backups for single item recovery?" The answer depends on your backup requirements and your capacity planning. Today many customers minimize the deleted item retention window, yet they maintain long backup retention time periods (from 14 days to several months to years). Let's consider a customer that currently maintains backups for 90 days and only retains deleted items within Exchange for 5 days. This customer is performing backup restores on a weekly basis to recover deleted items for end users. If the customer moved to Exchange 2010 they could move that process into Exchange by simply increasing their mailboxes capacity for dumpster: Users send/receive 100 messages per work day and have an average message size of 50KB Single Item Recovery is enabled and the deleted retention window is configured to be 90 days 10% of items are edited Mailbox capacity calculations 5 work days * 100 emails = 500 emails / week For Purges: 500 emails / week * 13 weeks = 6500 emails / retention period 6500 emails * 50KB ? 318MB For Versions: 500 emails / week * 13 weeks = 6500 emails / retention period 6500 emails * .1 = 650 emails 650 emails * 50KB ? 32MB Total Space Required per mailbox: 350MB By increasing each mailbox's capacity by a minimum of 350MB, backups are no longer needed for single item recovery. Single item recovery can be maintained and performed within Exchange. But let's not stop there. What if the requirement is that items must be recoverable for 1 year? Assuming the same assumptions used in the previous example with the exception that deleted item retention is now configured for 365 days, each mailbox needs an additional minimum 1.4GB of space. Ultimately, if the storage subsystem is planned and designed appropriately and mailbox resiliency features are leveraged, traditional point-in-time backups can be relegated to a disaster recovery mechanism, if they are even needed at all. Conclusion Exchange 2010 introduces the concept of single item recovery. Single Item Recovery prevents purging of data and provides versioning capability (the ability to retain the unaltered form of the item). By default this data is retained until the age of the deleted item has exceeded the deleted item retention window. In addition, Exchange 2010 enables long-term preservation of data for litigation hold scenarios by preventing the purging of data all together. The following table summarizes the behavior in Exchange 2010: Feature State Soft-deleted items kept in dumpster Modified (versions) and store hard-deleted items kept in dumpster User can purge items from dumpster MRM automatically purges items from dumpster Single item recovery disabled Yes No Yes Yes, 14 days by default and 120 days for calendar items Single item recovery enabled Yes Yes No Yes, 14 days by default and 120 days for calendar items Litigation hold enabled Yes Yes No No - Ross Smith IVAccepted Domains, Safe Senders List and You
You may have noticed a change in the behavior of the Safe Senders list within Outlook starting in Exchange 2010. Users can no longer add accepted domains to Outlook’s Safe Senders list. Figure 1: Adding an accepted domain to Outlook's Safe Senders list This was done as an anti-spam deterrent as we have all seen cases where Joe The Spammer spoofs the mail from your own domain. Adding your own domain to the Safe Senders list would bypass all Outlook client-side anti-spam checks, dumping that message from the Nigerian prince (spoofed using your own domain) into your users’ Inboxes. Not so good unless you were really waiting for that business opportunity. A valid SPF record and our anti-spam agents (specifically the SenderID agent) would go a long way to block these types of spam. However, many customers out there have not exactly jumped on the SPF bandwagon. You can learn more about SenderID filtering in Sender ID and Understanding Sender ID. Use the Sender ID Framework SPF Record Wizard to create an SPF record for your domain. With Exchange 2010, you CAN still add individual email addresses from your own accepted domains to the Safe Senders list - you just can’t add the entire domain, as shown in the screenshot above. What happens if you DO decide to add the whole domain? If you try to do this for a user via the Shell, you will get the very helpful error below: “<@yourdomain.com>” is your e-mail address or domain and can’t be added to your Safe Senders and Recipients list. Figure 2: If you try to add an accepted domain to user’s Safe Senders list using the Shell, you get an error indicating its your domain or e-mail address We tell you exactly why we are throwing an error. How about when a user does this via Outlook? First, Outlook lets the user add a domain. Figure 3: Although users can add an accepted domain to their Safe Senders list in Outlook, it is automatically removed in a few minutes However after a few minutes the entry will magically disappear. The Disappearing Safe Senders List In Exchange 2010 SP1, a bug was introduced where if the user added the accepted domain to his/her Safe Senders list via Outlook - not only would the accepted domain entry disappear but it would take the user’s entire safe senders list with it. This was fixed in E2010 SP1 RU3v3 where we are back to the expected behavior. Allowing app servers to send as your own domain Many customers have various applications that submit mail anonymously to Exchange where the messages come from email addresses from your accepted domains. Some of you have apps submitting from so many accepted domain addresses that it wouldn’t be feasible (let alone fun) attempting to add all of these addresses individually to the safe senders list in Outlook to ensure these messages do not end up in junk mail. Now that we can’t add the whole domain, what are our options? We know that adding all the addresses manually is an available albeit painful option A second option is to disable Outlook’s client side filtering (yeah... not a good idea, so do not seriously consider it. Spam checks out the window!) A third and best option is to install the anti-spam agents on your relay hub(s) and add the IPs of your app servers to the IP Allow list of the connection filtering agent as documented here. When the sending SMTP host’s IP address is on the IP Allow List in Exchange, it bypasses all anti-spam checks (except sender/recipient filtering) and the Content Filter agent stamps an SCL of -1 on the message which Outlook will honor. Here's what the message header will look like: X-MS-Exchange-Organization-AuthAs: Anonymous X-MS-Exchange-Organization-Antispam-Report: IPOnAllowList X-MS-Exchange-Organization-SCL: -1 So, go ahead and run the Install-AntispamAgents.ps1 from the Scripts folder on your Hub Transport server, and then add IP addresses or subnets of our application servers to the IP Allow List. Figure 4: Adding IP address or address range of internal app servers to the IP Allow List using the EMC If using the Shell, use this command to add an IP address to the IP Allow List: Add-IPAllowListEntry –IPAddress 192.168.10.120 What Not To Do: Using Externally Secured Authentication Now I know what you’re thinking: Why don’t we just add Externally Secured Authentication as an authentication type on a Receive Connector, scope the connector’s remote IP range to the sending application servers and call it a day? Well, not so fast... you see, while Externally Secured gives the sending IP the ms-Exch-Bypass-Anti-Spam extended right, this only circumvents the Exchange Anti-Spam checks, not Outlook’s. And it is Outlook that’s moving the message into junk mail in this case. Also note that Externally Secured does not stamp any SCL X-headers on the message as an SCL of -1 would’ve bypassed Outlook’s checks. The only header this authentication type creates is the one below: X-MS-Exchange-Organization-AuthAs: Internal If you're still confused about Exchange and Outlook anti-spam checks, take a look at Exchange anti-spam myths revealed. Big thanks to Tak Chow for tech reviewing this post. Tom Kern
