Exchange 2010 SP1 FAQ and Known Issues

[The_Exchange_Team]

Last week we released Exchange Server 2010 Service Pack 1. It has received some great feedback and reviews from customers, experts, analysts, and the Exchange community.

The starting point for SP1 setup/upgrade should be the What's New in SP1, SP1 Release Notes, and Prerequisites docs. As with any new release, there are some frequently asked deployment questions, and known issues, or issues reported by some customers. You may not face these in your environment, but we're posting these here along with some workarounds so you're aware of them as you test and deploy SP1.

1. Upgrade order

   The order of upgrade from Exchange 2010 RTM to SP1 hasn’t changed from what was done in Exchange 2007. Upgrade server roles in the following order:

   1. Client Access server
   2. Hub Transport server
   3. Unified Messaging server
   4. Mailbox server

   The Edge Transport server role can be upgraded at any time; however, we recommend upgrading Edge Transport either before all other server roles have been upgraded or after all other server roles have been upgraded. For more details, see Upgrade from Exchange 2010 RTM to Exchange 2010 SP1 in the documenation.

2. Exchange 2010 SP1 Prerequisites

   Exchange 2010 SP1 requires the installation of 4-5 hotfixes, depending on the operating system – Windows Server 2008, or Windows Server 2008 R2. To install the Exchange 2010 SP1 administration tools on Windows 7 and Windows Vista, you requires 2 hotfixes.

   Note: Due to the shared code base for these updates, Windows Server 2008 and Windows Vista share the same updates. Similarly, Windows Server 2008 R2 and Windows 7 share the same updates. Make sure you select the x64 versions of each update to be installed on your Exchange 2010 servers.

   Update 2/11/2011: Windows 2008 R2 SP1 includes all the required hotfixes listed in this table — 979744, 983440, 979099, 982867 and 977020. If you're installing Exchange 2010 SP1 on a server running Windows 2008 R2 SP1, you don't need to install these hotfixes separately. For a complete list of all updates included in Windows 2008 R2 SP1, see Updates in Win7 and WS08R2 SP1.xls.

   Here’s a matrix of the updates required, including download locations and file names.

   Hotfix	Download	Windows Server 2008	Windows Server 2008 R2	Windows 7 & Windows Vista
   979744 A .NET Framework 2.0-based Multi-AppDomain application stops responding when you run the application	MSDN or Microsoft Connect	Windows6.0-KB979744-x64.msu (CBS: Vista/Win2K8)	Windows6.1-KB979744-x64.msu (CBS: Win7/Win2K8 R2)	N. A.
   983440 An ASP.NET 2.0 hotfix rollup package is available for Windows 7 and for Windows Server 2008 R2	Request from CSS	N. A.	Yes	N.A.
   977624 AD RMS clients do not authenticate federated identity providers in Windows Server 2008 or in Windows Vista. Without this update, Active Directory Rights Management Services (AD RMS) features may stop working	Request from CSS	Select the download for Windows Vista for the x64 platform.	N.A.	N.A.
   979917 Two issues occur when you deploy an ASP.NET 2.0-based application on a server that is running IIS 7.0 or IIS 7.5 in Integrated mode	MSDN	Windows6.0-KB979917-x64.msu (Vista)	N. A.	N. A.
   973136, FIX: ArgumentNullException exception error message when a .NET Framework 2.0 SP2-based application tries to process a response with zero-length content to an asynchronous ASP.NET Web service request: "Value cannot be null".	Microsoft Connect	Windows6.0-KB973136-x64.msu	N.A.	N. A.
   977592 RPC over HTTP clients cannot connect to the Windows Server 2008 RPC over HTTP servers that have RPC load balancing enabled.	Request from CSS	Select the download for Windows Vista (x64)	N.A.	N. A.
   979099 An update is available to remove the application manifest expiry feature from AD RMS clients.	Download Center	N. A.	Windows6.1-KB979099-x64.msu	N. A.
   982867 WCF services that are hosted by computers together with a NLB fail in .NET Framework 3.5 SP1	MSDN	Windows6.0-KB982867-v2-x64.msu (Vista)	Windows6.1-KB982867-v2-x64.msu (Win7)	X86: Windows6.1-KB982867-v2-x86.msu (Win7) x64: Windows6.1-KB982867-v2-x64.msu (Win7)
   977020 FIX: An application that is based on the Microsoft .NET Framework 2.0 Service Pack 2 and that invokes a Web service call asynchronously throws an exception on a computer that is running Windows 7.	Microsoft Connect	N. A.	x64: Windows6.1-KB977020-v2-x64.msu	x64: Windows6.1-KB977020-v2-x64.msu X86: Windows6.1-KB977020-v2-x86.msu

   Some of the hotfixes would have been rolled up in a Windows update or service pack. Given that the Exchange team released SP1 earlier than what was planned and announced earlier, it did not align with some of the work with the Windows platform. As a result, some hotfixes are available from MSDN/Connect, and some require that you request them online using the links in the corresponding KBAs. The administrator experience when initially downloading these hotfixes may be a little odd. However, once you download the hotfixes, and receive two of the hotfixes from CSS, you can use the same for subsequent installs on other servers. In due course, all these updates may become available on the Download Center, and also through Windows Update.

   These hotfixes have been tested extensively as part of Exchange 2010 SP1 deployments within Microsoft and by our TAP customers. They are fully supported by Microsoft.

3. Prerequisite download pages linked from SP1 Setup are unavailable

   When installing Exchange Server 2010 SP1 the prereq check may turn up some required hotfixes to install. The message will include a link to click for help. Clicking this link redirects you to a page saying that the content does not exist.

   We're working to update the linked content.

   Meanwhile, please refer to the TechNet article Exchange 2010 Prerequisites to download and install the prerequisites required for your server version (the hotfixes are linked to in the above table, but you'll still need to install the usual prerequisites such as .Net Framework 3.5 SP1, Windows Remote Management (WinRM) 2.0, and the required OS components).

4. The Missing Exchange Management Shell Shortcut

   Some customers have reported that after upgrading an Exchange Server 2010 server to Exchange 2010 SP1, the Exchange Management Shell shortcut is missing from program options. Additionally, the .ps1 script files associated with the EMS may also be missing.

   We’re actively investigating this issue. Meanwhile, here’s a workaround:

   1. Verify that the following files are present in the %ExchangeInstallPath%\bindirectory:
      • - CommonConnectFunctions.ps1
      • - CommonConnectFunctions.strings.psd1
      • - Connect-ExchangeServer-help.xml
      • - ConnectFunctions.ps1
      • - ConnectFunctions.strings.psd1
      • - RemoteExchange.ps1
      • - RemoteExchange.strings.psd1

      NOTE: If these files are missing, you can copy the files from the Exchange Server 2010 Service Pack 1 installation media to the %ExchangeInstallPath%\bin directory. These files are present in the \setup\serverroles\common folder. < /note>

   2. Click Start -> AdmiinistrativeTools ->, right-click Windows PowerShell Modules, select Send to -> Desktop (as shortcut)
   3. Go to the Properties of the shortcut and on Target replace the path to C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe -noexit -command ". 'C:\Program Files\Microsoft\Exchange Server\V14\bin\RemoteExchange.ps1'; Connect-ExchangeServer -auto"

      Note: if the Exchange installation folder or drive name is different than the default, you need to change the path accordingly.

5. Upgrading Edge Transport on Forefront Threat Management Gateway (TMG) and Forefront Protection for Exchange 2010

   If you upgrade a server with the Edge Transport server role running with ForeFront Threat Management Gateway (TMG) and ForeFront Protection for Exchange (FPE) enabled for SMTP protection, the ForeFront TMG Managed Control Service may fail to start and E-mail policy configuration settings cannot be applied.

   The TMG team is working on this issue. See Problems when installing Exchange 2010 Service Pack 1 on a TMG configured for Mail protection on the ForeFront TMG (ISA) Team Blog. Exchange 2010 SP1 Release Notes has been updated with the above information.

   The ForeFront TMG product team has released a software update to address this issue. See Software Update 1 for Microsoft Forefront Threat Management Gateway (TMG) 2010 Service Pack 1 now av....

6. Static Address Book Service Port Configuration Changes

   The location for setting the port the address book service should use has changed in SP1. In Exchange 2010 RTM you had to edit the Microsoft.exchange.addressbook.service.exe.config to configure the service port. In SP1 you must use the following registry key:
   Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\MSExchangeAB\Parameters
   Value name: RpcTcpPort
   Type: REG_SZ (String)

   
   

   When you apply SP1 to a machine where you had previously configured a static port by editing the Microsoft.exchange.addressbook.service.exe.config file, the upgrade process will not carry forward your static port assignments. Following a restart, the Address Book Service will revert to using a dynamic port instead of a static port specified in the config file. This may cause interruptions in service.

   As with all upgrades where servers are in load balanced pools, we recommend you perform a rolling upgrade — removing servers from the pool, updating them and then moving the pool to the newly upgraded machines. Alternatively, we recommend that you upgrade an array of servers by draining connections from any one machine before you upgrade it.

   There are times when these approaches may not be possible. You can maintain your static port configuration, and have it take effect the moment the address book service starts for the first time following the application of the service pack, by creating the registry key BEFORE you apply SP1 to your server. The registry key has no impact pre SP1, and so by configuring it before you apply the Service Pack you can avoid the need to make changes to set the port post install, and avoid any service interruptions.

7. iPhone, OWA Premium and POP3 & IMAP4 issues due to invalid accepted domain

   After applying E2010 SP1:

   1. iPhone users may not be able to view the content of incoming messages in their Inboxes, and when they try to open a message, they get an error saying:

      This message has not been downloaded from the server.

      Admins may see the following event logged in the Application Event Log on Exchange 2010 CAS Server:

      Watson report about to be sent for process id: 1234, with parameters: E12, c-RTL-AMD64, 14.01.0218.011, AirSync, MSExchange ActiveSync, Microsoft.Exchange.Data.Storage.InboundConversionOptions.CheckImceaDomain, UnexpectedCondition:ArgumentException, 4321, 14.01.0218.015.

   2. OWA Premium users may not be able to reply or forward a message. They may see the following error in OWA:

      An unexpected error occurred and your request couldn't be handled. Exception type: System.ArgumentException, Exception message: imceaDomain must be a valid domain name.

   3. POP3 & IMAP4 users may also not be able to retrieve incoming mail and Admins will see the following event logged in Event Log:

      ERR Server Unavailable. 21; RpcC=6; Excpt=imceaDomain must be a valid domain name.

   Resolution

   Please run the following command under Exchange Management Shell and verify that there is one domain marked as ‘Default’ and it's DomainName & Name values are valid domain names. We were able to reproduce the issue by setting a domain name with a space in it, like "aa bb"

   Get-AcceptedDomain | fl

   If you also have an invalid domain name there (for example, a domain name with a space in it), then removing the space and restarting the server will fix the EAS (iPhone), OWA, POP3 & IMAP4 issues as mentioned above.

   Command to run under EMS would be:

   Set-AcceptedDomain –Identity alue>lue>ue>e>>
   <
   value>-Name “ValidSMTPDomainName”

   Thes examples update the Name parameter of the "My Company" and "ABC Local" accepted domains (the space is removed from both):

   Set-AcceptedDomain –Identity “My Company” –Name “MyCompany.Com”
   Set-AcceptedDomain –Identity “ABC Local” –Name “ABC.Local”

8. Error when adding or removing a mailbox database copy

   If a server running Exchange 2010 RTM (or Exchange 2010 SP1 Beta) is upgraded to Exchange 2010 SP1, administrators may experience an error when using the Add-MailboxdDatabaseCopy or Remove-MailboxDatabaseCopy cmdlets to add or remove DAG members.

   When you try to add a DAG member, you may see the following error:

   Add-MailboxDatabaseCopy DAG-DB0 -MailboxServer DAG-2

   The result:

   WARNING: An unexpected error has occurred and a Watson dump is being generated: Registry key has subkeys and recursive removes are not supported by this method.
   
   Registry key has subkeys and recursive removes are not supported by this method.
   + CategoryInfo : NotSpecified: (:) [Add-MailboxDatabaseCopy], InvalidOperationException
   + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.Exchange.Management.SystemConfigurationTasks.
   AddMailboxDatabaseCopy

   The command is not successful in adding the copy or updating Active Directory to show the copy was added. This happens due to presence of the DumpsterInfo registry key.

   Workaround: Delete the DumpsterInfo key, as shown below.

   1. Identify the GUID of the database that is being added using this command:

      Get-MailboxDatabase DAG-DB0 | fl name,GUID

      The result:

      Name : DAG-DB0
      Guid : 8d3a9778-851c-40a4-91af-65a2c487b4cc

   2. On the server specified in the add command, using the database GUID identified, remove the following registry key:
      HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v14\Replay\State\<db-guid>\DumpsterInfo

      The GUID identified in this case is 8d3a9778-851c-40a4-91af-65a2c487b4cc. With this information you can now export and delete the DumpsterInfo key on the server where you are attempting to add the mailbox database copy. This can be easily done using the registry editor, but if you have more than a handful of DAG members, this is best automated using the Shell.

      This example removes the DumpsterInfo key from the 8d3a9778-851c-40a4-91af-65a2c487b4cc key:

      Remove-Item HKLM:\Software\Microsoft\ExchangeServer\V14\Replay\State\8d3a9778-851c-40a4-91af-65a2c487b4cc\DumpsterInfo

      To automate this across all servers in your organization, use the DeleteDumpsterRegKey.ps1 script.

      File: deletedumpsterregkey_ps1.txt
      Description: The DeleteDumpsterRegkey.ps1 script can be used to delete the offending DumpsterInfo registry keys that can cause this problem on all Exchange 2010 SP1 Mailbox servers in the organization. Rename the file to DeleteDumpsterRegkey.ps1 (remove the .txt extension).

      For more info, see Tim McMichael’s blog post Exchange 2010 SP1: Error when adding or removing a mailbox database copy.

Thanks to all the folks in CSS and Exchange teams who helped identify, validate and provide workarounds for some of the issues mentioned above, and to the Exchange community and MVPs for their feedback.

Bharat Suneja, Nino Bilic
M. Amir Haque, Greg Taylor,
& Tim McMichael

Updates:

• 9/7/2010: Updated list of files for the missing Exchange Management Shell shortcut issue
• 9/15/2010: Udpated pre-reqs table:
  - 982867 required on Windows 2008 SP2
  - 983440 not required on Windows 2008 SP2
  - 977020 required on Windows 2008 R2
• 9/21/2010: Added link to Software Update 1 for ForeFront Threat Management Gateway (TMG) 2010 Service Pack 1
  - Replaced "Request from CSS..." verbiage for KB 979917 with link to KB 979917 download on MSDN
• 9/22/2010: Updated correct default Exchange install path (highlighted) in 'The Missing Exchange Management Shell Shortcut" section: C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe -noexit -command ". 'C:\Program Files\Microsoft\Exchange Server\V14\bin\RemoteExchange.ps1'; Connect-ExchangeServer -auto"