Outlook clients receive error 0x8004010f when downloading the Offline Address Book
EDIT 10/01/2008: To read the updated (and more comprehensive) guidance for troubleshooting errors 0x8004010f, please go here. There are a multiple reasons for why an Outlook client can receive the 0x8004010f sync error. Unfortunately, 0x8004010f is just a generic MAPI error and will show up for a variety of problems. Here is what the error looks like under err.exe (Microsoft Exchange Server Error Code Look-up Tool): C:\WXP\system32>err 0x8004010f # for hex 0x8004010f / decimal -2147221233 ecNotFound ec.h ecAttachNotFound ec.h ecUnknownRecip ec.h ecPropNotExistent ec.h MAPI_E_NOT_FOUND mapicode.h # 5 matches found for "0x8004010f" This is what the error looks like from the sync log from within Outlook: 12:45:53 Synchronizing Mailbox <dgoldman> 12:45:53 Done 12:45:54 Microsoft Exchange offline address book 12:45:54 0x8004010f Some of the most common reasons for Outlook clients to receive the 0x8004010f error with regards to downloading the OAB are listed below. Most of these are documented and I have linked articles to each of these to help everybody out. Please note that these solutions can change a small bit depending on unknown factors in a company's environment. An administrator decommissioned the last Exchange server in a site and never pushes replicas to another Exchange server. A new OAB is created in the active directory and the information store never reads the active directory during its maintenance schedule. This will result in the OAB files never being generated and the Outlook client will fail to download anything. The information store has an invalid EntryID that points to the legacy EX:/ folders. Again there is nothing for the client to download. An outlook client logging in from one domain to another domain and attempting to log in to another users mailbox. The OAB was never generated or some OAB folders are missing from the public folder store. Multiple OAB Version folders exist of the same type. Clients are attempting to download the OAB files from a public folder store that have not received the replicated updates. The offline address book list object has a missing address list. The offline address book list object has an incorrect address list. Send/As changes in the store affect users accounts with no mailbox full rights to another mailbox. If you are seeing this error on an Exchange 2007 server and your OAB is generated by an Exchange 2007 server, please make sure of the following: 1. Make sure that you have added the replicas of OAB to the Exchange 2007 server 2. Make sure public folder replication is working. 3. Make sure the OAB is public folder enabled and you have OAB Version 2, OAB Version 3 and OAB Version 4 checked off so your legacy clients can download the OAB files from the public folder store. 4. Make sure that if you are using an Outlook 2007 client, your OAB is Web Distribution enabled and the OAB files have been replicated over to the Client Access Server. For more information on this process please see this blog: http://blogs.msdn.com/dgoldman/archive/2006/10/23/outlook-client-fails-to-download-the-oab-with-error-0x8004011b.aspx and http://blogs.msdn.com/dgoldman/archive/2006/08/25/How-Exchange-2007-OAB-Files-are-replicated-to-a-Client-Access-Server-for-download.aspx 5. If you are removing your last Exchange 2003 server from the org, please make sure that you follow our documentation on this process. - Dave GoldmanMoving mailboxes? Make sure your Deleted Mailbox Retention is not set to 0.
I wanted to bring to light a specific issue that may occur during mailbox moves in complex environments with multiple Active Directory Sites and multiple Exchange servers. There are several other settings which must be in place in order for this issue to present itself. First, here are the settings which need to be in place: 1. On both the Source and Target mailbox stores, set Deleted Mailbox Retention to 0 days. 2. Note what the Online Maintenance schedule is set to on both stores. The moves need to be scheduled to coincide with Online Maintenance on the Target mailbox store. *Note - for the purposes of my testing, I set Online Maintenance to run Always for my databases. 3. There must be multiple Active Directory Sites so as to introduce AD replication latency (Inter-Site connectors can only replicate every 15 minutes), and there need to be Exchange servers in these different AD sites (or Exchange must be configured to statically point to the GC of the other site). 4. Enable Diagnostics Logging under MSExchangeIS\Mailbox, and set the category General to Maximum (only minimum is necessary though). This is necessary in order to log Online Maintenance events. Some background here: one of the tasks performed by Online Maintenance is to identify mailboxes that are past the retention date and to delete them. As part of this task, it will also run the Cleanup Agent, which is the task that identifies orphaned/reconnected mailboxes. If you recall, in Exchange 2000 and 2003, if you delete a mailbox, it is not immediately deleted, nor is it marked as disconnected. If a mailbox is orphaned (no user connected to it), it gets marked as disconnected; if it is disconnected and past the retention date, it gets deleted. For more information on the tasks run during Online maintenance, please see the following link: http://technet.microsoft.com/en-us/library/aa996226(EXCHG.65).aspx During a mailbox move, the destination mailbox will be created so that data can be copied to it. This destination mailbox is not yet associated with a user account though. This process of associating with the user account does not happen until the very end of the mailbox move and all content has been copied to the destination mailbox. During the final step of the move, the user attributes are updated to point to the new server (if applicable) and new database. This process of updating attributes is what associates the destination mailbox with the user account. If you manually run the Cleanup Agent on the destination mailbox store during a move, you will find that the destination mailbox will show up as a disconnected mailbox. If you then run the Cleanup Agent again right after the move completes, you will find that in some cases the destination mailbox will also be purged. This can cause problems, because when a move is successful, the source mailbox is removed as well. If the Cleanup Agent is run from another Exchange server that is pointing at a different Active Directory Site, and the Exchange attributes (homeMDB, homeMTA, msExchHomeServerName) have not yet been updated, the Cleanup Agent will detect that this mailbox is not attached to a user account, and that it is past the retention date; and therefore, the mailbox will be deleted. You should see the following events indicating that this is occurring: Event 9533 The user account for "user" does not exist in the directory or is not enabled for Exchange mail. This mailbox will be removed from the mailbox store "database" in 0 days. Following that, you will see: Event 9535 Cleanup of deleted mailboxes that are past the retention date is finished on database "database" The first time I saw this event, the mailbox was not deleted. Then we have round 2, a few minutes later: Event 9531 Starting cleanup of deleted mailboxes that are past the retention date on database "database" Event 9535 Cleanup of deleted mailboxes that are past the retention date is finished on database "database" 1 deleted mailboxes have been removed Event 9532 The user account "user" does not exist in the directory or is not enabled for Exchange mail. This mailbox has been removed from the mailbox store "database" There are two ways to prevent this from happening: Make sure that Deleted Mailbox Retention is set to a non-zero value. Configure the Online Maintenance interval so that it does not overlap with your scheduled mailbox moves. Alternatively, set Online Maintenance to not run on days when you will be scheduling moves. If you have encountered a situation similar to this, we want to hear from you! Please post a comment. - Ben WinzenzAre ghosts modifying distribution groups in your mixed-mode environment?
We've seen a few issues recently where members of DGs (Distribution Groups) in mixed-mode Exchange 200x and 5.5 seem to randomly and mysteriously disappear. We thought we'd share one known root-cause and also show you how to prevent the problem while doing your migration, if for whatever reason you are still running Exchange 5.5 =). One easy way to prevent this problem (and a few others) is to ALWAYS use the Exchange 2003 post SP1 cross-site mailbox migration wizard if you ever have to move mailboxes in a mixed-mode environment. With that said, typically the problem occurs when you use a directory export/import to move mailboxes between Exchange 5.5 sites. For example, if you have two Exchange 5.5 sites Site S (Source) and Site D (Destination) and you would like to move all the mailboxes in Site S to Site D you might perform the following steps. (When we refer to a DL we mean a Distribution List on the Exchange 5.5 side and when we refer to DG we mean a Distribution Group on the Active Directory side) Extract all the DLs of the mailboxes you intend to migrate Export the directory Information from the source Exchange 5.5 server in Site S Delete the source mailboxes in Site S Wait for the two Exchange 5.5 sites to synchronize the mailbox deletions. You may force DRC (Directory Replication Connector) replication at this stage to speed things up. Also wait for the Active Directory Account to become mailbox-disabled. Do a directory import of the mailbox information in Site D to recreate the mailboxes. Move the mailboxes from the Exchange 5.5 server in Site D to the Exchange 200x server in the same site The problem occurs after step 4. The following key points should help you understand why. Replication between Exchange 5.5 sites is governed by USN-Changed values. Any changes made to an object in a directory in one site only replicate to other sites if that object's USN-Changed value is higher than the corresponding value in some other site. For the special case of a mailbox deletion, you would expect the USN-Changed value for any DL that the mailbox is a member of to increment after the mailbox deletion. This is, however, not necessary because when you delete a mailbox from its 'home' site, the member property of the DL changes and the read-only copy of the mailbox in all the other sites gets deleted. When the read-only copy is deleted in all the other sites the DL membership is updated as well. The salient point is without incrementing a DL's USN-Changed, all the necessary changes for the sites to be in-sync are properly accounted for. This works well for a pure Exchange 5.5 environment but creates a problem for a mixed-mode one with an ADC (Active Directory Connector) in the mix. As we know, the ADC compares the msExchServer2HighestUSN value on the RCA (Recipient Connection Agreement) to the USN-Changed value of an object in the Exchange 5.5 directory to determine whether the Exchange 5.5 object should be replicated to the AD. If the USN-Changed on an object is greater than the msExchServer2HighestUSN on the RCA, the object has been changed in the Exchange 5.5 directory and needs to be updated in the AD. If msExchServer2HighestUSN is greater than or equal to USN-Changed, no changes have occurred on the 5.5 object that need to be updated (replicated to) on the AD side. See the following knowledge base article for further details 253840. In this situation, since the DLs' USN-Changed isn't incremented in the 5.5 directory when you delete the mailboxes, the corresponding DG (Distribution Group) membership in the AD isn't updated. If you later make changes that increment the USN-Changed on the DL, the entire object (including the earlier deletions) is replicated to the AD side and so some members seem to randomly disappear from the DG's members list. While replication in Exchange 5.5 is object-based and replication in the AD is attribute-based, Exchange 5.5 to AD replication is still object-based (think lowest common denominator). The solution to this problem is to force the USN-Changed on the DL to increment after performing the deletions in step 3. A DL's USN-changed increments under the following scenarios: If you use the Exchange 5.5 Administrator program to open the DL's properties, modify any value and click Apply or OK (Or if you modify the DL's properties by doing an directory export/import with the standard header fields) If you use the Exchange 5.5 Administrator program to open a mailbox's properties and remove a DL from the list of DLs that the mailbox is a member of If the DL is updated by DRC or ADC replication changes from other 5.5 sites or from the AD respectively Again, a DL's USN-Changed does not increment when you use the Exchange 5.5 administrator program to delete a mailbox from the DLs membership. To force the USN-Changed value on the DL to increment you need to make a 'dummy' change that falls under a, b or c after step 3. Our new steps to ensure we avoid the problem would therefore be: Extract all the DLs of the mailboxes you intend to migrate Export the directory Information from the source 5.5 server in Site S Delete the source mailboxes in Site S Use the Exchange 5.5 Administrator program to modify the "notes" field on all the DLs that contained the deleted mailboxes to force an increment of USN-Changed value. This step is critical Wait for the two Exchange 5.5 sites to synchronize the mailbox deletions. You may force DRC (Directory Replication Connector) replication at this stage to speed things up. Also trigger RCA replication and make sure that the 'member of' list for the AD user accounts that correspond to the deleted mailboxes is empty (except built-in groups such as Domain Users etc) Do a directory import of the mailbox information in Site D to recreate the mailboxes Move the mailboxes from the Exchange 5.5 server in Site D to the Exchange 200x server in the same site. Add the migrated mailboxes to the corresponding DLs manually using the Exchange 5.5 Administrator program. This step may not seem necessary but it is. Usually an administrator may notice that the member list is still present on the AD after performing the previous steps and therefore assume that nothing more needs to be done on the Exchange 5.5 side. Later when some other change increments the USN-Changed the deletions replicate to the AD and members seem to randomly disappear from DGs. Everything should work fine and dandy at this point and you needn't worry about 'ghosts' modifying your DGs! - Jasper Kuria and William YangI'll have some transaction log files for breakfast day
With viruses spreading quick out there today, we had several cases where Exchange transaction logs got either deleted, quarantined or "cured" by file-level anti-virus software that is running on Exchange servers... the result is a bad thing... stores down, transaction logs missing. In some cases the only thing you can do is go back to the last backup if one that is good is available. Otherwise - you are looking at possible repair (= data loss) + Isinteg (maybe 2-3 times) + mandatory offline defrag = a lot of time that is lost :( Please, do not let Exchange directories be scanned by file-level AV. Not "on-demand" one, not the memory resident one. Have Exchange directories excluded, the M: drive excluded, and actually - exclude specifically .log, .edb and .stm files too just to be extra careful. To be more specific, excluding the following on Exchange server is a GREAT idea: Exchange databases and log files. By default, these are located in the Exchsrvr\Mdbdata folder. You can verify the locations by pulling up properties of your databases in ESM and checking the Database tab. Exchange MTA files in the Exchsrvr\Mtadata folder. Additional log files such as the Exchsrvr\server_name.log file. The Exchsrvr\Mailroot virtual server folder. The working folder that's used to store streaming temporary files used for message conversion. By default, this folder is located at \Exchsrvr\MDBData, but you can configure the location. The temporary folder that is used in conjunction with offline maintenance utilities such as Eseutil.exe. By default, this folder is the location where the .exe file is run from, but you can configure where you run the file from when you run the utility. Site Replication Service (SRS) files in the Exchsrvr\Srsdata folder. Microsoft Internet Information Service (IIS) system files in the %SystemRoot%\System32\Inetsrv folder. The Exchange 2000 Server drive M. More appropriate reading: 328841 XADM: Exchange and Antivirus Software 823166 Overview of Exchange Server 2003 and Antivirus Software Nino BilicWhat's new with Microsoft Remote Connectivity Analyzer? A lot!
Have you ever gotten a call from someone you know whose Outlook client is having trouble connecting to their email server? Don’t you just wish you had a tool that you could send to them that would walk them through a connectivity test and provide a simple way for them to send the results back to you? Or perhaps you’re an administrator that has been using the Remote Connectivity Analyzer for years to verify email configuration, but wish you had a way to verify the same tests from within your organization, that provided the same diagnostic details as the RCA website? We’ve heard requests like these, and our team has built a tool for just these scenarios. I’d like to introduce you to the Microsoft Connectivity Analyzer (beta), a portable version of the Remote Connectivity Analyzer website. Here is a short 49 second video that introduces the Microsoft Connectivity Analyzer. This tool contains the same tests as the website, in a slightly simpler UI – something you can share with your users, family, or your neighbor down the street. In addition, we’ve split the results into two views – a simplified view which only shows results that your mom could understand, and the full detailed results that only an administrator would love (yes, that’s you). And the additional details for administrators is shown under “Review all the tests we ran”: Or ask your user to save the detailed results and send them to you! The detailed results are saved as an HTML file and can be viewed in your browser. Microsoft Connectivity Analyzer Pre-Requisites The tool supports the following operating systems: 64bit Windows 7, Windows 7, Windows Server 2008+ Microsoft .NET Framework 4.5 is required. Browsers requirements: The installation will work on all modern browsers. However… If you are using Google Chrome, you will need to install ClickOnce for Google Chrome If you are using FireFox, you will need to install Microsoft .Net Framework Assistant for FireFox. You can find the full release notes here. We're not finished yet. We have plans to add additional tests. Just like when we first released the website, this first version of the Microsoft Connectivity Analyzer is a work in progress. Having a tool on site with the end user, we’ re in a unique position to extend the testing to include common problems with all types of connectivity. This early version of the tool is just the beginning. And that’s not all! In addition to providing a new tool, we’ve joined forces with the Lync Remote Connectivity Analyzer, and brought the two tools together. You’ll notice we’ve dropped the “Exchange” from the name, and we have a new way to access the website: http://testconnectivity.microsoft.com. That’s right – all the same great tests for Lync and Exchange available on the same website. Tests include: Microsoft Lync Mobile Auto-Discover Web Service Remote Connectivity Test will test your remote connectivity to the Microsoft Lync Mobile Auto-Discover Web Service Server Microsoft Office Communications Server Test will test your remote connectivity to the Microsoft Office Communications Server, it will auto-discover the Access Edge and port to connect to, OR allow you to specific an Access Edge server. Microsoft Lync Server Remote Connectivity Test will test your remote connectivity to the Microsoft Lync Server, it will auto-discover the Access Edge and port to connect to, OR allow you to specific an Access Edge server. But wait, there’s more… Did we mention we’ve been busy this year? On the Office 365 tab, you’ll notice two new tests – the Office 365 Lync DNS test, and the Free/Busy test. The Office 365 Lync DNS Test will check the external domain name settings for your custom domain user in Office365. Thanks to the Office 365 Deployment Services team, the RCA website now hosts the Office 365 Free/Busy test. This test verifies an Office 365 mailbox can access the free/busy information of an on-premises mailbox and vice versa. This includes: a check to confirm the system time of the hybrid server is not offset by more than five minutes, which causes failures when requesting delegation tokens from the Microsoft Federated Gateway. a check to verify inbound connectivity to the hybrid server does not require firewall pre-authentication; that is, the firewall allows pass-through authentication. a check to verify the hybrid server meets the minimum Exchange Server version requirement (Exchange Server 2010 SP1). a basic free/busy query against the target Availability Service. Links to guidance on the Hybrid Configuration Wizard, a common source of hybrid deployment misconfiguration Phew! Thanks for reading this far. We hope you enjoy the new updates, and we look forward to hearing from you, Nicole Allen On behalf of the RCA team
