Why can't I do a Deep Traversal (subfolder) Search on the MAPI Public Folder Tree?
I am sure many of you have at one time or another wanted to do a Deep Traversal (subfolder) Search on the MAPI Public Folder Tree. However, as many of you probably know, Exchange does not allow for Deep Traversal (subfolder) Searches on the MAPI Public Folder Tree: http://support.microsoft.com/default.aspx?scid=kb;en-us;254911. The reason you cannot do this is because when the Public Folder system was designed, it was decided that load balancing would be accomplished by putting only some of the content on some of the servers. For example, if there exists 3 folders and 3 servers. Folder1 may be replicated between servers 1 & 2, folder2 between servers 2 & 3, and folder3 between 1 & 3. If the client created a search folder that simultaneously searched all three folders, they’d get different results depending on which actual Public Folder Server they were connected to. To address this issue, search folders cannot be created that search the content of multiple folders; deep traversal, or subfolder, being a special case of “multiple folders”. One way to get around this “limitation” is by setting up a Non-MAPI Public Folder Tree because you can do a Deep Traversal (subfolder) Search against that: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/e2k3/e2k3/_exch2k_specifying_a_deep_traversal.asp?frame=true. However, you cannot view this Public Folder Tree via MAPI (i.e Outlook). So what is the difference you ask? Well, the thought behind Application Public Folder Stores was that, for the most part, the content of the folders would not be replicated to any other server. There’s no actual architectural limitation to enforce this – Exchange originally just foresaw organizations would create only one Application Public Folder Store at all and so replication wouldn’t be an issue. Therefore, users can create Search Folders that can search the content of multiple folders. Given this, though, you do actually step back into the original problem. Namely, the search folder itself is replicated among servers but the content is always generated locally. Also, clients are never referred to a different server – search folders always seem (to the client) to have their exclusive replica on the server presently being queried. So, users should be aware that they will get different results depending on which server they contact for the content, and of course replication latency adds a new dimension to the differing results. - Chris AhlersUnderstanding Exoledb default folders
EXOLEDB creates a number of system folders under the NON_IPM_SUBTREE during the Accept Clients phase of MDB initialization. Some of the folders remain for historic reasons, but most have real purposes and impact on the server should they be deleted. None of these folders should ever be replicated. The folders that are created are: \NON_IPM_SUBTREE\schema-root\ \NON_IPM_SUBTREE\schema-root\Default \NON_IPM_SUBTREE\schema-root\Microsoft\ \NON_IPM_SUBTREE\schema-root\Microsoft\exchangeV1 \NON_IPM_SUBTREE\schema-root\Microsoft\exchangeV1\exchweb \NON_IPM_SUBTREE\schema-root\Microsoft\exchangeV1\exchweb\controls \NON_IPM_SUBTREE\schema-root\Microsoft\exchangeV1\exchweb\img \NON_IPM_SUBTREE\schema-root\Microsoft\exchangeV1\exchweb\views \NON_IPM_SUBTREE\StoreEvents\ \NON_IPM_SUBTREE\StoreEvents\GlobalEvents \NON_IPM_SUBTREE\StoreEvents\Internal \NON_IPM_SUBTREE\OWAScratchPad In all cases, Subfolders named with the GUID correspond to the MDB object with the same guid. The first folders created are the Schema folders: Schema-Root \NON_IPM_SUBTREE\schema-root\ - introduced in Exchange 2000 RTM. \NON_IPM_SUBTREE\schema-root\Default - introduced in Exchange 2000 SP1 \NON_IPM_SUBTREE\schema-root\Microsoft\ - introduced in Exchange 2000 SP1 \NON_IPM_SUBTREE\schema-root\Microsoft\exchangeV1 - introduced in Exchange 2000 SP1 The typical schema path for a public mdb looks like: File://.BackOfficeStorage/<domain>/<TLHName>/NON_IPM_SUBTREE/schema-root/microsoft/exchangeV1 The private MDB schema path is under the system attendant mailbox. EXOLEDB supports multiple schemas, or property type definitions. This was part of support for the "never fully implemented" exchange web store development platform. The idea was that Folder items could reference various versions of the schema and live happily alongside of each other. At one point (Exchange 2000 RTM) schema files lived in the schema root folder, and changes to the schema effectively propagated to all items. This lead to problems in the application development workspace where each item needed to be touched to remove or add props as appropriate, so we adopted a versioning method. Under schemaroot, we create subfolders with application and version elements to allow effectively seamless upgrades. EXOLEDB watches the schema folders for changes so that it can propagate the entries, dump the schema cache, and repopulate on the fly. The \schemaroot\default folder is where normal folder items pull their schema from, and the schema root folder is flagged as pointing to the ExchangeV1 folder. EXOLEDB populates the schema entries from the .xml files, which are processed by an event sink - EXSCHEMA.EXE. The schema event sink binding cannot be deleted or removed, as it does not have an entry in the EventBindings folder like most events. EXCHWEB, Views, IMG, Controls \NON_IPM_SUBTREE\schema-root\Microsoft\exchangeV1\exchweb \NON_IPM_SUBTREE\schema-root\Microsoft\exchangeV1\exchweb\controls \NON_IPM_SUBTREE\schema-root\Microsoft\exchangeV1\exchweb\img \NON_IPM_SUBTREE\schema-root\Microsoft\exchangeV1\exchweb\views Introduced in Exchange 2000 SP1, no longer populated in Exchange 2000 Post SP3 hotfixes and Exchange 2003. For the Local Store to open items that referenced OWA control functionality the files had to be in a sync-able folder. These folders once contained copies of the web data for OWA to allow LIS stored items to open, but have never actually been used outside of LIS. Next, EXOLEDB starts the event binding system, which creates StoreEvents All store event folders have been present since Exchange 2000 RTM. \NON_IPM_SUBTREE\StoreEvents\ \NON_IPM_SUBTREE\StoreEvents\GlobalEvents \NON_IPM_SUBTREE\StoreEvents\Internal This is the event binding folder, where EXOLEDB stores information on events build to a given MDB. At startup EXOLEDB must enumerate the events here, which can lead to long store startup times with large event sink numbers. Titanium performance in this area is greatly improved, but time to mount an MDB is still affected by number of rows. Each binding is validated for class, having a valid event method (onsave, ontimer, ect), valid clsid, and sink parameters. Events with a match class of ANY can only be registered in the GlobalEvents subfolder. After creating the Schema folders and starting the event bindings system, EXOLEDB creates the OWA scratch pad OWAScratchPad \NON_IPM_SUBTREE\OWAScratchPad - Introduced in Exchange 2000 SP1 Posts have to start out somewhere in order to have attachments, and for public store logons, that place is the OWA scratch pad. Because DAV doesn’t do cross mdb operations, we need a point on every mailbox where we can always write posts to, so that we can support adding attachments. The Posts are "Staged" in the OWAScratchPad until all attachments are added (or they are saved). The size limit on the OWA scratch pad controls the size of attachments that can be added via OWA. Attempts to post larger messages should result in an error: "This item exceeds the maximum size defined for this folder and cannot be saved. Contact your administrator to have the folder limits increased." The size of OWAScratchPad is always reset to 1mb at EXOLEDB initialization if the registry key "HKLM\System\CurrentControlSet\Services\MSExchangeWeb\OWA", a REG_DWORD value "Message Size Limit" is not set. This is required for Sharepoint, because EXOLEDB has no idea of we are running in magma mode or not. OWA posts to the scratch pad are done in flat url format, meaning they directly reference the folder and message. This is to support deep vroots where the friendly URL might be too long. What causes duplicate system folders? This question really falls into two categories. AD Objects -Because when a store is deleted (disaster recovery & friends) we have no way to tell the AD the PF objects went away, then when folders are re-created they do not get "attached" to the corresponding DS objects. New ones are created. Actual Folders - If the folders are set to replicate, and the store in question is deleted, EXOLEDB will recreate the folders on startup, and replication can then create a second, duplicate of any such folders. This wreaks havoc with event bindings. Deleting said folder via friendly URL is dangerous, as the two will often have duplicate friendly URLs.. Why do they get strange names? When the number of system folders with the same number grows, we append a random number to the DS proxy to make it unique, resulting in names like "controls12345678". Why can I not delete them? First off, if you were to delete the folders, EXOLEDB would put them back. Secondly, most of these folders have real uses that will adversely affect the operation of the server if not present. - Jason NelsonSo how does this Deferred Delivery thing work?
There are actually 2 different types of delayed message delivery... I believe stemming form the quaintness of X400. The following applies to Exchange2000/Exchange2003. They have somewhat different behaviors, but both result in getting the message delivered to the intended recipients at some time after the specified time. The first type is delayed send. This is the typical functionality you see when using Outlook to send email. This functionality is controlled by the MAPI property PR_DEFERRED_SEND_TIME. When this property is set, the the message is kept in the MSExchangeIS SendQ. To end users, this means that their messages are visible in their Outbox. They can still double-click on them to open them, and will need to click "Send" for them to resubmitted for delivery. For an administrator, they will need to log into the users mailbox to see the message in that users Outbox. Unless the mailbox is deleted and purged The second type is deferred delivery (controlled by the MAPI property PR_DEFERRED_DELIVERY_TIME). The key difference is that the delay happens on the "delivery" side once transport "owns" the message. While the phrase "deferred delivery" might make you think that message is held on the destination server, it is in fact held in the queues on the source server. From the end-users perspective, the message immediately disappears from their mailbox and they cannot modify it if something occurs to them later. This is somewhat moot, as the past several versions of Outlook do not use this feature. For the email administrator, the message is queued up in the MDB temp tables along with all of the other mail. In Exchange2003, these messages are also exposed through the queues node of the Exchange System Manager. Deleting and purging the mailbox have no effect on these messages, but deleting the MDB they are queued on will make the messages disappear. - Michael SwaffordI'll have some transaction log files for breakfast day
With viruses spreading quick out there today, we had several cases where Exchange transaction logs got either deleted, quarantined or "cured" by file-level anti-virus software that is running on Exchange servers... the result is a bad thing... stores down, transaction logs missing. In some cases the only thing you can do is go back to the last backup if one that is good is available. Otherwise - you are looking at possible repair (= data loss) + Isinteg (maybe 2-3 times) + mandatory offline defrag = a lot of time that is lost :( Please, do not let Exchange directories be scanned by file-level AV. Not "on-demand" one, not the memory resident one. Have Exchange directories excluded, the M: drive excluded, and actually - exclude specifically .log, .edb and .stm files too just to be extra careful. To be more specific, excluding the following on Exchange server is a GREAT idea: Exchange databases and log files. By default, these are located in the Exchsrvr\Mdbdata folder. You can verify the locations by pulling up properties of your databases in ESM and checking the Database tab. Exchange MTA files in the Exchsrvr\Mtadata folder. Additional log files such as the Exchsrvr\server_name.log file. The Exchsrvr\Mailroot virtual server folder. The working folder that's used to store streaming temporary files used for message conversion. By default, this folder is located at \Exchsrvr\MDBData, but you can configure the location. The temporary folder that is used in conjunction with offline maintenance utilities such as Eseutil.exe. By default, this folder is the location where the .exe file is run from, but you can configure where you run the file from when you run the utility. Site Replication Service (SRS) files in the Exchsrvr\Srsdata folder. Microsoft Internet Information Service (IIS) system files in the %SystemRoot%\System32\Inetsrv folder. The Exchange 2000 Server drive M. More appropriate reading: 328841 XADM: Exchange and Antivirus Software 823166 Overview of Exchange Server 2003 and Antivirus Software Nino BilicHow does duplicate detection work?
Duplicate detection of delivered messages is done by the Exchange store. The store does duplicate detection based on two properties on the message - the Internet Message Id and the client submit time. We would have liked to do duplicate detection based solely on the Internet Message Id but there are several not-to-be named applications out there that use the same Internet Message Id on all their messages. The store keeps track of duplicates using a table in JET called the DeliveredTo table. When a message is delivered to a user, the store checks this table and if no entry is found the message is delivered to the user and a row is added to this table to indicate that the user received the message. If an entry is found, the message is turfed. The store only tracks duplicates for 1 hour by default. This can be changed by changing the value of registry setting: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\<Server Name>\<Private/Public-Guid>\Track Duplicates (in hours) The maximum value that the store will accept for this registry value is 49 days, if a value greater than 49 days is set, the store will ignore the value and keep duplicates for 24 hours. But keep in mind that increasing this value will cause this table to grow really large and this could slow down delivery. The store will periodically delete the old items from the deliveredTo table which is handled by the background cleanup thread which runs every hour. This is also configurable and the registry setting is: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\<Server Name>\<Private/Public-Guid>\Background Cleanup (in msecs). There is still a chance that you will get duplicate email if the delivery of email is delayed because of the following reasons: 1) If either the internet message id or the submit time is different on the two messages, the second message will not be treated as a duplicate. 2) If the two are the same, but time interval between the arrival of the two messages is greater than 1 hour, the store cleanup task would have deleted the original entry in the deliveredTo table and the user will get duplicates. 3) If the user is moved. The deliveredTo table is per database and the information is not moved when the user's mailbox is moved. 4) In older versions of Exchange we had a problem where duplicates would occur when a message was sent to a user and a DL containing the user using OWA. When the message was submitted the store would stamp an Internet Message Id on the outgoing message. However, since OWA submits messages with native MIME and the fact that the Internet Message Id stamped by the store on submission did not update the MIME Message Id header, the MAPI message was out of sync with the native MIME. The message would then be bifurcated by Transport and this would result in messages with different Internet Message Ids and therefore, duplicates. We changed this in Exchange 2003 so that the store only stamps the Internet Message Id on a message if it detects that the MIME has to be regenerated or if it is a pure MAPI message. - Jaya MatthewHow the M: Drive came about
In Exchange 2000, we introduced a new feature called IFS. IFS stands for “Installable File System”. This uses a little known and even less used feature of NT that allows the OS’s file system (like NTFS or FAT) to be replaced. The initial reason for doing that was as an optimization: it would allow protocols, such as NNTP and SMTP, to transfer the MIME messages directly as files. In Exchange 5.5, MIME messages are broken down into MAPI properties and stored in database tables. When they need to be accessed as MIME, they are put back together. In E2K, MIME messages are stored as MIME files in IFS and only converted into MAPI if a MAPI client (such as Outlook) accesses them. The other perceived benefit of IFS was that the Exchange storage objects could then be made visible through the file system. So you could go to a drive letter (M: was chosen for two reasons: first, “M” for “Mail”, and second, because it was in the middle of the alphabet and least likely to collide either with actual storage drives -which start at A and move up - or mapped network drives - which start at Z and move down), and get a list of mailboxes, navigate to mail folders via cmd or windows explorer and look at actual messages. This was considered pretty neat at the time and since it didn’t seem to be much more work to allow that access, it was thrown in (there may have been other, better reasons but I’m not aware of them). This ended up causing some challenges down the line related to the intricacies in how email objects need to be handled and mapping the file access behavior to them One of the biggest problems encountered was around security descriptors. This is difficult to explain without a detailed understanding of NT security descriptors, so I will simplify the explanation for the purpose of this discussion. The main part of an NTSD is called a DACL (discretionary access control list). It contains a list of users and groups and what they can do to that object. There are two main types of entries: allows, which say what an entity can do; and denies, which say what they can’t. The order of this list is very important. A standard sequence of entry types is called “canonical”. NT canonical form calls for a particular sequence. Because of legacy issues, MAPI canonical form requires a difference sequence of entry types. Applications that modify security expect a particular sequence and will behave erratically if the sequence is wrong. By creating or modifying objects through the M: drive, it will change the canonical format of the DACL’s and result in unexpected security behavior. This is bad. A related issue here has to do with item level security. E2K also introduced this feature, which is that items in a folder can be secured independently of each other and the folder. While this has some great uses, for many email systems this level of security is not needed. When a message has the folder default security, it simply references that property in the folder. When a message has its own security, there is an additional property that needs to be stored (this also has an affect on how folder aggregate properties, such as unread count, are computed). Having lots of individual security descriptors can result in both increased storage size and poor performance. When a message is created or modified through the M: drive, it will always get an individual security descriptor stamped on it, even if it is exactly the same as the folder default. This can also lead to unexpected behavior. For instance, if you change the default security on the folder, it will not change the security on any messages in it that have their own security descriptors. They have to be resecured individually. Another challenge is in relation to virus scanners. Virus scanners typically look for valid storage drives and spin through all the files on those drives and check them against virus signatures. The M: drive appears as a normal drive, so virus scanners were picking this up and processing it. This can have very detrimental affects on system performance and may also result in message corruption in some cases. Finally, IFS runs in kernel mode. This is a privileged execution path and it means that problems in this area can have much more severe affects (and be harder to track down) than in other areas of Exchange, which all run in user mode. Blue screens are one possibility if something goes wrong. IFS has given Exchange 2000 and Exchange 2003 a lot of advantages: we maintain content parity for MIME and make MIME message handling faster and more efficient as well as increasing the performance of such messages retrieved via internet protocols. But as I described above, there can be problems if IFS is misused via the M: drive. In Exchange 2003 we have disabled the M: drive by default to hopefully help reduce the likelihood that customers will encounter any of the issues described above. I encourage every system administrator to keep this disabled on E2K3 and disable it on all E2K servers as well. Jon AvnerAnnouncing the Jetstress 2013 Field Guide
Due to the success of the Jetstress 2010 field guide, we have decided to continue the tradition by releasing an updated version of the guide for Jetstress 2013 . As with the previous version, the aim of the document is as follows: Explain how Jetstress works. Provide storage validation planning and configuration guidance. Provide Jetstress results interpretation guidance So, what’s changed? Well, the good news is that Jetstress 2013 is very similar to Jetstress 2010. There are some modifications to accommodate the storage changes within Exchange Server 2013, however the planning, configuration and results interpretation process remain largely the same as they were in Jetstress 2010. Change overview in Jetstress 2013 The Event log is captured and logged to the test log. These events show up in the Jetstress UI as the test is progressing. Any errors are logged against the volume that they occurred. The final report shows the error counts per volume in a new sub-section. A single IO error anywhere will fail the test. In case of CRC errors (JET -1021), Jetstress will simulate the same behaviour as Exchange “page patching”. Detects -1018, -1019, -1021, -1022, -1119, hung IO, DbtimeTooNew, DbtimeTooOld. Threads, which generate IO, are now controlled at a global level. Instead of specifying Threads/DB, you now specify a global thread count, which works against all databases. Updates in the Jetstress 2013 Field Guide Not content with simply updating Jetstress, we have also added some more information into the field guide. Updated internals section to reflect changes made in Jetstress 2013 [4] Updated validation process flow charts [5.1] Improved failure mode testing section [5.4] Updated initialisation time table [5.6.1] Updated installation section [6] Updated report data section [9] Updated thread count section [Appendix A] The Jetstress Field Guide will be the only documentation released for Jetstress 2013, so if you have any feedback please feel free to share it with us here. You can download the new version of Jetstress field guide as an attachment of this post. Thanks, Neil Johnson Senior Consultant, MCS UKHow to test the disks on your Exchange server
If there's one thing that's true of all busy Exchange servers, it's that they generate massive amounts of disk I/O. There's a joke around here that Exchange is the world's biggest hard disk diagnostics program. Typically, your disks will be the first component of your Exchange server that starts groaning as you add load. And, frequently, you'll find that if you get your disks out of the redline area of the dial, that other performance issues suddenly heal themselves too. Why is this so? Exchange databases use transactional logging. As new data comes in, the most urgent priority is getting the new stuff secured on disk in a log file. If you are experiencing "log stalls," then everything else that needs to happen with that data must wait. This can lead to a cascade of other bottlenecks. There is a very good KB article on log stalls. The article tells you how to use System Monitor to tell if you have a log stall problem, and how to tune Exchange if necessary: XADM: Log Stalls/sec Are Regularly Greater than 0 (Zero) http://support.microsoft.com/?id=188676 But all the fine tuning in the world won't help if you are plain just demanding too much from your disk system. How can you tell what kind of load your disk system can really sustain? For years, the Exchange database test team here at Microsoft has used a homegrown tool called Jetstress to simulate heavy disk I/O loads. It can be downloaded from: http://www.microsoft.com/downloads/details.aspx?FamilyId=94B9810B-670E-433A-B5EF-B47054595E9C&displaylang=en You may have used LoadSim in the past. JetStress has some similarities, but is not a replacement for LoadSim. JetStress is a more sharply focused tool than LoadSim. It is intended only to simulate Exchange disk I/O activity. LoadSim lets you simulate network and client activity, and thus indirectly works out the disk system. JetStress goes right at the disk system, no indirection about it. You don't even have to have Exchange installed to use JetStress. You simply copy a few files to a server and start pounding it to its limits. JetStress generates a test database from scratch, of whatever size you want. Typically, to get valid results, you only need to generate a database that is 5% the size of your intended real database. You can then tell JetStress to make the same changes to the database that happen during normal operation. It adds, deletes, replaces and reads records from the database. By using System Monitor, you can see how much real Exchange load your disks can handle. You can change your disk configurations and re-run the same tests to see what kind of difference it makes. There are two basic kinds of testing you can do with JetStress: Performance Testing Disk Subsystem Stability Testing We usually recommend that you let JetStress run at least 2 hours when you're testing to see what kind of sustained throughput your disk system can handle. If you're doing stability testing, the recommendation is 24 hours. Now, what exactly do I mean by stability testing? Exchange can subject your server to very complex random I/O. As you push computer systems closer and closer to their tested limits, and as you run huge amounts of data through the system, you're more likely to encounter glitches and even bugs in the ability of the system to reliably process and preserve data. JetStress will let you load your system up till it's running as fast as it can, and will keep it under stress to see if it remains reliable in both storing and retrieving data. The way you can tell if the system is performing reliably is to look for error -1018 from your Exchange database. This error occurs whenever a page is read from the database, and the checksum on the page is wrong. Every page in an Exchange database is checksummed as it is written, and the checksum is verified every time the page is looked at again. If even a single bit is wrong on the page, Exchange declares the page bad and reports a -1018 error. You can learn more about Exchange page checksums and how we detect corruption in the database in this KB article: XADM: Understanding and Analyzing -1018, -1019 and -1022 Exchange Database Errors http://support.microsoft.com/?id=314917 If your database has -1018 pages after the stress test, then the disk system cannot be considered reliable at the load level under which it was tested. When you download the JetStress utility, you get excellent documentation along with it. The documentation will walk you through every phase, from setting up tests and monitoring their progress, to interpreting their results. It even tells you which System Monitor (Performance Monitor) counters to look at, and what values are OK. It also tells you how to validate the integrity of the database after a stability test. Mike LeePutting the Restrictions on Restrictions
One problem we encounter almost on a weekly basis is general performance problems. Most performance problems we have seen lately have been related to either inadequate disk configuration or MAPI restrictions. Mike Lee has already explained the bit about Disk issues in his entry, so I am going to tackle the other issue of MAPI restrictions. Let’s start with the basics. MAPI represents data typically in the form of a table. Everything is a table; there is a table that is presented to the client when it requests a list of providers, a table for folders, folder contents, attachments, etc. Each table is comprised of columns. Each column is a different MAPI property representing things like the sender, subject, and delivery time. Each row represents an individual item, so for a folder contents table, each row would represent messages. Now the client can do some interesting things with the tables, like resort them. The client can seek in the table to a specific row that matches criteria. This operation is referred to as a FindRow(). It can also request that only items fitting a certain criteria be included in the table. An example would be to only include items created on a specific day. This is what is known as a restriction. The resulting folder contents table would only have items in the table that meet the given criteria. Restrictions are used when it is expected the client will be requesting the same representation of data on a frequent basis. So now you maybe asking, “Why is that so bad?” Well to understand it we need to take a look at how the Store actually stores the data a MAPI client requests and how it interprets requests such as FindRow and Restrict. Inside the storage schema of the store we have various tables that collectively represent things such as mailboxes, folders, folder contents, and messages. This is what allows the store to do things like Single Instance Storage. When a client requests a list of the contents of a folder, that request is mapped to a special folder referred to as a MessageFolder Table (MsgFolder for short). Each folder created in the system has a separate message folder table. The purpose of the MsgFolder table is to map a folder to its contents. Keep in mind that for Single Instance Storage we have to allow multiple folders to keep a reference to a single message and multiple messages may appear in any given folders. In the database terminology this is known as a many-to-many relationship which requires an intermediate mapping table, the MsgFolder table (one for each folder). So to accommodate the client request to only be presented with messages from a specific date range, we could get a reference to the Folder’s MsgFolder table, then selectively remove unwanted rows by traversing the table in memory, but that would be very expensive operation. To handle the expectations of a Restrict call (frequent re-request for the same data) we create a new special folder (and corresponding MsgFolder table) that is referred to as a Restricted Search Folder. This folder is linked back to the original folder and logical relationship exists between these two folders. We place a condition on the search folder such that it should only include items that meet the criteria specified by the restriction. In this search folder, a backlink to the original row in the MsgFolder table exists for each message in the MsgFolder table that meets the criteria of the restriction. The performance issues that is encountered is the time it takes to manage the update to each of the search folders. When a change takes place on the original folder, the change is compared to each of the restricted search folders associated with the folder in question to determine if they need to be updated as well. This has a bigger impact when a lot of search folders exists for one or many folders. The second issue encountered is the creation of the restricted search folders. The creation of restricted search folder requires a full pass of the original folder to extract individual items that need to be linked into the restricted search folder. If this process occurs as a direct result of an action of the client, the time make experience a hang or receive the Outlook popup box indicating a request is taking a long time process. The time it takes to create the restricted search folder is proportional to the number of items in the regular folder. In Exchange 5.5 we did not put a limit on the number of the restricted search folders that we would allow, but the default timeout was 8 days. If the restricted search folder was not used in 8 days, it would be removed as a part of background store maintenance. The led to performance problems as lots of restricted search folders could be created all of which would need to be updated every time the original folder had any add/delete/changes/moves. In Exchange 2000/2003 we put a cap on the maximum number (11) of restricted search folder we would allow on a per folder basis, however the default lifetime was increased from 8 to 40 days. This has proven to be adequate for most, but from time to time we see the opposite problem occur. If a folder already has 11 restricted search folders associated with it and a new restrict request is made, the list of search folders is FIFO based using the last time the restriction was actually used. So this means the stalest restricted search folder is removed to make room for the new request. As mentioned above, this requires a full pass of the regular MsgFolder table and if done on the “Clients time” the client may perceive a performance issue while the table is being build for the client. So it is possible that on a daily or weekly basis more than 12 restricted search folders are used/created while the limit is 11. Eventually the client hits a restriction request that currently doesn’t have a matching search folder and results in the delete and creation of new restricted search folder on the client’s time. So how do you identify this situation and what can be done about them? Increase Diagnostic Level for MSExchangeIS Private Views or MSExchangeIS Public Views to minimum and look for events similar to the following to determine the frequency of restrictions being created for particular folders or users. Event ID: 1167 Source:MSExchangeIS Private / Public Type:Information Category:Views Description: created a new restricted view on folder . Observe the perfmon counter MSExchangeIS Private –or- Public \ Categorization Count. The is an overall number of restricted search folders + regular search folders in the system. Watch for sharp increases, especially after implementing any 3 rd party application that takes advantage of MAPI interfaces. In Exchange 5.5, dump the Private or Public information store using ISINTEG. This isn’t as productive in Exchange 2000/2003 since we limited the maximum number of restricted search folders. However you could use the information to correlate the event ids above to the folders possibly being affected. isinteg -pri|pub -dump -l logfilename Examine the log file and look for any folders with large numbers of entries under the following fields: Search FIDs= Recursive FIDs= Search Backlinks= Categ FIDs= For example: Search FIDs=0001-000000000418,0001-00000000041B,0001-000000000421, 0001-000000000423,0001-000000000424,0001-000000000428,0001-00000000042D Also take a look at the following KB articles: http://support.microsoft.com/default.aspx?scid=kb;en-us;216076 http://support.microsoft.com/default.aspx?scid=kb;en-us;328355 Hope this was helpful! Jeremy KellyThe history of content conversion in Exchange
The Exchange server and the client (that evolved into Outlook) were originally (circa 1992) based on the MAPI standard (stands for Messaging Application Programming Interface). Broadly this can be divided into the MAPI data model, and the MAPI object schema. Very simplistically, the MAPI data model can be summed as: message stores (think mailboxes) that contain folders that contain items that contain attachments. And all of the above entities are simply a collection of scalar properties (name, value pairs). The original MAPI schema laid out the common list of attributes applicable to email messages, for eg: Subject, Received Time etc. This has since been extensively extended by Outlook and Exchange for various advanced functionality. The Exchange server store was implemented to store the above data model in a very efficient manner, using a database technology called internally as Jet Blue. This data model, although simplistic in this day and age, has proved extremely flexible and surprisingly powerful to model many things that the original inventors could never have even imagined. And the Exchange server, due to its excellent implementation of this data model, has reaped great rewards. The primary mail transport protocol used for business e-mail back then was X.400. At about 1995 or so, the internet wave came along. The Web, the Browser, and HTML changed the world. Along with them the popularity of other standards based protocols for email increased, such as SMTP for mail exchange and MIME as the serialization format for email messages. Both of these rocked the ship of Exchange back then. The Exchange product had already been in development since '92 and more delays to complete it and ship it would have pretty much meant disbanding it altogether. The pressure from the executive level to ship it was intense. But folks in Exchange back then were wise enough to see that this internet wave was too big to just be a passing fancy and that these new standards must be embraced. As a compromise, support for SMTP as a mail exchange protocol (instead of just X.400) and MIME as a data interchange format (instead of just MAPI) were added to the product, but on the periphery so that the core product won't be affected much. Fortunately, the existing transport architecture had supported the notion of Gateways to connect to disparate other mail systems that existed then (again circa '92) who were foolish enough to not support X.400 natively! For example to connect with ccMail, GroupWise and the likes. We decided to lump this new kid on the block (SMTP) in the same category and modelled it simply as a gateway add-on to connect to this thing called the "internet"! This was called the Internet Mail Connector or simply the IMC. This IMC also then got the fun task of back and forth conversion between the data format stored native in the Exchange store (MAPI) and the data format that was the new internet standard (MIME). A conversion library was created for this purpose and was named, somewhat oddly now, as IMAIL (Internet Mail). This was the how we shipped Exchange 4.0 (March 13 1996). The next release, 5.0, was a very focused one in which we made very minor changes, although we did also add POP support. Both the IMC and IMAIL stayed pretty much the same as they were in 4.0. In the next release, 5.5, we decided to add support for IMAP4 as well. Now, POP and IMAP were similar to SMTP in insisting on MIME as the data interchange format. So it was decided to move this IMAIL conversion library closer to the actual data store (that was still a MAPI store of course) to improve performance. In addition, we invested significantly in this library to do a top notch job of translating to and from these 2 formats, and this has been quite a success overall. Exchange 5.5 was a very successful release thanks to the right balance of functionality and simplicity. The next release was Exchange 2000. By this time the domination of the internet protocols was so complete that we was decided to swap the roles of SMTP and X.400. That is, to make SMTP our primary mail exchange protocol even between Exchange servers, and treat X.400 as a one-off Gateway to reach pockets of the world that were still using it (mostly Europe). Also, we decided to go even further along the road of embracing MIME as the data format. We decided to actually enhance the core Exchange store to make it natively store and "understand" MIME. In this release we also invested in a new file system technology that would let us store these large "streams" of MIME content more efficiently. See Jon Avner's blog on the M: drive for more details on this one. In order to satisfy our richest client (Outlook) that continued to demand a MAPI data model, we had to engineer the Exchange store to do on-demand or deferred conversions back and forth between the 2 formats. As one can imagine, this was a very challenging piece of engineering, fraught with subtle traps. In the Exchange 2003 release we did not make any significant changes in this area, and status quo from the 2000 release remains. Naresh Sundararajan
