cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Exchange 2007 Search - Part 3: The Search Process
By
The_Exchange_Team
Published Oct 16 2009 01:44 AM 8,037 Views
The_Exchange_Team
Microsoft
‎Oct 16 2009 01:44 AM

Exchange 2007 Search - Part 3: The Search Process

‎Oct 16 2009 01:44 AM

This is the final part of the three part series on Exchange Search in Exchange 2007. See Part 1 and Part 2. In this post, we take a look at the search process in Exchange 2007.

Search Methods for Exchange 2007

There are five basic ways to search mailbox data.

Three types of search Methods can be using when querying data directly against an Exchange 2007 Sever.

    1. Exchange Search Exchange Search is the new content indexing feature that'ss used when a search request for mailbox data is received from an Outlook client in online mode and indexing is enabled on the Mailbox database.
    2. Store Search is the default search method that's used if content indexing is disabled on the Exchange server. This is also the method used in previous versions of Exchange. Outlook is in online mode and indexing is disabled.

This blog post focuses on the first option - Exchange Search with Outlook 2007 in online mode and Exchange 2007 indexing is enabled. The difference between searching the Index with and without WDS installed on the Outlook 2007 client machine is covered in detail later in this section.

  1. Outlook Online Search w. WDS - Outlook is online mode with WDS enabled to index the store.

    This method is disabled in WDS by default beginning with WDS version 3.01 and not recommended because it causes performance issues on the Exchange 2007 server. It can be enabled or disabled on (a) an individual client machine or (b) by group policy. The Administration Guide for 3.01 and the Group Policy for Windows Search provide detailed information, while the 4.0 Administrator's Guide provides general information:

    Note: The latest version of WDS is 4.0

    For further information on why this method is disabled by default beginning with WDS 3.01 and the performance problems it causes if enabled, please see this reference:

    905184  Exchange 2000 Server and Exchange Server 2003 performance may be affected when desktop search engine software is running on Outlook or other MAPI client computers

    http://support.microsoft.com/default.aspx?scid=kb;EN-US;905184

    Two additional methods of search are available with Outlook 2007.

  2. Outlook Cached Search - Outlook is in cached mode and Windows Desktop Search is not installed.
  3. Outlook Cached Search w. WDS - Outlook in cached mode and Windows Desktop Search is installed, the local index files are used.

The Instant Search feature, new to Outlook 2007 and which is only available if Windows Desktop Search is installed on the client machine, gives the user the following functionality:

  • Hit Highlighting, which helps the user understand why an item was returned with their search results by highlighting the matching text.

  • Word Wheeling, automatically filters the items in the current view based on what the user types in, and continues to update as the search is modified, this is sometimes referred to as "Search as I type".
  • Query Builder, allows the user to use query syntax, explained in further detail later in this section.
    All Folder Search, also examined in further detail later in this chapter this feature allows an Outlook user to search multiple folders in their Outlook profile, including multiple top-level folders.

Note: If you are accessing an Exchange 2007 Server with Outlook 2007 (without WDS) hit highlighting will still function.

Searching Mailbox Data - More Information

Locale and Language

In order to return consistent search results in environments of multiple locales, the following points must be true:

  • The locale of the message must match the language the email was written
  • Search queries are submitted in a single language
  • The language of the search query must match the locale of the client computer as identified by the connection to the server

Searching Junk and Dumpster Items

All your mailbox folders are available for search in Outlook 2007 and OWA 2007, including the junk mail folder.  It is not currently possible to search your mailbox dumpster from Outlook or OWA.  For a dumpster workaround, see the section below on Cross Mailbox Search.

Cross Mailbox Search

An Exchange Administrator has the ability using the Exchange Management Shell to search across multiple mailboxes using the Export-Mailbox cmdlet. This cmdlet will search all mailbox items in multiple, or a single mailbox. What is interesting about this cmdlet is that a search of a single mailbox or across mailboxes will include the dumpster of a mailbox or multiple mailboxes. Deleted items are converted to regular messages and included in the search. Items that match your search are then exported to a specified pst or specified single target mailbox. You can use Cross Mailbox Search to search for text in the Subject or Body of the message. For additional information on how to perform a Cross Mailbox search, see the links below:

Searching Attachments

With the use of installed IFilters Exchange Search indexes attachments. Depending on the client you are using to search, you may need to take additional steps to include attachments in the search.

  • In OWA 2007, attachments are automatically included in your search
  • In Outlook 2007, use Instant Search or Advanced Find
  • In Outlook 2003, use Find or Advanced Find and select Subject Field & Message Body
  • In Outlook 2002, use Advanced Find and select Subject Field & Message Body

Note that indexing is currently performed on embedded items for one level. If you have an email within an email, Exchange 2007 Search will index both items. If there is another email embedded in the second email, that email will not be indexed.

Another issue is that if you combine attachment keyword searching with another form of searching such as a keyword in an attachment (which gives hits) and a keyword in the TO: field (which gives hits), you will not get any hits if you combine these two parameters into one search because the attachment keywords and the TO: keywords are kept in different store tables. The only workaround currently is to do the searches separately and then compare the results for matching items.

Exchange Search vs. Store Search

When a user is in online mode in Outlook 2003 or Outlook 2007 and creates a search request within Outlook, the request is sent to the Exchange 2007 Server and serviced by one of two methods, Exchange Search or Exchange Store Search.  A table comparing the two methods is below, followed by an overview of the process of querying using Exchange Search.

ModeWDS InstalledNot Installed
  Advanced Find Find Pane Advanced Find Find Pane
Cached no no yes yes
Online yes yes yes yes

The query processor checks on the status of the msftesql-Exchange and MSExchangeSearch services every 30 seconds and caches their status in Memory; if either is unavailable Exchange Store Search is used.  Note that if the catalog is unavailable, or the database is not enabled for full text search, then Exchange Store Search is used.  Note that Substring searching refers specifically to a substring anywhere within a word rather than simply Prefix searching which refers to a substring at the beginning of a word.

Exchange Search Process without WDS Installed

The following Exchange Search procedure assumes the Outlook online mode client is using the Advanced Find feature, or does not have Windows Desktop Search (WDS)/Instant Search, or any other local indexing application installed.

A client (Outlook or Web Client) creates a search query, a search query contains the restriction and the scope, the "What" and "Where". 

    1. The Exchange Server Query Processor inside the store takes the query and builds a request based on the restriction and scope. The restriction is evaluated and converted to a MAPI restriction tree, providing the parameters of the query. The scope is the folder list and sets the boundaries of the query. 

           i. During this query evaluation (if a Noise Word File has been implemented), the query processes common words called Noise Words are removed. "A", "and", "the", are all examples of noise words.

    2. The query is now sent to MSSearch.

  1. MSSearch creates a temporary folder called a Search Folder that will contain its results. MSSearch then reads the Index Catalogs and returns the Document ID's from the catalog that match the request.

    The Search Folder is stored in a hidden folder hierarchy with a top level folder named Restrictions. These search folders are only temporary and the client does not directly access them.

  2. The Query Processor then reads the Property Store to find the Entry ID's that match the Document ID's returned by MSSearch, these back links are then stored in the Search Folder.

  3. A restricted view is then created based on the contents of the search folder. The information store then sends this view to the client. The client can now see all the items that match the restriction and scope of their query.

    1. The search folder is deleted by the store when not used for a certain amount of time.

Exchange Search Process with WDS Installed

In order to perform Advanced Searching in online mode (against the Index, not against the Store), Windows Desktop Search (WDS) must be installed. That can be illustrated by using an advanced search example of Phrase Searching.

In our example (Outlook 2007 SP1 in online mode) phrase searching using Instant Search fails against Exchange 2007 SP1 with indexing enabled against mailboxes if WDS is not installed. Phrase searches use quotation marks. For example, if you do a phrase search using Instant Search for "One Two Three" you receive the same results if you search for One Two Three (without "").  The result you receive is that One and Two and Three all exist in the email, whether they are in the subject or the title of the email - all three terms have to be there to get hits, but One Two Three does not have to be in exact order.
By design in Exchange 2007 with Indexing enabled, Phrase Searching (i.e., using quotation marks to search for a phrase in the subject or body of a message) using Instant Search in Outlook 2007 online mode only works if Windows Desktop Search (WDS) is installed. This is by design in Outlook 2007 because the Semantic Analyzer Component of Windows Desktop Search (WDS) is required by Instant Search for advanced searching syntax (also called advanced search grammar or advanced searching language). This type of searching is officially known as Advanced Query Search or AQS that is only added to Instant Search from WDS.

Phrase and Keyword Searching
If you want to do phrase searching (using "") with Outlook 2007 Instant search in online mode with Exchange 2007, you must install Windows Desktop Search to go along with Outlook 2007. In summary, this is by design in Exchange 2007 and by design in Outlook 2007. This applies to all types of advanced queries against the Exchange 2007 index - WDS must be installed on the Outlook 2007 client in online mode.
When you install Outlook 2007, Outlook 2007 shows a prompt to have WDS installed
because Outlook 2007 would like to make use of the semantic analyzer component in WDS. Once WDS is installed, WDS enables Instant Search to perform AQS queries such as phrase searching (using ""), keyword searching and more. Online mode search works without WDS, but the AQS or Advanced Query Syntax such as using quotes for searching does not work. AQS is the query language for WDS and can be used to narrow the scope of your searches. For example, using AQS you can use keywords like sent:<date>, from:<person>, read:yes/no, NOT, AND, +, OR, to name a few.

See the following resources for a complete list of AQS search options:

Outlook 2007 without WDS installed will not support the Advance Query Syntax. This functionality is only provided with Windows Desktop Search.

Outlook 2007 with WDS installed supports Prefix and Exact Phrase Matching when double quotes are used around any word or words to specify an Exact Phase. 
Windows XP does not have Windows Desktop Search (WDS) installed by default. If you want to use the Advanced Query Search functions in Instant Search in Outlook 2007 online mode such as searching for phrases by using quotation marks, you will need to include Windows Desktop Search as a part of the Outlook 2007 install or add WDS after installing Outlook 2007.

WDS is installed by default on Windows Vista.  Therefore, Outlook 2007 installed on Windows Vista, in online mode, is able to utilize the Advanced Query Search functions by default.

OWA uses Basic Search instead of Advanced Search Query. Therefore, when using OWA to search the Exchange 2007 index on the server, it is by design that phrase searching and other advanced queries do not work. 

Prefix matching is the default for the Advanced Find feature in earlier versions of Outlook but this is not necessarily the case in Outlook 2007. The table below lists the different modes and searching features in Outlook 2007 and whether or not Prefix searching is supported. Note that the table below refers to Prefix matching specifically (the beginning of a word), rather than Substring searching (a string found anywhere in word). Note also that when Indexing is enabled and available and the mailbox has been crawled, Advanced Find in Online Mode uses Exchange Search when it searches for a property in a message that is indexed by Content Indexing, uses Store Search when it searches for a property in a message that is indexed by Content Indexing, and uses a combination of Exchange Search and Store Search for a complex search of a combination of properties in a message, one of which is indexed by Content Indexing and one of which is not indexed by Content Indexing.

ModeWDS InstalledWDS Not Installed
  Advanced Find Find Pane Advanced Find Find Pane
Cached no no yes yes
Online yes yes yes yes

Support Matrix for Prefix Matching

Further Information on WDS and Outlook 2007

Outlook 2007 is the first client version providing Instant Search in a pane which integrates Windows Desktop Search (WDS) into the Outlook interface. Instant Search makes it possible to easily search all mail folders in online mode and all personal folder files at the same time. Instant Search works in online mode against Exchange 2007 mailboxes. Instant Search also works in online mode against Exchange 2003 and Exchange 2000 mailboxes. For further information and to download WDS, see the following link:

Windows Desktop Search 4.0

http://www.microsoft.com/windows/products/winfamily/desktopsearch/choose/windowssearch4.mspx

Outlook 2007 with WDS installed allows you to return results for the selected folder (only) or All Folders. There is not an option for subfolder searching. However, this can always be performed through the Advanced Find feature.

Note that in previous versions of Outlook, WDS is not integrated into the Outlook interface. In Outlook 2003 and Outlook 2002, the Advanced Find dialog only allows you to search one mailbox folder plus its subfolders. You can choose to search the Mailbox top level folder and its subfolders and you can then search all items in the Mailbox in online mode. Or you can search one personal folder plus its subfolders at a time. However, you cannot search multiple top level folders or multiple .pst files (or a combination of folders and .pst files) at the same time. In Outlook 2003 and Outlook 2002, you must use the WDS interface by itself to be able to search all items in the mailbox and all items in personal folders at the same time.

Searching Exchange 2007 Public Folder Databases with Outlook 2007

Using Instant Search in Outlook 2007 or Advanced Find in Outlook 2007 or Outlook 2003, you can search Public folders only in cached mode and only your Public Folder Favorites and only when you check the Cached Exchange Mode Settings box which says Download Public Folder Favorites. You can also optionally mandate local cached indexing of public folder data on a per machine basis or by group policy. 

Indexing Exchange 2007 Public Folder Databases

Note that Exchange 2007 indexes only mailbox store databases. In Exchange 2000 and Exchange 2003, full-text indexing could also be implemented for the public store database. That functionality has been replaced with SharePoint 2007 SP1 server running on Windows 2003 to index Exchange Server 2007 SP1 public folders for Exchange 2007 SP1. For further information, see the following links:

942390 Description of the SharePoint Server 2007 issues that are fixed by the 2007 Microsoft Office servers Service Pack 1
http://support.microsoft.com/default.aspx?scid=kb;EN-US;942390

If the SharePoint 2007 server is on Windows 2008 64-bit, there is an issue that is fixed in SharePoint 2007 SP2 which will allow SharePoint 2007 on Windows 2008 to be able to index Exchange 2007 SP1 public folders. Note that it is required that you apply BOTH Windows SharePoint Services 3.0 Service Pack 2 and Microsoft Office SharePoint Server 2007 Service Pack 2 (64-bit versions):

KB Article Links

Downloads

Summary

In summary, there are five basic ways to search Exchange 2007 mailbox data. This blog post focused on the first option - Exchange Search with Outlook 2007 in online mode with Exchange 2007 indexing enabled. Searching the Index with WDS installed on the Outlook 2007 client machine provides a new feature called Instant Search and includes the ability to do Advanced Query Searching. Also, using Instant Search that has WDS integrated into the Outlook 2007 interface allows you to search all mailbox folders in online mode and all .pst files simultaneously in a supported and reliable manner.  For performance reasons, indexing the mailbox store by WDS is disabled by default.

Search queries are performed in one language only, the language of the client performing the search. Cross Mailbox Search to a target mailbox or target .pst file is available through the Export-Mailbox command. Attachment searching, new to Exchange 2007, is only provided with the use of installed default Filters and additional Filters may be installed for other document types. In order to do attachment searching, you must be in online mode or use OWA. Instant Search automatically includes attachments and Advanced Find can search attachments in Outlook 2007, Outlook 2003, and Outlook 2002. OWA 2007 includes attachment searching automatically.

Exchange Search (default) is preferred over Store Search (only used when the Index Catalog is not present or not available). Exchange Search is much faster than Store Search, searches words and phrases instead of streams of bytes, searches attachments, and can search in various languages.

Searching Exchange 2007 Public Folder Databases with Outlook 2007 (and Outlook 2003) is only possible if you are in Cached Exchange Mode, and only on Public Folder Favorites, and only if you check the box which says Download Public Folder Favorites.

By design, Exchange 2007 only indexes mailbox store databases, while both previous versions Exchange 2003 and 2000 also indexed public folder databases. By design, the current method of indexing Exchange 2007 public folder databases is to use SharePoint 2007.

In Exchange 2007 SP1, SharePoint 2007 SP1 is used to index public folder databases. SharePoint 2007 SP1 servers can currently index public folder databases on Exchange 2007 SP1 servers. SharePoint 2007 SP2 servers running on Windows 2008 are expected to be able to index Exchange 2007 SP1 public folder databases.

Conclusions

Exchange 2007 full-text indexing is now a best practice for all Exchange 2007 mailbox databases due to the fact that searching is much faster, mail is immediately indexed when it arrives, the reduced catalog size, and both Search and Index are much less resource intensive than previous versions.

Crawling in Exchange 2007 is now performed automatically as new items arrive or a significant event such as a mailbox move to a database occurs rather than on a schedule as in previous versions. This means that the Catalog Index is always up-to-date within seconds of new items arrival. Exchange 2007 provides a Noise Word file capability per the language of the client.

Exchange 2007 Content Indexing for High Availability SCC servers is much like Clustered Servers content indexing for Exchange 2003 and Exchange 2000: there is one copy of the Catalog. For Exchange 2007 CCR servers, there are two copies of the mailbox database and each database has its own unique catalog which is only crawled at any one time on the active node.

There are five basic ways to search Exchange 2007 mailboxes, and this bulletin concentrates specifically on the new Features available for Outlook 2007 online mode clients with WDS installed against Exchange 2007 mailbox database with indexing enabled such as searching attachments and using Advanced Query Syntax. Indexing of Exchange 2007 public folder databases has been completely deprecated for Exchange 2007 and searching for Exchange 2007 public folder databases has been partially deprecated in Outlook 2007. The current method of indexing and searching for Exchange 2007 public folder databases is now using SharePoint 2007.

In conclusion, deploying Exchange 2007 together with Outlook 2007 and SharePoint 2007 gives advanced features, much faster and up-to-date indexing and searching than previous versions of full-text indexing for Exchange 2003 and Exchange 2000. Full-text indexing and the use of Exchange Search rather than Store Search are now recommended as a Best Practice for all Exchange 2007 mailbox databases.

Exchange 2007 Search Resources and Links

Technet

Microsoft Exchange Team Blog and Team Wiki

KB Articles

  • 944516  How to register Filter Pack IFilters with Exchange Server 2007
  • 945077 The Outlook Web Access search function does not work for some users in Exchange 2007

Office Online

Advanced Query Syntax

Windows Desktop Search

Bob Want and Jack French

0 Likes
Like
8 Comments
Not applicable
‎Oct 17 2009 09:20 PM
‎Oct 17 2009 09:20 PM
Hi Jack,

99% of our outlook users are running outlook 2007 within terminal services, so cached mode is not an option for us.  Are there plans with ex2010 and outlook 2010 to improve the search options for online mode?  many of my users are unhappy that they cannot use the more advanced outlook search...

thanks,
Wes
0 Likes
Like
Not applicable
‎Oct 18 2009 04:24 PM
‎Oct 18 2009 04:24 PM
@pesos:
Lookeen, a search tool for Outlook can search in cached and unchached mode in Outlook (with TS, Exchange, ..).
There are also features added like the shared index...
0 Likes
Like
Not applicable
‎Oct 19 2009 08:19 PM
‎Oct 19 2009 08:19 PM
Hi Wes,

In order to use the Advanced Options with Outlook 2007, you must install Windows Search 4.0 on the Terminal Server where Outlook 2007 is installed. Then you will have access to the Advanced Query Syntax within the Outlook 2007 client when you do your searches.

Here is the link to download Windows Search 4.0:

Windows Search 4.0
http://www.microsoft.com/windows/products/winfamily/desktopsearch/choose/windowssearch4.mspx?tab=Ins...

Exchange 2010 has just released and it includes several new enhancements to online search, including a much better cross mailbox search and better attachment handling. We will cover Exchange 2010 Search in a future blog series.

Outlook 2010 is planned for release later on, and will take advantage of new search syntax and new enhancements in Exchange 2010 search.

Thanks Wes,

Bob
0 Likes
Like
Not applicable
‎Oct 19 2009 09:15 PM
‎Oct 19 2009 09:15 PM
So if we install Windows Search but essential disable it from indexing anything people still get the better search inside Outlook 2007? Any hints on how to make Windows Search the least intrusive, and to not index anything?
0 Likes
Like
Not applicable
‎Oct 28 2009 02:26 AM
‎Oct 28 2009 02:26 AM
When Windows Search 4 is installed but not indexing Outlook, Outlook displays the "Search results may be incomplete because items are still being indexed. Click here for more details." bar, and when I click it says "471 items remaining across all open mailboxes." is that information coming from Exchange 2007, or is Windows Search 4 making that up from somewhere?
0 Likes
Like
Not applicable
‎Dec 13 2009 07:59 PM
‎Dec 13 2009 07:59 PM
When you rebuild the full-text index, while the information is recalled, is it normal to see the following error:

“Search results may take a long time to appear because Microsoft Exchange Search is unavailable.  Results will not include matches in the e-mail body.”

The mailbox store is rather large and I expect the rebuild to take approx 30 hours.  I just want to verify that the error will go away after the indexing is complete.  Thanks!
0 Likes
Like
Not applicable
‎Dec 21 2009 05:41 PM
‎Dec 21 2009 05:41 PM
Billy - yes this is normal, while the index is being rebuilt you are using Store Search which is slower, and why you are seeing that warning, which will go away once the rebuild is done.
0 Likes
Like
Not applicable
‎Dec 21 2009 05:43 PM
‎Dec 21 2009 05:43 PM
Michael - If you install Windows Search but do not index then you will not be able to use the advanced search logic in Outlook. So if you do want to take advantage of those features you will need to allow Windows Search to index your data.
0 Likes
Like
Version history
Last update:
‎Oct 16 2009 01:44 AM
Updated by: