Indexing in Active Directory

First published on CloudBlogs on Apr, 01 2008

We end up taking indexing for granted most of the time until and unless we encounter a problem.  Why don’t we start off with the question of “what is indexing?”.

So an index is essentially a way of organizing data for fast retrieval or at least retrieving a quick answer.  For a thorough understanding of how searches against AD work you can look to this How Active Directory Searches Work detailed article as a great backgrounder.  For general information on how indexed attributes work look here .

A deeper understanding of how Active Directory works as a directory can help understanding.  The basic idea to know is that Active Directory is at its core an Extensible Storage Engine database.  We won’t be going into the details on the different types of indices in this blog post, instead the goal is to give an understanding of what indices in AD do for you, how to see when things may not be going well with an index, or when you may need an index and don’t have one.

A simple example for our purposes would be that my database contains about 10,000 users, of which 63 have the first name Bob and last name of something else.  If I need to find all of the Bobs I have I would normally need to search each of the 10,000 user object’s name attribute to compile my Bob List.  That’s a pretty resource expensive operation for such small gain.  What if I have to do that very frequently and concurrently for some reason, like someone’s home grown application which does that for its own reasons?

The above scenario is just an example but the idea is that if you had a list of all of the Bobs that you could refer to that would be a much less resource intensive operation.  Particularly if you just needed to find one Bob in particular from that list.   That precompiled Bob List is the basic idea behind having an index: find your Bob or Bobs quicker (or whatever it is you search frequently on).  There are more complicated indexing scenarios and real world examples but I don’t want to broaden the topic past what we need to know to administer our directories.

Active Directory has some indices that are built in which are there to aid on commonly queried attributes in a default Active Directory environment.  These are there for things that we know will speed up the performance of the AD when answering queries from clients and we know that are likely to be queried for.  We have an older published list of these here .  This list has not been updated for Server 2003 or Server 2008 AD.

The times when indexing more commonly becomes a topic for an administrator is when domain controllers seem to be taking longer to answer specific LDAP queries or general LDAP queries for that matter, or when someone is discussing Linked Value Replication first being enabled in an environment.

How to Tell If You Should Index

So, how do you tell if you indexing an attribute may help speed up the performance of your AD in replying to queries?  There are several ways.

One more general method to tell if your DC is busy as a result of LDAP queries is Server Performance Advisor’s AD test.  I won’t go into great detail here except to point you to my prior blog post on that.

Next, if your LDAP query seems to be taking longer than you expect it to or would want it to you can find out whether the indexing may help by getting some diagnostic data from the domain controller being queried against.  This is a pretty easy thing to enable-simply set the registry values below and then retry your query and you will see detailed events regarding the LDAP query in your Directory Services event log.  These events will actually give you an assessment on whether your query is deemed expensive or not.  If your query is expensive and an attribute or attributes being queried for are not indexed then this may suggest that you go ahead and create an index for them.

To add these events edit your DCs registry like this.  If the values below are missing then add them.  A reboot of the domain controller is not necessary:

HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesNTDSDiagnostics

15 Field Engineering =5

(the default is 0 so we want Field Engineering at 5)

HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesNTDSParameters

Expensive Search Results Threshold

Inefficient Search Results Threshold

The defaults are 10,000 and 1000 respectively

What you should look for in your Directory Services event log on the DC you alter the registry on are NTDS General Events, category Field Engineering, event ID number 1614.  In the description area of the event you’ll see the LDAP query received as well as the number of database entries which were looked at to get the answer.  The general idea is that if you see 10,000 entries for a particular attribute examined then that is expensive.  Keep in mind that the “expensive” determination is subjective-if you query 5,000 very frequently and it’s affecting performance of your DC then you would want to consider 5,000 your actual threshold.  Also keep in mind that sometimes a query is expensive simply as a result of the query taking place against the “wrong” container-like RootDSE, and that narrowing the query to a particular OU when possible could save a lot of hassle.

How to Actually Index an Attribute

**This should only be done after thorough non-production testing**

1.       On your schema master go to StartàRun and type in “regsvr32 schmmgmt.dll” (without the quotes), then press enter.

2.       Open MMC and add the Active Directory Schema snapin.

3.       Click Start , point to Programs , point to Administrative Tools , and then click Active Directory Schema Console .

4.       In the console tree, click Attributes .

5.       In the details pane, right-click the attribute that you want to index, and then click Properties .

6.       Click Index this attribute in the Active Directory.

How to Tell If an Attribute Is Already Indexed

There are several ways to check to see if an attribute is already indexed or not.  I would first suggest searching MSDN for the default schema definition for that attribute.  A comprehensive list is here , but you can simply use your favorite search engine to find them just as well.

An alternate method, one that is more definitive for your individual environment, would be to look at that schema attribute using LDP.EXE.  The basic idea is that if the searchFlags value is 1 then the attribute will be indexed, like below:

To look on a domain controller and quickly see whether the adding/creating of an index for an attribute has occurred the Directory Service events 1464 and 1137 can be used,  and for seeing  the removal of an index DS event 1185 can be used.

Other Considerations

A few more things to keep in mind when considering whether to index.  First, whatever index you create must be replicated throughout your domain or forest depending.  Really what this means is not that the index itself is replicated as an entity but rather the attribute in the schema is set to index and then the different replica domain controllers take that action to index things then they will do so.

Why is that a consideration?  Because creating an index can be an expensive operation on the domain controllers as an initial task to create an index requires a thorough search through the database.  This taking place on a domain controller could impact other services as resources are directed toward compiling the index for the first time, so the general guidance is 'don’t index during your busiest time and don’t index unless you need to'.  Among other things, AD replication could be delayed significantly as the index is created.

Also keep in mind that an index can actually make your AD database larger, depending on the type and scope of the attributes being indexed, so don’t be terribly surprised if your nice compact AD ntds.dit gains a few pounds (or kilos for you metric system folks out there) after creating an index or indices.  Those out there trying to save some money on hardware and not migrating off of or upgrading your DCs (you know who you are) particularly need to keep that in mind.

Also, please be very cautious about altering your schema, regardless of how innocuous that alteration may seem, without planning and testing first in a non-production environment.  A mistake updating your schema may result in hours or days of extra work for you….and that’s in the “good” case.

One other point to make here:  if the attribute(s) which are showing as taking an extended amount of time to search for are already indexed then the lack of an index is clearly not your problem.  Sometimes indices, perhaps through frequent changes or other reasons, need to be re-indexed to remove "whitespace" or other problems.  So checking the integrity of the database or doing an offline defrag may be the way to go.

In other instances the overall query being passed is simply too complex altogether to rely on fixing the issue on the domain controller (directory) end.  In those cases the query itself needs to be revised, altered, made simpler so as not to over tax the directory.  There's always a balance to be had and human judgment will enter to discern what that balance is.

Finally, for those that are in the midst of writing an application which is intended to query Active Directory here is a very good article on MSDN which can help you.

Hopefully this blog post gives everyone a nice springboard from which to start asking the right questions regarding Active Directory searches and performance so that you can make the right decisions when you need to.  We haven’t covered every aspect of searches and indexing in AD but we’ve gotten a good idea across here to get you started.

Thanks to Yassine Khelifi for this great topic suggestion.