First published on TechNet on Feb 09, 2014
Hi, my name is Tom Ausburne and I am a Premier Field Engineer specializing in Active Directory. I recently had a customer ask if Microsoft had any tools to do Active Directory “stress” testing. A simple search leads you to this tool.
Active Directory Performance Testing Tool (ADTest.exe)
http://www.microsoft.com/en-us/download/details.aspx?id=15275
Now you are thinking that this tool is perfect until you get to the System Requirements section.
- Supported Operating System
Windows 2000, Windows Server 2003
Now that’s a bit old isn’t it? Windows Server 2003 R2 ended its Mainstream Support on 7/10/2010 and ends its Extended Support on 7/14/2015. Most companies have either moved past these or are in the process of doing so. What about Windows Server 2008, Windows Server 2008 R2, Windows Server 2012 and Windows Server 2012 R2? Will this tool run in these environments?
Glad you asked. The ADTest Tool still works fine on the newer versions of Active Directory. The tool uses LDAP so compatibility isn’t an issue. So everything is great, right? Not so fast. Yes the tool works but setting it up by following the documentation proved to be less than straightforward. While searching for answers I also ran across this whitepaper that not only discusses using ADTest but also supplies a rewritten configuration file for 64 bit OS’s.
Active Directory Performance for 64-bit Versions of Windows Server 2003
http://www.microsoft.com/en-us/download/details.aspx?id=4948
Some of what’s here can be found in that article but this is more of a straightforward let’s get this done and on to testing kind of post. I’m going to assume that you know how to install and configure Active Directory so I won’t go through those steps. Let’s run though the setup so you can start testing that new hardware with Server 2012 R2 AD-DS. In my tests I was using Windows Server 2012 R2 with Windows 7 & 8 clients.
Before we get started let me also note that this tool is no longer supported by Microsoft . This article is just to show that you can still get it to work and use it on our newer operating systems.
I also need to remind you that since this tool makes changes to Active Directory, and puts a load on the domain controller, it should be used in a TEST environment only . No need getting a bunch of help desk calls about people not being able to log in because the domain controllers are too busy! Now that we have that out of the way let’s get started.
Server Setup
1. Install Active Directory Domain Services (AD-DS).
2. Set dSHeuristics bit so that the userPassword attribute is treated like a password and not a string attribute.
a. Click Start , click Run, type adsiedit.msc , and then click OK .
b. Double-click Configuration, CN=Configuration, CN=Services, CN=WindowsNT, CN=Directory Service .
c. Right-click CN=Directory Service , and then click Properties .
d. Click dSHeuristics .
e. Click Edit .
f. Set the 9th bit to 1 . (For example, change the value to 000000001).
g. Click OK .
h. Click Apply .
i. Click OK .
3. You need to increase the number of user connections that are allowed to the server. The MaxUserPort value controls the maximum port number used when an application requests any available user port from the system.
a. Launch PowerShell (or you can use a CMD window but you really need to get used to PowerShell).
b. Type netsh int ipv4 set dynamicport tcp start=1025 num=64511
You might be asking yourself what about modifying the TcpWindowSize. Well that registry keyword from Windows Server 2003 is no longer supported, and is ignored in Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008.
For network adapters that allow manual configuration of resources, such as receive and send buffers, you should increase the allocated resources. Some network adapters set their receive buffers low to conserve allocated memory from the host. The low value results in dropped packets and decreased performance. Therefore, for receive-intensive scenarios, we recommend that you increase the receive buffer value to the maximum.
Other than that we suggest you leave everything to their defaults. The operating system does a very good job of tuning itself for best performance.
4. Create an ADTest User. In my examples I created a user with the name of perftest. This account is used to create OUs, Users, Groups and to run the stress tests against Active Directory. Make this user a member of Domain Admins . By default the password used in the x64Performance.ats script is ss-123456. If you use a different password (as I did) then make sure to modify this file. The contents of the x64Performance.ats file can be found at the end of this article.
Client Setup
1. You need to increase the number of user connections that are allowed. The MaxUserPort value controls the maximum port number used when an application requests any available user port from the system.
a. Launch PowerShell (or you can use a CMD window but you really need to get used to PowerShell).
b. Type netsh int ipv4 set dynamicport tcp start=1025 num=64511
2. Install the ADTest Tool on each client. You can get it here:
Active Directory Performance Testing Tool (ADTest.exe)
http://www.microsoft.com/en-us/download/details.aspx?id=15275
All of the screenshots in this article assume the install was done to c:\ADTest.
3. Copy the x64Performance.ats file to each client you are using in the test and place it in the ADTest installation folder. You can get that file at the end of this article.
4. Join the clients to the domain. Make sure the ADTest user you created is a member of the local Administrators group on each machine.
Set up the Test Environment
Before I get into the setup of the environment I know some of you will ask why we aren’t using PowerShell to create all of this. That’s a very good question. You can use PowerShell to create your OUs, Groups and Users if you either have a script or want to write one. You would need to create these using the same names seen below “OR” modify the configuration file to reflect the structure you create. For me, I just figured why create extra work for myself when this tool will create everything in a few minutes so I just used the built-in functionality.
Let’s get started. Create an organizational unit named BaseOU at the root of the domain. Now create an organizational unit named Groups underneath that.
The commands to automatically create the OU structure, users, groups and adding users to groups are fairly straightforward. I found that everything ran much smoother if I entered the username and password that we created earlier as part of each command.
To create 10 OUs 3 levels deep use this command:
adtest -r NewRoot -f x64Performance.ats -user perftest -password perftest -root 0 -t 10 -sf -e -o newroot.log
Now we need to add some users. This command will add 1000 users to each team OU.
adtest -r AddUser -f x64Performance.ats -user perftest -password perftest -root 0 -t 10 -sf -e -o adduser.log
We need to add some global security groups. This command will add 20 groups.
adtest -r AddGlobalSecurityGroup -f x64Performance.ats -user perftest -password perftest -root 0 -t 1 -sf -e -o addgroups.log
The last thing to do is to add users to the security groups. You can change the GROUP=1 to reflect the group you want to add users to.
adtest -r AddMembers -f x64Performance.ats -user perftest -password perftest -root 0 -t 1 -sf -e -set GROUP=1 -o addmembers1.log
I went through this set up quite a few times and ended up just creating a cmd file with all of the commands. You’ll notice I ran the AddMembers command several times to add users into various groups. Once you get the commands just the way you like them this is a great method to get things set up quickly.
Running the tests
So now you are all set up and ready to run the tests. You will run this command on each client you are testing from:
adtest -r %1 -f x64Performance.ats -user perftest -password perftest -root %2 -t %3 -sf -e
Where:
- -r specifies the name of the test (%1). For example, Search_Base_1Attr or update_1attr are possible test names.
- -f specifies a custom file (the x64Performance.ats script that was provided) instead of the default adTest.ats.
- -user and –password identify a member of the Administrators group on each server.
- -root specifies where in the OU structure to begin (%2). For example, 0 starts at ou_0000.
- -t specifies number of threads concurrently running to generate load (%3). Typically this number ranges from 1-3, going as high as 5 or 6 for faster operations, such as searches and fast binds.
- -sf shows the output of the tests.
- -e encrypts the instructions.
Change the values for these parameters accordingly for each test. Here is an example of a test of NTLM Logons against the first OU running 6 threads:
adtest -r NTLM_Logon -f x64Performance.ats -user perftest -password perftest -root 0 -t 6 -sf –e
The following two tests put a heavier load on the DC by updating and searching for 10 attributes at a time:
adtest -r Update_10Attr -f x64Performance.ats -user perftest -password perftest -root 0 -t 6 -sf -e
adtest -r Search_Base_10Attr -f x64Performance.ats -user perftest -password perftest -root 0 -t 6 -sf -e
You can get a list of tests that are available by looking in the x64Performance.ats file. If you want to see some real testing results using this tool, check out the document (also referenced earlier in this post) to see performance data for 3,000,000 users. Just remember that this was using Windows Server 2003.
Active Directory Performance for 64-bit Versions of Windows Server 2003
http://www.microsoft.com/en-us/download/details.aspx?id=4948
There are several ways you can measure the performance of Active Directory and new server hardware. One of which is to use the built-in Performance Monitor. You can start the Active Directory Diagnostics to gather some very useful data during your setup and testing.
This brings up another good point. It’s always a good idea to have a baseline of performance data for your production network. This is a good way to get that information. Just run the tool during a normal workday, say from 7:00am to 10:00am on a Monday morning so you can compare the information when things aren’t running so well.
I didn’t mean to jump off topic but you will find this information very useful. After you run the collector set you can see the results under Reports.
Troubleshooting
If you run into any issues you can add debugging output to any command. Just add a –d 5 and run the command again.
EX: adtest -r NTLM_Logon -f x64Performance.ats -user perftest -password perftest -root 0 -t 6 -sf –e –d 5
In this case I saw this error in the output
----------------------------------------
SYSTEM ERROR
Error: 1326
System Message: The user name or password is incorrect.
Extended Message: Logon Failed for User="u01_000713", Password="password123!",Domain="Hay-Buv"
----------------------------------------
Looking back at the x64Performance.ats file I looked at the section for the test I was running.
Then I looked at the password I had set when I created the users. Capitalization is important!
Once I changed the password to Password123! Everything ran fine. Most of the problems I had in getting this set up were resolved in the debugging.
Sometimes you will get failures because of the values you set. For example if the sum of root (%2) + number of threads (%3) is greater than the highest numbered OU. In the previous case, if root is 0 and number of threads is 11, an error is returned because there is no OU 10.
If you are like me you might be tempted to just skim over the setup instructions and get right to the testing because you’ve done this kind of thing a million times. If so you might run into a couple of issues.
- If you get an errors when you run the first command, and you enable debugging, you might see something like this:
LDAP ERROR -----------------------------
Host: ADTestSrv. Hay-Buv.local
Extended Message: 00000005: SecErr: DSID-031521E1, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0-----------------------------------------------
This is a simple one. You forgot to make your test user a member of Domain Admins .
- Now it should be clear sailing except none of the users are being created. Adding the –d 5 (debugging) command shows this type of error:
LDAP ERROR -----------------------------
Host: ADTestSrv.Hay-Buv.local
Extended Message: 0000052D: SvcErr: DSID-031A129B, problem 5003 (WILL_NOT_PERFORM), data 0----------------------------------------------
Also a quick fix. You need to set the 9th bit for dSHeuristics value to 1 .
Now this wasn’t that hard was it? Most of the time we do a great job at giving people very detailed instructions and for many that works just fine. But some of us (cough, cough) would rather see a list of “do these things in this order” and be done with it. Welcome to my world! Maybe I’ll have to start doing more of these and call it Concise Computing Tips .
X64Performance.ats
Use Notepad and paste the following code into a new file. Save it as x64Performance.ats and place it in the same folder where you install ADTest .
//---------------- DEFINES ------------------------//
// [ROOT] is specified on the command line (e.g. -r 32 for [ROOT] = 32)
//Number of users in an organization unit
#define $DefaultRange #(0-299999)
//Users are formatted u0001_0001
#define $Padding ######
#define $UserName u##([ROOT])_$Padding(*)
//Organization Unit Hierarchy
#define $Division ou##([ROOT])_division
#define $Unit ou##([ROOT])_unit
#define $Team ou##([ROOT])_team
#define $DivisionBranch ou=$Division,ou=BaseOU,[DOMAIN]
#define $UnitBranch ou=$Unit,ou=$Division,ou=BaseOU,[DOMAIN]
#define $TeamBranch ou=$Team,ou=$Unit,ou=$Division,ou=BaseOU,[DOMAIN]
#define $FQDN cn=$UserName,ou=$Team,ou=$Unit,ou=$Division,ou=BaseOU,[DOMAIN]
#define $BaseOU ou=BaseOU,[DOMAIN]
#define $GroupOU ou=Groups,ou=BaseOU,[DOMAIN]
//----------------- TESTS ------------------------//
//----------------- AUTHENTICATIONS -------------//
NTLM_Logon
{
TEST [LOGON]
LOOP RAND
RANGE $DefaultRange
DN $UserName
PWD ss-123456!T
OP LOGON32_LOGON_NETWORK
SCOPE LOGON32_PROVIDER_WINNT40
}
Kerberos_Logon
{
TEST [LOGON]
LOOP RAND
RANGE $DefaultRange
DN $UserName
PWD ss-123456!T
OP LOGON32_LOGON_INTERACTIVE
SCOPE LOGON32_PROVIDER_DEFAULT
}
//-------------------- BINDS --------------------------//
Fast_Bind
{
INIT FASTBIND_ON
TEST [BIND]
LOOP RAND
RANGE $DefaultRange
DN $FQDN
PWD ss-123456!T
}
Simple_Bind
{
TEST [BIND]
LOOP RAND
RANGE $DefaultRange
DN $FQDN
PWD ss-123456!T
}
//-------------------- UPDATES --------------------------//
Update_1Attr
{
TEST [MODIFY]
LOOP RAND
OP LDAP_MOD_REPLACE
RANGE $DefaultRange
DN $FQDN
ATTR homePhone:###(0-999)-###(0-999)-####(0-9999)
}
Update_10Attr
{
TEST [MODIFY]
LOOP RAND
OP LDAP_MOD_REPLACE
RANGE $DefaultRange
DN $FQDN
ATTR homePhone:###(0-999)-###(0-999)-####(0-9999)
ATTR pager:###(0-999)-###(0-999)-####(0-9999)
ATTR mobile:###(0-999)-###(0-999)-####(0-9999)
ATTR facsimileTelephoneNumber:###(0-999)-###(0-999)-####(0-9999)
ATTR ipPhone:###(0-999)-###(0-999)-####(0-9999)
ATTR company:Microsoft
ATTR title:Program Manager
ATTR postOfficeBox:101010
ATTR streetAddress:1 Microsoft Way
ATTR postalCode:98052
}
//-------------------- SEARCHES -----------------------//
Search_Base_1Attr
{
TEST [SEARCH]
LOOP RAND
RANGE $DefaultRange
DN $FQDN
FILTER (objectClass=*)
SCOPE LDAP_SCOPE_BASE
ATTR cn
}
Search_Base_10Attr
{
TEST [SEARCH]
LOOP RAND
RANGE $DefaultRange
DN $FQDN
FILTER (objectClass=*)
SCOPE LDAP_SCOPE_BASE
ATTR postalCode;postOfficeBox;preferredLanguage;roomnumber;streetAddress
ATTR homePhone;ipPhone;telephonenumber;title;wWWHomePage
}
Search_One_Level_1Attr
{
TEST [SEARCH]
LOOP RAND
RANGE $DefaultRange
DN $TeamBranch
FILTER (userPrincipalName=$UserName)
SCOPE LDAP_SCOPE_ONELEVEL
ATTR cn
}
Search_One_Level_10Attr
{
TEST [SEARCH]
LOOP RAND
RANGE $DefaultRange
DN $TeamBranch
FILTER (userPrincipalName=$UserName)
SCOPE LDAP_SCOPE_ONELEVEL
ATTR postalCode;postOfficeBox;preferredLanguage;roomnumber;streetAddress
ATTR homePhone;ipPhone;telephonenumber;title;wWWHomePage
}
Search_Subtree_1Attr
{
TEST [SEARCH]
LOOP RAND
RANGE $DefaultRange
DN $BaseOU
FILTER (userPrincipalName=$UserName)
SCOPE LDAP_SCOPE_SUBTREE
ATTR cn
}
Search_Subtree_10Attr
{
TEST [SEARCH]
LOOP RAND
RANGE $DefaultRange
DN $BaseOU
FILTER (userPrincipalName=$UserName)
SCOPE LDAP_SCOPE_SUBTREE
ATTR postalCode;postOfficeBox;preferredLanguage;roomnumber;streetAddress
ATTR homePhone;ipPhone;telephonenumber;title;wWWHomePage
}
Search_NonIndexed_1Attr
{
TEST [SEARCH]
LOOP RAND
RANGE $DefaultRange
DN $BaseOU
FILTER (department=Windows2003CapacityPlanning-####(*))
SCOPE LDAP_SCOPE_SUBTREE
ATTR cn
}
Search_NonIndexed_10Attr
{
TEST [SEARCH]
LOOP RAND
RANGE $DefaultRange
DN $BaseOU
FILTER (department=Windows2003CapacityPlanning-####(*))
SCOPE LDAP_SCOPE_SUBTREE
ATTR postalCode;postOfficeBox;preferredLanguage;roomnumber;streetAddress
ATTR homePhone;ipPhone;telephonenumber;title;wWWHomePage
}
//-------------------- SETUP --------------------------//
// Note: Order of these tests is not important but are processed more
// quickly at startup time if you define the inner tests first (e.g., AddDivision)
// and the outer tests second (e.g., NewRoot).
// Use NewRoot to run AddDivision, AddUnit and AddTeam all at once
AddDivision
{
TEST [ADD]
LOOP SEQ | ONCE
OP LDAP_MOD_ADD
DN $DivisionBranch
ATTR ObjectClass:organizationalUnit
ATTR name:$Division
ATTR instanceType:4
}
AddUnit
{
TEST [ADD]
LOOP SEQ | ONCE
OP LDAP_MOD_ADD
DN $UnitBranch
ATTR ObjectClass:organizationalUnit
ATTR name:$Unit
ATTR instanceType:4
}
AddTeam
{
TEST [ADD]
LOOP SEQ | ONCE
OP LDAP_MOD_ADD
DN $TeamBranch
ATTR ObjectClass:organizationalUnit
ATTR name:$Team
ATTR instanceType:4
}
// Typically run adtest -run newroot -root # where # is the root you want
NewRoot
{
RANGE #(1)
LOOP SEQ | ONCE
TEST AddDivision
TEST AddUnit
TEST AddTeam
}
AddUser
{
TEST [ADD]
LOOP SEQ | ONCE
OP LDAP_MOD_ADD
RANGE $DefaultRange
DN $FQDN
ATTR ObjectClass:user
ATTR userAccountControl:66048
ATTR cn:$UserName
ATTR SAMAccountName:$UserName
ATTR userPrincipalName:$UserName
ATTR aCSPolicyName:UnicodeString
ATTR adminCount:1
ATTR adminDescription:Uni
ATTR adminDisplayName:Uni
ATTR comment:Uni
ATTR company:Microsoft
ATTR CountryCode:8
ATTR department:Windows2003CapacityPlanning-$Padding(*)
ATTR description:Uni
ATTR desktopProfile:UnicodeString
ATTR destinationIndicator:PrintableString
ATTR displayName:$UserName
ATTR displayNamePrintable:UniP
ATTR division:Uni
ATTR employeeID:Uni
ATTR extensionName:23456
ATTR facsimileTelephoneNumber:425-555-1227
ATTR givenName:$UserName
ATTR groupPriority:UnicodeString
ATTR homeDirectory:UnicodeString
ATTR homeDrive:Uni
ATTR homePhone:425-555-1218
ATTR ipPhone:425-555-1217
ATTR initials:Uni
ATTR maxStorage:300000
ATTR mhsORAddress:Uni
ATTR mobile:425-555-1216
ATTR otherHomePhone:425-555-1215
ATTR otherIpPhone:Uni
ATTR otherMailbox:Uni
ATTR otherMobile:425-555-1213
ATTR otherPager:425-555-1214
ATTR otherTelephone:425-555-1220
ATTR pager:425-555-1219
ATTR personalTitle:PM
ATTR physicalDeliveryOfficeName:Uni
ATTR postalAddress:Uni
ATTR postalCode:Uni
ATTR postOfficeBox:Uni
ATTR preferredLanguage:English
ATTR roomnumber:1326
ATTR streetAddress:Uni
ATTR telephonenumber:425-555-1212
ATTR title:SDE
ATTR wWWHomePage:www.microsoft.com
ATTR userPassword:ss-123456!T
}
AddGlobalDistributionGroup
{
TEST [ADD]
LOOP SEQ | ONCE
OP LDAP_MOD_ADD
RANGE #(0-25)
DN cn=GrpAcc_$Padding(*),$GroupOU
ATTR ObjectClass:group
ATTR groupType:2
ATTR name:GrpAcc_$Padding(*)
ATTR sAMAccountName:GrpAcc_$Padding(*)
ATTR instanceType:4
}
AddGlobalSecurityGroup
{
TEST [ADD]
LOOP SEQ | ONCE
OP LDAP_MOD_ADD
RANGE #(26-50)
DN cn=GrpAcc_$Padding(*),$GroupOU
ATTR ObjectClass:group
ATTR groupType:2147483650
ATTR name:GrpAcc_$Padding(*)
ATTR sAMAccountName:GrpAcc_$Padding(*)
ATTR instanceType:4
}
AddMembers
{
TEST [MODIFY]
LOOP SEQ | ONCE
OP LDAP_MOD_ADD
RANGE $DefaultRange
DN cn=GrpAcc_$Padding([GROUP]),$GroupOU
ATTR member:$FQDN
}
-Tom Ausburne