For the first technical post I thought it would be best to lay the groundwork and mention a tool which can be used in a wide variety of AD replication scenarios. We'll talk about one of the useful features in this post, and follow up with one other for the next post before we move onto another topic.
In my personal point of view, there are only two kinds of AD admins: those who love REPLMON, and those who love REPADMIN. As you may be able to infer from the title of this post, I am the latter. Not that there is anything necessarily evil about REPLMON advocates, but let's face it: it's not the cat's meow (or the bee's knees) of AD replication troubleshooting. Not like Repadmin.exe anyhow.
For those who haven't had an opportunity to use this tool before it is the general purpose, command line tool for troubleshooting directory replication. Repadmin.exe is included in the Windows Support Tools, which can be downloaded for free (free is one of my favorite words) here:
We had the tool available in Windows 2000, but in the Windows 2003 time frame it has more features. Syntax between the versions is slightly different, with a few deprecated commands. One useability thing which was a great improvement, generally speaking, was the addition of code to help with lookup of domain controller names. Windows 2000 would often require the MSDCS GUID to be used in the syntax. General idea is that our development folks (you know who you are) have been improving the tool as needed over the years, and they deserve HUGE kudos for it.
Back to the technical part...
One of the more frequent calls we at Microsoft Directory Services support get is problematic network connections which prevent AD replication. I used the big fancy generic term "problematic" since the actual originating causes of the AD replication failures are many. Could be misconfigured DNS on the DC NIC, wrong binding order of same, closed networking ports anywhere between the DCs in question, missing SRV records-or any of a myriad others. Bottom line is that, once you've fixed that problem, you need to test to make sure you are, in fact, replicating succesfully.
Repadmin's your tool for that.
Let's say that I add a bit of text to a user account in order to see if that data arrives. The Mark One Eye-ball is one method (just open DSA.MSC on the opposite end) but there's a better way, and it can track it as the change travails the forest.
REPADMIN /SHOWOBJMETA * "cn=user,ou=weisenheimers,dc=wacky,dc=org" >usermeta.txt
The asterisk can be replaced by a single domain controller name (like DC1). The asterisk runs against all in forest, so have your EA creds ready for that, and make sure to take into account network topology concerns since the tool queries all DCs for the object metadata.
The distinguishedname in quotes can actually be replaced by the objectGUID. Very cool for when you receive an event citing an objectGUID but no other identifying piece of info. The syntax of that field is “<GUID=objectguid>”.
The command above will output the replication metadata to a text file. This data is the information which the directory replicator uses to track updates for an object's attributes. Since AD replicates changes on a per attribute basis (or in the case of linked value replication, single value update for a multivalued attribute) you will see attribute name, local USN, time and date stamp (good for forensics), and originating DC and USN (also good for forensics).
Here's a sample:
repadmin running command /showobjmeta against server DC1.wacky.org
36 entries.
Loc.USN Originating DC Org.USN Org.Time/Date Ver Attribute
========= =============================================== =========
20341573 CrazylandDC1 20341573 2004-09-10 13:12:48 1 objectClass
80472995 CrazylandDC1 80472995 2005-10-13 07:49:08 3 cn
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 description
20341573 CrazylandDC1 20341573 2004-09-10 13:12:48 1 instanceType
20341573 CrazylandDC1 20341573 2004-09-10 13:12:48 1 whenCreated
20341575 CrazylandDC1 20341575 2004-09-10 13:12:48 2 displayName
20341573 CrazylandDC1 20341573 2004-09-10 13:12:48 1 nTSecurityDescriptor
80472995 CrazylandDC4 33405674 2005-10-13 07:49:06 5 name
80336274 CrazylandDC5 28321657 2005-10-10 14:35:17 4 userAccountControl
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 codePage
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 countryCode
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 homeDirectory
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 homeDrive
80391546 CrazylandDC1 80391546 2005-10-11 18:59:09 15 dBCSPwd
20341573 CrazylandDC1 20341573 2004-09-10 13:12:48 1 localPolicyFlags
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 scriptPath
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 logonHours
80566031 CrazylandDC1 80566031 2006-6-11 18:59:09 15 unicodePwd
80391546 CrazylandDC1 80391546 2005-10-11 18:59:09 15 ntPwdHistory
80391546 CrazylandDC1 80391546 2005-10-11 18:59:09 15 pwdLastSet
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 primaryGroupID
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 userParameters
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 profilePath
20341573 CrazylandDC1 20341573 2004-09-10 13:12:48 1 objectSid
33433553 CrazylandDC2 33433553 2006-06-12 15:16:35 1 comment
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 accountExpires
80391546 CrazylandDC1 80391546 2005-10-11 18:59:09 15 lmPwdHistory
20341573 CrazylandDC1 20341573 2004-09-10 13:12:48 1 sAMAccountName
20341573 CrazylandDC1 20341573 2004-09-10 13:12:48 1 sAMAccountType
20341645 CrazylandDC1 20341645 2004-09-10 13:13:00 1 operatingSystem
20341645 CrazylandDC1 20341645 2004-09-10 13:13:00 1 operatingSystemVersion
20341577 CrazylandDC1 20341577 2004-09-10 13:12:48 1 dNSHostName
80416919 CrazylandDC5 28388830 2005-10-12 08:19:49 11 servicePrincipalName
20341573 CrazylandDC1 20341573 2004-09-10 13:12:48 1 objectCategory
20341574 CrazylandDC1 20341574 2004-09-10 13:12:48 1 isCriticalSystemObject
80450630 CrazylandDC1 80450630 2005-10-12 20:51:07 22 lastLogonTimestamp
So, in our replication test to see if things were working, we made a change on DC2-we added a comment to the comment field of a user object in DSA.MSC. Since there is a replication connection where DC1 pulls changes from DC2. Find the “comment” entry above and you’ll see it arrived. Run the SHOWOBJMETA command for this object again against all DCs and you can track it as it successfully replicates throughout the forest.
Final concepts on tracking AD replication using SHOWOBJMETA. First, keep in mind that we have writable and non-writable partitions. In other words, a DC in the WACKY.ORG domain hosts a writable partition for dc=wacky,dc=org, but since it is a GC it will host the non-writable partitions for all other domains in the forest, such as dc=very,dc=wacky,dc=org and dc=notso,dc=wacky,dc=org.
The reason this is mentioned is that if the object and attributes are published as part of the partial attribute set in it’s schema definition, then you may show Global Catalog servers responding in the SHOWOBJMETA that they have (or haven’t) gotten that update.
The other end of that idea is that the non-writable DCs (meaning DCs not in the domain which the user belongs to) which are also not GCs will show the message below:
repadmin running command /showobjmeta against server DC8.very.wacky.org
DsReplicaGetInfo() failed with status 8333 (0x208d):
Directory object not found.
So, we haven’t really gone over any stunningly difficult issues today, but we’ve made an intro to one of our best AD replication troubleshooting tools.
Next post, we’ll going to go over the REPADMIN /SHOWREPL command and how to use it. In the meantime, if anyone has any questions or specific requests then fire away. This blog is for you all, so requests are taken...
Microsoft
