First published on TechNet on Feb 05, 2012

 

Introduction

 

With the introduction of Windows 2003 and the new DNS application partitions, I have helped numerous customers resolve the issue of having multiple copies of the same DNS zone. So, today we're going to cover the following:

 

1.) What exactly does this mean?

2.) What are the symptoms?

3.) How does this scenario occur?

4.) How to resolve the problem?

 

What Does this Mean?

 

Quick history lesson...With the introduction of Windows 2003, Microsoft created two new DNS-related application partitions. Let's quickly discuss why Microsoft did this. To do this, we'll have to take a look at how Windows 2000 implemented DNS. With Windows 2000, when you created a new primary Active Directory integrated DNS zone, it automatically got stored in the domain partition. This is the same partition that contains your users and groups. You could locate these DNS zones by navigating to Domain Partition>System>MicrosoftDNS. All domain controllers in the domain receive this partition regardless of whether they're actually running DNS. Additionally, all GC's in the forest receive a read-only, partial-attribute copy of this partition. Based on this explanation, if a DNS zone is stored in the domain partition, it will replicate to a domain controller that NOT running DNS. And when it replicates to all the GC's in the forest, it will also replicate to a domain controller that is NOT running DNS.

 

Ultimately, we were replicating DNS information to a DC that can't even use it..Now, doesn't that sound really inefficient. Microsoft went back to the drawing board and came up with this new idea of DNS application partitions that ONLY replicate to DNS servers. Brilliant!

 

 

With the new DNS-related application partitions you see above, we could now put DNS zones in these locations, which would only replicate to domain controllers that are also DNS servers. But Microsoft also kept the ability to store DNS in the domain partitions, for backwards compability purposes. DomainDNS will only replicate to DNS servers in the same domain and ForestDNS will only replicate to DNS servers in the entire forest. No more inefficient replication! The caveat here now is that DNS now has three places to store a DNS zone, which can lead to the possibility that you could have multiple copies of the same DNS zone stored in different locations:

 

 

The key to remember here though is the order of precendence: Domain Partition, then DomainDNS, and then ForestDNS. If DNS finds the same DNS zone in multiple partitions in Active Directory, the above precedence will determine which copy it uses.

 

What are the symptoms?

 

Inconsistent DNS records - The first symptom you'll probably notice on a child domain DNS server is that the DNS records are not the same as within the same DNS zone on a another domain's DNS server

 

Event ID's - Additionally, when this happens, you see the following Event ID on the affected DNS servers:

 

Event ID: 4515
Event Source: DNS
Event Type: Warning
Event Description: The zone contoso.com was previously loaded from the directory partition ForestDnsZones.contoso.com but another copy of the zone has been found in directory partition contoso.com.

 

How does this happen?

 

Scenario #1

 

This first scenario involves a single forest, single domain environment. Before the problem started occurring, the contoso.com DNS zone was configured to replicate to all DNS servers in the forest via the ForestDNS application partition. If you open up the DNS console and go to the properties of the DNS zone and under the replicate scope click change, the option selected was "Replicate to all DNS servers in the Active Directory forest contoso.com":

 

 

One day, the admin has to stand up a new domain controller in some remote location that has really slow WAN link. He runs DCPromo, everything seems to be working fine. He knows that without DNS, nothing will work so he opens the DNS console and notices that it doesn't have the contoso.com DNS zone yet . He gets really impatient and decides to manually create the contoso.com DNS zone hoping to give it a kick in the butt. So he creates a new DNS zone called contoso.com, and decides to replicate it in the domain partition within Active Directory by choosing "Replicate to all domain controllers in the Active Directory domain contoso.com":

 

 

He thens waits 15-30 minutes depending on the replicates times and immediately his phone is ringing that all enterprise name resolution is not working. He starts connecting to all the DNS servers in the domain and notices all the DNS records that existed previously are GONE!

 

What happened? In this scenario, he actually has multiple copies of contoso.com. Although their was already a copy of contoso.com in the ForestDNS application partition, due to the slow WAN link, it hadn't replicated to this DNS server yet. He got impatient and created another copy of contoso.com in the domain partition. Based on the precedence we discussed earlier, DNS started using the newly created domain partition copy rather than the one in the ForestDNS application partition. If we were to use adsiedit and connect to the following partitions, we would have found the following:

 

 

 

Scenario #2

 

This scenario is the more common one but is slightly more complex because the environment has multiple domains. Nonetheless, what we learned earlier still applies here. This environment is a single forest with six domains. Before the problem occurred, the admins decided to replicate the forest-root DNS zone, contoso.com to all DNS servers in the forest via the ForestDNS application partition.

 

 

One day, a domain admin in one of the child domains opens up the DNS console, notices that he has a copy of the contoso.com DNS zone. He also notices that it is replicating to all DNS servers in the forest. He wants a copy all to himself so he changes the scope of replication on the contoso.com DNS zone to "Replicate to all domain controllers in the Active Directory domain child.contoso.com".

 

 

Under the hood, DNS copies the contoso.com DNS zone into the child.contoso.com domain partition and then attempts to delete the copy stored in the ForestDNS application partition. Since he isn't an Enterprise Admin or Domain Admin in the forest-root domain, he doesn't have permission to delete it from the ForestDNS partition. The DNS servers in child domain now have two copies of contoso.com; one in their domain partition and another one in their ForestDNS application parition. If we were to use adsiedit and connect to the following partitions, we would have found the following:

 

 

 

 

Based on the precedence of DNS we discussed earlier, all DNS servers in this child domain will operate off of the copy in the domain partition while the other DNS servers in the rest of the AD forest will continue to operate off of the copy in the ForestDNS application partition. Over time, this misconfiguration and problem will only become magnified.

 

Over time, the two DNS zones will get further and further out of sync because all the DNS updates/changes that are occurring throughout the forest are being replicating to the ForestDNS copy while the domain partition copy begins to get stale. Perhaps even the scavenging process runs on the child domain partition copy of contoso.com and they get further out of sync. Eventually, the admin discovers this, don't know what to do so they manually start making records for the resources just to get name resolution working.

 

Resolution

Warning: Before executing on the solutions below, please confirm that the above scenario apply. If you are not sure, please contact Microsoft support. Also, due to the critical nature of DNS, please perform these resolutions during off-peak hours.

 

First off, if either of these scenarios occur, they will impact all the domain controllers that are DNS servers in the entire respective domain. If you suspect this is happening, connect to the folllowing partitions in adsiedit and document what you find:

 

dc=Domain,dc=Com

dc=DomainDNSZones,dc=domain.dc=Com

dc=ForestDNSZones,dc=ForestRootDomain,dc=Com

 

Next, find out how many domains exist in the forest. Use the following guidelines to resolve this issue:

 

Only 1 domain: If the problem just started happing, from within adsiedit, determine which DNS zone copy has the most recent, up-to-date DNS records and delete the other DNS zone. From adsiedit, right-click and delete the respective DNS zone. If the problem has been occurring for a month or longer, delete the other copies of the DNS from either dc=DomainDNSZones,dc=Domain,dc=Com or from dc=ForestDNSZones,dc=ForestRootDomain,dc=Com.

 

Multiple domains: In the entire forest, determine how many DNS servers are operating off the one DNS zone  vs. how many are operating off of the other DNS zone. Keep the one that is being used by more DNS servers. The logic here is that the more DNS servers that are using it, the more update-to-date records it must have. For example

 

1 forest with 6 domains and each domain has 6 domain controllers. If the problem exists in only 1 child domain then:

 

5 domains x 6 DNS servers = 30 DNS servers or 83% are using the ForestDNS copy.

1 domain x 6 DNS servers = 6 DNS servers or 17% are using their Domain or DomainDNS copy.

 

Since 83% of the DNS servers are using the ForestDNS copy of DNS, that is the one we will keep.

 

Consequently, on the affected child domain DNS servers, open up the DNS console, right-click the DNS zone and choose export list. This allows us to back-up any static records. Next right-click and select delete. The DNS service polls for changes to DNS zones every 3 minutes and will then find the ForestDNS copy and load it. Lastly, restart the netlogon service on the domain controllers in the affected child domain to ensure the domain controllers re-register their necessary SRV records.

 

Prevention

 

Scenario #1: The only way to avoid scenario #1 is to be patient and allow AD replication to populate the already existing contoso.com DNS zone.

 

Scenario #2: Based on the default permissions set within Active Directory, the only way to avoid this scenario is through education and awareness, or eliminating all child domain admins 🙂 Joking aside, make all child domain admins aware that certain DNS zones will be replicating forest-wide and that their scope of replication must not be changed.