DFS replication issues

[nikitamobile855]

Hello everyone, 

We are running Windows Server 2016 as a Primary Domain Controller. We don't have DFS management tool installed however I'm getting 6002 errors in event viewer. Please advise on how to investigate further. Object CN could not be found from ADSI edit

[#2]

can I just ignore that last error since I am fuly migrated?


S C:\Users\administrator.IOSDOMAIN> dfsrmig /getmigrationstate

ll domain controllers have migrated successfully to the Global state ('Eliminated').
igration has reached a consistent state on all domain controllers.
ucceeded.
S C:\Users\administrator.IOSDOMAIN>

and my replication looks good from what I can tell too

Windows PowerShell
Copyright (C) 2016 Microsoft Corporation. All rights reserved.

PS C:\Users\administrator.IOSDOMAIN> repadmin /replsummary
Replication Summary Start Time: 2022-06-10 04:44:41

Beginning data collection for replication summary, this may take awhile:
  ..........


Source DSA          largest delta    fails/total %%   error
 BABBAGE                   51m:58s    0 /  10    0
 IOS-SEA-A1                13m:00s    0 /  20    0
 IOSLA-DCFS                12m:59s    0 /  15    0
 OR-VM-1                   10m:25s    0 /  10    0
 PS-BAY-AD1                46m:10s    0 /  10    0
 PS-I-AD1                  48m:24s    0 /  15    0
 PS-I-AD2                  47m:06s    0 /  25    0


Destination DSA     largest delta    fails/total %%   error
 BABBAGE                   46m:13s    0 /   5    0
 IOS-SEA-A1                10m:27s    0 /  20    0
 IOSLA-DCFS                08m:02s    0 /  15    0
 OR-VM-1                   13m:00s    0 /  20    0
 PS-BAY-AD1                52m:01s    0 /  15    0
 PS-I-AD1                  47m:07s    0 /  15    0
 PS-I-AD2                  48m:26s    0 /  15    0


PS C:\Users\administrator.IOSDOMAIN>

[#3]

@Zalastar 

Yeah, it looks better.

What you need to do now is:

1. Edit "CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local";
2. Edit the "serverReferenceBL" attribute, setting it to a value of "CN=IOS-SEA-A1,CN=Topology,CN=Domain System Volume,CN=DFSR-GlobalSettings,CN=System,DC=iosdomain,DC=local".

Cheers,

Lain

[#4]

can you tell me where to find or how to connect to CN=NTDS Settings? I cant seem to get it right

[#5]

@Zalastar 

Yeah, sure.

So, assuming you're still using ADSIEdit, you want to right-click the "ADSI Edit" top-most node and choose "Connect to".

Then, you want to change the "well known Naming Context" drop-down to "Configuration" as shown below.

You can then browse away to that location I provided using the new tree showing up in the top-left.

Cheers,

Lain

[#6]

I see ServerReference but not ServerRefrenceBL

[#7]

@Zalastar 

Have a look within the Properties box down the bottom-right, under the Filter button, and see if Backlinks is enabled/checked while "show only attributes with values" is unchecked:

Cheers,

Lain

[#8]

Getting closer! Its not letting me edit the field though

[#9]

@Zalastar 

Yeah, okay - that makes sense. My powers of observation and memory are failing me in my old age.

I wasn't sure if through being a backlink it would be editable. I thought it might be, but clearly I thought wrong.

What that means is a forward reference stored on another object will be missing, which results in this backlink also showing up as missing.

If you navigate back to this object in the default naming context:

CN=IOS-SEA-A1,CN=Topology,CN=Domain System Volume,CN=DFSR-GlobalSettings,CN=System,DC=iosdomain,DC=local

Does it have a value for "serverReference" equal to:

CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local

Edited to remove the question mark in the line above, which got caught in the <pre> formatting tag.

If serverReference value is missing, set it to the above value and then re-check the object in the configuration partition. You should find it now shows the serverReferenceBL value.

Cheers,

Lain

[#10]

First I want to thank you for helping me, your time has been invaluable.
My attention span was fading after being up all night. I decided to go to bed before I made a bigger mistake.
So your advice fixed the issue . It brought me back to how I got into this mess.
I was trying to follow these instructions and deleted the objects somehow

The DFS Replication service stopped replication on the folder with the following local path: C:\Windows\SYSVOL\domain. This server has been disconnected from other partners for 948 days, which is longer than the time allowed by the MaxOfflineTimeInDays parameter (60). DFS Replication considers the data in this folder to be stale, and this server will not replicate the folder until this error is corrected. 
 
To resume replication of this folder, use the DFS Management snap-in to remove this server from the replication group, and then add it back to the group. This causes the server to perform an initial synchronization task, which replaces the stale data with fresh data from other members of the replication group. 
 
Additional Information: 
Error: 9061 (The replicated folder has been offline for too long.) 
Replicated Folder Name: SYSVOL Share 
Replicated Folder ID: 8EC04617-53A4-4148-AB24-4CDD050A1716 
Replication Group Name: Domain System Volume 
Replication Group ID: 860C23C1-6213-4EDC-8AA3-0C4B75F238DA 
Member ID: 43A19C80-D9AB-4DCC-8434-7BC4987D3B5B

[#11]

so I just performed a D2 Sysvol Replication but that didn't solve it

The DFS Replication service stopped replication on the replicated folder at local path C:\Windows\SYSVOL\domain.

Additional Information:
Error: 9003 (The replication group is invalid)
Additional context of the error:
Replicated Folder Name: SYSVOL Share
Replicated Folder ID: 8EC04617-53A4-4148-AB24-4CDD050A1716
Replication Group Name: Domain System Volume
Replication Group ID: 860C23C1-6213-4EDC-8AA3-0C4B75F238DA
Member ID: 43A19C80-D9AB-4DCC-8434-7BC4987D3B5B

[#12]

@LainRobertson 

Hey its fixed!! No more log errors either.

I must not have waited long enough for replications to finish before setting the flag back and running dfsrdiag pollad  

When using adsiedit.msc to change those objects do I need to restart the DFS Rep service after each change?

Thank you again for your help

S C:\Users\administrator.IOSDOMAIN> repadmin /syncall /aed
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=PS-BAY-AD1,CN=Servers,CN=BAY,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=OR-VM-1,CN=Servers,CN=Oregon,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=BABBAGE,CN=Servers,CN=BAY,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=PS-I-AD2,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=BABBAGE,CN=Servers,CN=BAY,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=PS-I-AD2,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=PS-BAY-AD1,CN=Servers,CN=BAY,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=OR-VM-1,CN=Servers,CN=Oregon,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=OR-VM-1,CN=Servers,CN=Oregon,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=OR-VM-1,CN=Servers,CN=Oregon,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=IOSLA-DCFS,CN=Servers,CN=CA-LA,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=IOSLA-DCFS,CN=Servers,CN=CA-LA,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=PS-I-AD1,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=PS-I-AD1,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=PS-I-AD2,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=PS-I-AD2,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: SyncAll Finished.
yncAll terminated with no errors.

S C:\Users\administrator.IOSDOMAIN> dcdiag

irectory Server Diagnosis

erforming initial setup:
  Trying to find home server...
  Home Server = IOS-SEA-A1
  * Identified AD Forest.
  Done gathering initial info.

oing initial required tests

  Testing server: SEATTLE\IOS-SEA-A1
     Starting test: Connectivity
        ......................... IOS-SEA-A1 passed test Connectivity

oing primary tests

  Testing server: SEATTLE\IOS-SEA-A1
     Starting test: Advertising
        ......................... IOS-SEA-A1 passed test Advertising
     Starting test: FrsEvent
        ......................... IOS-SEA-A1 passed test FrsEvent
     Starting test: DFSREvent
        ......................... IOS-SEA-A1 passed test DFSREvent
     Starting test: SysVolCheck
        ......................... IOS-SEA-A1 passed test SysVolCheck
     Starting test: KccEvent
        ......................... IOS-SEA-A1 passed test KccEvent
     Starting test: KnowsOfRoleHolders
        ......................... IOS-SEA-A1 passed test KnowsOfRoleHolders
     Starting test: MachineAccount
        ......................... IOS-SEA-A1 passed test MachineAccount
     Starting test: NCSecDesc
        ......................... IOS-SEA-A1 passed test NCSecDesc
     Starting test: NetLogons
        ......................... IOS-SEA-A1 passed test NetLogons
     Starting test: ObjectsReplicated
        ......................... IOS-SEA-A1 passed test ObjectsReplicated
     Starting test: Replications
        ......................... IOS-SEA-A1 passed test Replications
     Starting test: RidManager
        ......................... IOS-SEA-A1 passed test RidManager
     Starting test: Services
        ......................... IOS-SEA-A1 passed test Services
     Starting test: SystemLog
        ......................... IOS-SEA-A1 passed test SystemLog
     Starting test: VerifyReferences
        ......................... IOS-SEA-A1 passed test VerifyReferences


  Running partition tests on : ForestDnsZones
     Starting test: CheckSDRefDom
        ......................... ForestDnsZones passed test CheckSDRefDom
     Starting test: CrossRefValidation
        ......................... ForestDnsZones passed test CrossRefValidation

  Running partition tests on : DomainDnsZones
     Starting test: CheckSDRefDom
        ......................... DomainDnsZones passed test CheckSDRefDom
     Starting test: CrossRefValidation
        ......................... DomainDnsZones passed test CrossRefValidation

  Running partition tests on : Schema
     Starting test: CheckSDRefDom
        ......................... Schema passed test CheckSDRefDom
     Starting test: CrossRefValidation
        ......................... Schema passed test CrossRefValidation

  Running partition tests on : Configuration
     Starting test: CheckSDRefDom
        ......................... Configuration passed test CheckSDRefDom
     Starting test: CrossRefValidation
        ......................... Configuration passed test CrossRefValidation

  Running partition tests on : iosdomain
     Starting test: CheckSDRefDom
        ......................... iosdomain passed test CheckSDRefDom
     Starting test: CrossRefValidation
        ......................... iosdomain passed test CrossRefValidation

  Running enterprise tests on : iosdomain.local
     Starting test: LocatorCheck
        ......................... iosdomain.local passed test LocatorCheck
     Starting test: Intersite
        ......................... iosdomain.local passed test Intersite

[#13]

@Zalastar 

Glad you got it sorted!

No, you don't need to restart the service after each change to the Active Directory objects. You can, of course, but it's not mandatory.

The DFS-R service checks AD periodically, and if you're in a hurry, you can run the following from the member you're trying working on:

dfsrdiag pollad

That will cause DFS-R to check for changes immediately.

Cheers,

Lain

[#14]

Oh I spoke too sonn 😞

The DFS Replication service stopped replication on the replicated folder at local path C:\Windows\SYSVOL\domain.

Additional Information:
Error: 9003 (The replication group is invalid)
Additional context of the error:
Replicated Folder Name: SYSVOL Share
Replicated Folder ID: 8EC04617-53A4-4148-AB24-4CDD050A1716
Replication Group Name: Domain System Volume
Replication Group ID: 860C23C1-6213-4EDC-8AA3-0C4B75F238DA
Member ID: 43A19C80-D9AB-4DCC-8434-7BC4987D3B5B

[#15]

@Zalastar 

Okay, so I think you mentioned earlier something about the FRS migration status being okay, which leads me to assume that an FRS to DFS-R migration took place at some point?

There's still things we can do but I'm keen to check some things first, as they're all edging closer to that "point of last resort" concept.

Does the following command return anything?

Get-ADObject -Filter { (cn -ne "File Replication Service") } -SearchBase "CN=File Replication Service,CN=System,$((Get-ADRootDSE).defaultNamingContext)" -SearchScope Subtree | Select-Object -Property objectGUID, objectClass, distinguishedName

Secondly, were you following the process here when you reset DFS-R ealier?

Force synchronization for Distributed File System Replication (DFSR) replicated sysvol replication -...

Note: It says D2-like, but it's not actually the older BurFlags process.

Cheers,

Lain

[#16]

@LainRobertson 

The migration was done before I inherited the position, fortunately its the only DC in the forest with an issue.

The result of that last command

objectGUID                           objectClass     distinguishedName
----------                           -----------     -----------------
01c80ee4-dc5a-4bfa-8ddf-2500a62a3e7e nTFRSReplicaSet CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=iosdomain,DC=local
37442159-8097-461b-8466-bbddce3e6723 nTFRSMember     CN=WIN-NAAPRAMLU0A,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=iosdomain,DC=local

Yes I did follow that method but I am wondering how long I should wait for replication to finish before changing the flag and moving on to the next step? How do I tell how long my forest replication takes to finish?

[#17]

@Zalastar 

Okay, the command output is good. You can safely also remove those two leftover objects from the output, which will bring the FRS era completely to a close.

The main thing I didn't want to see - and we didn't, is any kind of direct or indirect reference to IOS-SEA-A1. So, life is good(-ish.)

From the Microsoft Docs article, assuming you're running the commands from IOS-SEA-A1 (while also ensuring things like ADSIEDIT are specifically pointing to IOS-SEA-A1), then for steps 2 and 6, use the following statement to push changes outward from IOS-SEA-A1 to the other domain controllers:

repadmin /syncall IOS-SEA-A1 DC=iosdomain,DC=local /d /e /P

The confirmational events are described in steps 4 and 8, but that doesn't help with your question on how long to wait.

The answer to that is: you don't have to wait since you're only dealing with a single domain controller. So long a you see the events listed from steps 4 and 8, you've done everything correctly.

Of course, it takes SYSVOL some time to repopulate and even then, if you find it fails again as you have before, then we might be at the point where you have to delete the DFS-R database off of IOS-SEA-A1.

While Microsoft does discourage this in favour of the process above, the process above also doesn't fix every kind of issue, meaning that's the direction we're heading.

Still, I'd do the above Docs process once more, being sure to note the events (maybe clear the DFS-R event log first to make things easier to digest and track) before looking to delete the database.

Rest assured, deleting the database isn't all "impending doom", either. Microsoft does note some issues that can arise but they're outweighed by getting DFS-R working again.

The directory the database resides in will throw an access denied if you try to access it, but as per the following article, it really is there (read under the Symptoms heading at the top.) But let's not get ahead of ourselves as maybe the Docs article above will help this time around.

DFSR databases crash on primary member - Windows Server | Microsoft Docs

Cheers,

Lain

[#18]

Ok its my under standing to go through the steps again but running the altered repadmin /syncall command you provided. And just to be clear. I am setting the flag to False on the affected DC which is Seattle , which makes in non authorative, and leaving the others set to True

Im not understanding how to do this

" You can safely also remove those two leftover objects from the output, which will bring the FRS era completely to a close."

Do you mean for me to find and delete them with ADSIEDIT or will following the above process and using that Repadmin command take care of this? i a not understanding where those references in the output came from

I am running everything on the Seattle controller at the moment. The PDC is the PS-I-AD1 DC but I understand for the most part that the work I am doing needs to be done on the affected DC, which is Seattle

[#19]

Ok I ran through the steps and I am beginning to understand what it did

After the Error 4114 I saw quite a few 4412 errors starting with this one

The DFS Replication service detected that a file was changed on multiple servers. A conflict resolution algorithm was used to determine the winning file. The losing file was moved to the Conflict and Deleted folder.

Additional Information:
Original File Path: C:\Windows\SYSVOL\domain\Policies\{1F60C4D9-39CC-4A52-956E-0715D20D84B2}\GPT.INI
New Name in Conflict Folder: GPT-{D9CFBC30-0905-453D-A1F6-403F6DF348CF}-v166.INI
Replicated Folder Root: C:\Windows\SYSVOL\domain
File ID: {6CF9F740-471F-4A0A-99D6-EA74625846E9}-v66
Replicated Folder Name: SYSVOL Share
Replicated Folder ID: 8EC04617-53A4-4148-AB24-4CDD050A1716
Replication Group Name: Domain System Volume
Replication Group ID: 860C23C1-6213-4EDC-8AA3-0C4B75F238DA
Member ID: 43A19C80-D9AB-4DCC-8434-7BC4987D3B5B
Partner Member ID: 806D1444-4493-4ADF-8A0F-DA75BBDF6FA9

and finally ended up with the magic error 4604

The DFS Replication service successfully initialized the SYSVOL replicated folder at local path C:\Windows\SYSVOL\domain. This member has completed initial synchronization of SYSVOL with partner OR-VM-1.iosdomain.local.  To check for the presence of the SYSVOL share, open a command prompt window and then type "net share". 
 
Additional Information: 
Replicated Folder Name: SYSVOL Share 
Replicated Folder ID: 8EC04617-53A4-4148-AB24-4CDD050A1716 
Replication Group Name: Domain System Volume 
Replication Group ID: 860C23C1-6213-4EDC-8AA3-0C4B75F238DA 
Member ID: 43A19C80-D9AB-4DCC-8434-7BC4987D3B5B 
Sync partner: OR-VM-1.iosdomain.local

[#20]

@LainRobertson 

and thank you sir

PS C:\Users\administrator.IOSDOMAIN> net share

Share name   Resource                        Remark

-------------------------------------------------------------------------------
C$           C:\                             Default share
IPC$                                         Remote IPC
UDP_APM$     C:\Program Files\Arcserve\Unified Data Protection\APM

ADMIN$       C:\Windows                      Remote Admin
NETLOGON     C:\Windows\SYSVOL\sysvol\iosdomain.local\SCRIPTS
                                             Logon server share
SYSVOL       C:\Windows\SYSVOL\sysvol        Logon server share
The command completed successfully.

PS C:\Users\administrator.IOSDOMAIN>

[#21]

@LainRobertson 

A big thank you for the time you spend here on this thread! I am so happy right now. Not a single error in the Forest. And my Delta sync times are way down now too

Source DSA          largest delta    fails/total %%   error
 BABBAGE                   26m:58s    0 /  10    0
 IOS-SEA-A1                13m:01s    0 /  20    0
 IOSLA-DCFS                08m:24s    0 /  15    0
 OR-VM-1                   07m:06s    0 /  10    0
 PS-BAY-AD1                21m:10s    0 /  10    0
 PS-I-AD1                  23m:23s    0 /  15    0
 PS-I-AD2                  22m:06s    0 /  25    0


Destination DSA     largest delta    fails/total %%   error
 BABBAGE                   21m:14s    0 /   5    0
 IOS-SEA-A1                   :26s    0 /  20    0
 IOSLA-DCFS                13m:02s    0 /  15    0
 OR-VM-1                   03m:01s    0 /  20    0
 PS-BAY-AD1                27m:01s    0 /  15    0
 PS-I-AD1                  22m:08s    0 /  15    0
 PS-I-AD2                  23m:25s    0 /  15    0

 I finally understood what you meant about getting rid of the FRS leftovers

PS C:\Users\administrator.IOSDOMAIN> Get-ADObject -Filter { (cn -ne "File Replication Service") } -SearchBase "CN=File Replication Service,CN=System,$((Get-A
DRootDSE).defaultNamingContext)" -SearchScope Subtree | Select-Object -Property objectGUID, objectClass, distinguishedName
PS C:\Users\administrator.IOSDOMAIN>