Forum Discussion

nikitamobile855's avatar
nikitamobile855
Brass Contributor
Apr 14, 2022

DFS replication issues

Hello everyone,    We are running Windows Server 2016 as a Primary Domain Controller. We don't have DFS management tool installed however I'm getting 6002 errors in event viewer. Please advise on h...
Active Directory
Windows Server
  • LainRobertson's avatar
    LainRobertson
    Apr 20, 2022

    nikitamobile855 

     

    Thanks, Nikita. That helped a great deal and saved a lot of time.

     

    I'll start at the end and work backwards.

     

    You want to run these commands to clean up what is a very broken DFS-R configuration. I'll provide an in-depth explanation after the commands

     

     

    Remove-ADObject -Identity "CN=45d9316b-1098-408e-a65d-8ce8449f0aaa,CN=DFSR-LocalSettings,CN=UZTASSRV01,OU=Domain Controllers,DC=sm,DC=local" -Recursive -Confirm:$false;
Remove-ADObject -Identity "CN=a7297769-fdcd-4490-ae1c-c80808f44d36,CN=DFSR-LocalSettings,CN=UZTASSRV01,OU=Domain Controllers,DC=sm,DC=local" -Recursive -Confirm:$false;
Remove-ADObject -Identity "CN=DFS,CN=DFSR-GlobalSettings,CN=System,DC=sm,DC=local" -Recursive -Confirm:$false;
Remove-ADObject -Identity "CN=DFS_IT,CN=DFSR-GlobalSettings,CN=System,DC=sm,DC=local" -Recursive -Confirm:$false;
Remove-ADObject -Identity "CN=UZTASSVR02,CN=Topology,CN=Domain System Volume,CN=DFSR-GlobalSettings,CN=System,DC=sm,DC=local" -Recursive -Confirm:$false;

     

     

    Overview

    What the JSON output you provided showed is:

     

    1. There are no DFS namespaces configured;
    2. The DFS replication group definitions are badly broken in places;
    3. The "built-in" SYSVOL DFS-R definition is fine but contains an orphaned reference to a domain controller no longer in existence (UZTASSVR02);
    4. Three DFS replications groups are defined:
      1. DFS;
      2. DFS_IT;
      3. Domain System Volume.

     

    Approach

    There were two options I could have pursued:

     

    1. Add the missing "MemberReference" values onto the existing "msDFSR-Subscriber" objects; or
    2. Delete the DFS and DFS_IT DFS replication groups.

     

    I have chosen option 2 since:

     

    1. There is no associated DFS namespace;
    2. The replication groups would only have a single member, which is pointless;
    3. It provides you with the cleanest outcome, since you can always create new DFS namespaces and replication groups if you decide to later on, and you won't have to worry about old, corrupt data lingering around.

     

    Explanation of each command

    LineComment
    1Removes the orphaned subscription to the "DFS" replication group from UZTASSRV01.
    2Removes the orphaned subscription to the "DFS_IT" replication group from UZTASSRV01.
    3Removes the "DFS" replication group.
    4Removes the "DFS_IT" replication group.
    5Removes the orphaned UZTASSVR02 reference from the "Domain System Volume" replication group.

     

    Once you have removed these objects, it will take DFS-R a little while to recognise the changes.

     

    If you want to hurry the process up, you can do any one of the following:

     

    1. Restart UZTASSRV01 (since it's the only remaining host); or
    2. Restart the the "DFSR" service on UZTASSRV01; or
    3. Run the following command on UZTASSRV01 (it may not be installed though, which is fine):
    dfsdiag pollad

     

    Anyhow, once you've either waited a bit or hurried things up, you should find the Event Viewer errors stop.

     

    Cheers,

    Lain

Zalastar's avatar
Zalastar
Copper Contributor

LainRobertson 

 

Hey its fixed!! No more log errors either.

I must not have waited long enough for replications to finish before setting the flag back and running dfsrdiag pollad  

 

When using adsiedit.msc to change those objects do I need to restart the DFS Rep service after each change?

 

Thank you again for your help

 

 

S C:\Users\administrator.IOSDOMAIN> repadmin /syncall /aed
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=PS-BAY-AD1,CN=Servers,CN=BAY,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=OR-VM-1,CN=Servers,CN=Oregon,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=BABBAGE,CN=Servers,CN=BAY,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=PS-I-AD2,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=BABBAGE,CN=Servers,CN=BAY,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=PS-I-AD2,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=PS-BAY-AD1,CN=Servers,CN=BAY,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=OR-VM-1,CN=Servers,CN=Oregon,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=OR-VM-1,CN=Servers,CN=Oregon,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=OR-VM-1,CN=Servers,CN=Oregon,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=IOSLA-DCFS,CN=Servers,CN=CA-LA,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=IOSLA-DCFS,CN=Servers,CN=CA-LA,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=PS-I-AD1,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=PS-I-AD1,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication is in progress:
   From: CN=NTDS Settings,CN=PS-I-AD2,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: The following replication completed successfully:
   From: CN=NTDS Settings,CN=PS-I-AD2,CN=Servers,CN=CA-Orange,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
   To  : CN=NTDS Settings,CN=IOS-SEA-A1,CN=Servers,CN=SEATTLE,CN=Sites,CN=Configuration,DC=iosdomain,DC=local
ALLBACK MESSAGE: SyncAll Finished.
yncAll terminated with no errors.

S C:\Users\administrator.IOSDOMAIN> dcdiag

irectory Server Diagnosis

erforming initial setup:
  Trying to find home server...
  Home Server = IOS-SEA-A1
  * Identified AD Forest.
  Done gathering initial info.

oing initial required tests

  Testing server: SEATTLE\IOS-SEA-A1
     Starting test: Connectivity
        ......................... IOS-SEA-A1 passed test Connectivity

oing primary tests

  Testing server: SEATTLE\IOS-SEA-A1
     Starting test: Advertising
        ......................... IOS-SEA-A1 passed test Advertising
     Starting test: FrsEvent
        ......................... IOS-SEA-A1 passed test FrsEvent
     Starting test: DFSREvent
        ......................... IOS-SEA-A1 passed test DFSREvent
     Starting test: SysVolCheck
        ......................... IOS-SEA-A1 passed test SysVolCheck
     Starting test: KccEvent
        ......................... IOS-SEA-A1 passed test KccEvent
     Starting test: KnowsOfRoleHolders
        ......................... IOS-SEA-A1 passed test KnowsOfRoleHolders
     Starting test: MachineAccount
        ......................... IOS-SEA-A1 passed test MachineAccount
     Starting test: NCSecDesc
        ......................... IOS-SEA-A1 passed test NCSecDesc
     Starting test: NetLogons
        ......................... IOS-SEA-A1 passed test NetLogons
     Starting test: ObjectsReplicated
        ......................... IOS-SEA-A1 passed test ObjectsReplicated
     Starting test: Replications
        ......................... IOS-SEA-A1 passed test Replications
     Starting test: RidManager
        ......................... IOS-SEA-A1 passed test RidManager
     Starting test: Services
        ......................... IOS-SEA-A1 passed test Services
     Starting test: SystemLog
        ......................... IOS-SEA-A1 passed test SystemLog
     Starting test: VerifyReferences
        ......................... IOS-SEA-A1 passed test VerifyReferences


  Running partition tests on : ForestDnsZones
     Starting test: CheckSDRefDom
        ......................... ForestDnsZones passed test CheckSDRefDom
     Starting test: CrossRefValidation
        ......................... ForestDnsZones passed test CrossRefValidation

  Running partition tests on : DomainDnsZones
     Starting test: CheckSDRefDom
        ......................... DomainDnsZones passed test CheckSDRefDom
     Starting test: CrossRefValidation
        ......................... DomainDnsZones passed test CrossRefValidation

  Running partition tests on : Schema
     Starting test: CheckSDRefDom
        ......................... Schema passed test CheckSDRefDom
     Starting test: CrossRefValidation
        ......................... Schema passed test CrossRefValidation

  Running partition tests on : Configuration
     Starting test: CheckSDRefDom
        ......................... Configuration passed test CheckSDRefDom
     Starting test: CrossRefValidation
        ......................... Configuration passed test CrossRefValidation

  Running partition tests on : iosdomain
     Starting test: CheckSDRefDom
        ......................... iosdomain passed test CheckSDRefDom
     Starting test: CrossRefValidation
        ......................... iosdomain passed test CrossRefValidation

  Running enterprise tests on : iosdomain.local
     Starting test: LocatorCheck
        ......................... iosdomain.local passed test LocatorCheck
     Starting test: Intersite
        ......................... iosdomain.local passed test Intersite

 

LainRobertson's avatar
LainRobertson
Silver Contributor
Jun 10, 2022

Zalastar 

 

Glad you got it sorted!

 

No, you don't need to restart the service after each change to the Active Directory objects. You can, of course, but it's not mandatory.

 

The DFS-R service checks AD periodically, and if you're in a hurry, you can run the following from the member you're trying working on:

 

 

dfsrdiag pollad

 

That will cause DFS-R to check for changes immediately.

 

Cheers,

Lain

  • Zalastar's avatar
    Zalastar
    Copper Contributor
    Jun 11, 2022

    LainRobertson 

     

    A big thank you for the time you spend here on this thread! I am so happy right now. Not a single error in the Forest. And my Delta sync times are way down now too

     

    Source DSA          largest delta    fails/total %%   error
 BABBAGE                   26m:58s    0 /  10    0
 IOS-SEA-A1                13m:01s    0 /  20    0
 IOSLA-DCFS                08m:24s    0 /  15    0
 OR-VM-1                   07m:06s    0 /  10    0
 PS-BAY-AD1                21m:10s    0 /  10    0
 PS-I-AD1                  23m:23s    0 /  15    0
 PS-I-AD2                  22m:06s    0 /  25    0


Destination DSA     largest delta    fails/total %%   error
 BABBAGE                   21m:14s    0 /   5    0
 IOS-SEA-A1                   :26s    0 /  20    0
 IOSLA-DCFS                13m:02s    0 /  15    0
 OR-VM-1                   03m:01s    0 /  20    0
 PS-BAY-AD1                27m:01s    0 /  15    0
 PS-I-AD1                  22m:08s    0 /  15    0
 PS-I-AD2                  23m:25s    0 /  15    0

     I finally understood what you meant about getting rid of the FRS leftovers

     

     

    PS C:\Users\administrator.IOSDOMAIN> Get-ADObject -Filter { (cn -ne "File Replication Service") } -SearchBase "CN=File Replication Service,CN=System,$((Get-A
DRootDSE).defaultNamingContext)" -SearchScope Subtree | Select-Object -Property objectGUID, objectClass, distinguishedName
PS C:\Users\administrator.IOSDOMAIN>

     

  • Zalastar's avatar
    Zalastar
    Copper Contributor
    Jun 11, 2022
    What is this strange life outside the box you are referring too 😉
  • Zalastar's avatar
    Zalastar
    Copper Contributor
    Jun 10, 2022
    Oh I spoke too sonn 😞

    The DFS Replication service stopped replication on the replicated folder at local path C:\Windows\SYSVOL\domain.

    Additional Information:
    Error: 9003 (The replication group is invalid)
    Additional context of the error:
    Replicated Folder Name: SYSVOL Share
    Replicated Folder ID: 8EC04617-53A4-4148-AB24-4CDD050A1716
    Replication Group Name: Domain System Volume
    Replication Group ID: 860C23C1-6213-4EDC-8AA3-0C4B75F238DA
    Member ID: 43A19C80-D9AB-4DCC-8434-7BC4987D3B5B
  • LainRobertson's avatar
    LainRobertson
    Silver Contributor
    Jun 11, 2022

    Zalastar 

     

    Okay, so I think you mentioned earlier something about the FRS migration status being okay, which leads me to assume that an FRS to DFS-R migration took place at some point?

     

    There's still things we can do but I'm keen to check some things first, as they're all edging closer to that "point of last resort" concept.

     

    Does the following command return anything?

    Get-ADObject -Filter { (cn -ne "File Replication Service") } -SearchBase "CN=File Replication Service,CN=System,$((Get-ADRootDSE).defaultNamingContext)" -SearchScope Subtree | Select-Object -Property objectGUID, objectClass, distinguishedName

     

    Secondly, were you following the process here when you reset DFS-R ealier?

    Force synchronization for Distributed File System Replication (DFSR) replicated sysvol replication - Windows Server | Microsoft Docs

     

    Note: It says D2-like, but it's not actually the older BurFlags process.

     

    Cheers,

    Lain

  • Zalastar's avatar
    Zalastar
    Copper Contributor
    Jun 11, 2022

    LainRobertson 

     

    The migration was done before I inherited the position, fortunately its the only DC in the forest with an issue.

     

    The result of that last command

    objectGUID                           objectClass     distinguishedName
----------                           -----------     -----------------
01c80ee4-dc5a-4bfa-8ddf-2500a62a3e7e nTFRSReplicaSet CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=iosdomain,DC=local
37442159-8097-461b-8466-bbddce3e6723 nTFRSMember     CN=WIN-NAAPRAMLU0A,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=iosdomain,DC=local

     

     

    Yes I did follow that method but I am wondering how long I should wait for replication to finish before changing the flag and moving on to the next step? How do I tell how long my forest replication takes to finish?

     

     

  • LainRobertson's avatar
    LainRobertson
    Silver Contributor
    Jun 11, 2022

    Zalastar 

     

    Okay, the command output is good. You can safely also remove those two leftover objects from the output, which will bring the FRS era completely to a close.

     

    The main thing I didn't want to see - and we didn't, is any kind of direct or indirect reference to IOS-SEA-A1. So, life is good(-ish.)

     

    From the Microsoft Docs article, assuming you're running the commands from IOS-SEA-A1 (while also ensuring things like ADSIEDIT are specifically pointing to IOS-SEA-A1), then for steps 2 and 6, use the following statement to push changes outward from IOS-SEA-A1 to the other domain controllers:

     

    repadmin /syncall IOS-SEA-A1 DC=iosdomain,DC=local /d /e /P

     

    The confirmational events are described in steps 4 and 8, but that doesn't help with your question on how long to wait.

     

    The answer to that is: you don't have to wait since you're only dealing with a single domain controller. So long a you see the events listed from steps 4 and 8, you've done everything correctly.

     

    Of course, it takes SYSVOL some time to repopulate and even then, if you find it fails again as you have before, then we might be at the point where you have to delete the DFS-R database off of IOS-SEA-A1.

     

    While Microsoft does discourage this in favour of the process above, the process above also doesn't fix every kind of issue, meaning that's the direction we're heading.

     

    Still, I'd do the above Docs process once more, being sure to note the events (maybe clear the DFS-R event log first to make things easier to digest and track) before looking to delete the database.

     

    Rest assured, deleting the database isn't all "impending doom", either. Microsoft does note some issues that can arise but they're outweighed by getting DFS-R working again.

     

    The directory the database resides in will throw an access denied if you try to access it, but as per the following article, it really is there (read under the Symptoms heading at the top.) But let's not get ahead of ourselves as maybe the Docs article above will help this time around.

     

    DFSR databases crash on primary member - Windows Server | Microsoft Docs

     

    Cheers,

    Lain

  • Zalastar's avatar
    Zalastar
    Copper Contributor
    Jun 11, 2022

    Ok its my under standing to go through the steps again but running the altered repadmin /syncall command you provided. And just to be clear. I am setting the flag to False on the affected DC which is Seattle , which makes in non authorative, and leaving the others set to True

    Im not understanding how to do this

    " You can safely also remove those two leftover objects from the output, which will bring the FRS era completely to a close."

    Do you mean for me to find and delete them with ADSIEDIT or will following the above process and using that Repadmin command take care of this? i a not understanding where those references in the output came from

    I am running everything on the Seattle controller at the moment. The PDC is the PS-I-AD1 DC but I understand for the most part that the work I am doing needs to be done on the affected DC, which is Seattle


  • Zalastar's avatar
    Zalastar
    Copper Contributor
    Jun 11, 2022

    Ok I ran through the steps and I am beginning to understand what it did

    After the Error 4114 I saw quite a few 4412 errors starting with this one

    The DFS Replication service detected that a file was changed on multiple servers. A conflict resolution algorithm was used to determine the winning file. The losing file was moved to the Conflict and Deleted folder.

    Additional Information:
    Original File Path: C:\Windows\SYSVOL\domain\Policies\{1F60C4D9-39CC-4A52-956E-0715D20D84B2}\GPT.INI
    New Name in Conflict Folder: GPT-{D9CFBC30-0905-453D-A1F6-403F6DF348CF}-v166.INI
    Replicated Folder Root: C:\Windows\SYSVOL\domain
    File ID: {6CF9F740-471F-4A0A-99D6-EA74625846E9}-v66
    Replicated Folder Name: SYSVOL Share
    Replicated Folder ID: 8EC04617-53A4-4148-AB24-4CDD050A1716
    Replication Group Name: Domain System Volume
    Replication Group ID: 860C23C1-6213-4EDC-8AA3-0C4B75F238DA
    Member ID: 43A19C80-D9AB-4DCC-8434-7BC4987D3B5B
    Partner Member ID: 806D1444-4493-4ADF-8A0F-DA75BBDF6FA9

     

    and finally ended up with the magic error 4604

    The DFS Replication service successfully initialized the SYSVOL replicated folder at local path C:\Windows\SYSVOL\domain. This member has completed initial synchronization of SYSVOL with partner OR-VM-1.iosdomain.local.  To check for the presence of the SYSVOL share, open a command prompt window and then type "net share". 
 
Additional Information: 
Replicated Folder Name: SYSVOL Share 
Replicated Folder ID: 8EC04617-53A4-4148-AB24-4CDD050A1716 
Replication Group Name: Domain System Volume 
Replication Group ID: 860C23C1-6213-4EDC-8AA3-0C4B75F238DA 
Member ID: 43A19C80-D9AB-4DCC-8434-7BC4987D3B5B 
Sync partner: OR-VM-1.iosdomain.local
  • Zalastar's avatar
    Zalastar
    Copper Contributor
    Jun 11, 2022

    LainRobertson 

     

    and thank you sir

    PS C:\Users\administrator.IOSDOMAIN> net share

Share name   Resource                        Remark

-------------------------------------------------------------------------------
C$           C:\                             Default share
IPC$                                         Remote IPC
UDP_APM$     C:\Program Files\Arcserve\Unified Data Protection\APM

ADMIN$       C:\Windows                      Remote Admin
NETLOGON     C:\Windows\SYSVOL\sysvol\iosdomain.local\SCRIPTS
                                             Logon server share
SYSVOL       C:\Windows\SYSVOL\sysvol        Logon server share
The command completed successfully.

PS C:\Users\administrator.IOSDOMAIN>
  • LainRobertson's avatar
    LainRobertson
    Silver Contributor
    Jun 11, 2022

    Zalastar 

     

    Sorry, I was just out doing the Saturday morning run-around with the family.

     

    Glad it's all sorted!

     

    Cheers,

    Lain

Share