Azure File Sync issues and errors - files/folders not syncing

Copper Contributor

We are testing Azure Files and in particular Azure File Sync currently.  Admittedly, we are throwing a lot of files and shares up as a test.  I have some shares that that show its 'health' with a green checkmark, but upon reviewing the Diagnostic Event Log in the Event Viewer, it appears that there are large numbers of files and directories that are not syncing (in other words they were never uploaded).  I am glad I found the Event Logs otherwise I probably would have assumed everything was uploaded. 

 

1. We are in a mixed Windows/Mac environment, and unfortunately over the years many Mac users were in the habit of naming files with a dot in the name. This is not a period, but a horizontally centered bold dot. I am trying to determine a way to rename these to an underscore in batch, but this is problematic as Windows will not search for that character. 

 

2. Other characters are also giving issues. I believe these issues also occur with OneDrive and I understand the complexity and I'm not complaining. Just making notes for any users going through the same thing. In particular the % sign, and folder names that have a period in the name.  So for instance, we have had users that have named a folder '12. Job Notes'.  The 12 represents the month December, and its just the convention they have used over the years. While the folder creates on the Azure side, the files do not upload. 

 

3. Some files will not sync, and I cannot determine a rhyme or reason.  The file does not appear to be in use or locked in any way. It does not have an invalid character (that I can tell) and the file path doesn't appear to be too long. 

 

Log Name: Microsoft-FileSync-Agent/Diagnostic
Source: Microsoft-FileSync-Agent
Date: 12/14/2017 2:43:35 PM
Event ID: 1101
Task Category: None
Level: Warning
Keywords:
User: SYSTEM
Computer: SERVER.ourdomain.local
Description:
The File Synchronization service failed to upload a file. File name:\\?\E:\Shares\Marketing\Public Relations\New Business - non-automotive\Carrey Della\Carrey-Della Pinterest.docx; Sync ID: {eedf85bb-61ad-4fc9-af5f-e14439f3fd00}; Error: 0x80c80065 - The file has been identified to produce persistent errors during sync. Hence, it is blocked from transfering data until the retry interval is reached.

Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-FileSync-Agent" Guid="{4E816BB7-D169-59A8-A02D-3892859A3B96}" />
<EventID>1101</EventID>
<Version>0</Version>
<Level>3</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x4000000000000000</Keywords>
<TimeCreated SystemTime="2017-12-14T20:43:35.247538400Z" />
<EventRecordID>5219603</EventRecordID>
<Correlation />
<Execution ProcessID="273892" ThreadID="130004" />
<Channel>Microsoft-FileSync-Agent/Diagnostic</Channel>
<Computer>SERVER.ourdomain.local</Computer>
<Security UserID="S-1-5-18" />
</System>
<EventData>
<Data Name="FileName">\\?\G:\Shares\Garage\Public Relations\New Business - non-automotive\Carrey Della\Carrey-Della Pinterest.docx</Data>
<Data Name="SyncId">{EEDF85BB-61AD-4FC9-AF5F-E14439F3FD00}</Data>
<Data Name="HResultStr">0x80c80065 - The file has been identified to produce persistent errors during sync. Hence, it is blocked from transfering data until the retry interval is reached.
</Data>
<Data Name="HResult">-2134376347</Data>
</EventData>
</Event>

 

4.  Lastly, 2 of the larger shares, they do not have a healthy status but instead have a red exclamation point.  If you look at the error you see: The Azure sync database write lease has been lost due to another database writer. The current operation will be aborted.  

 

Interestingly, one of the shares does in fact seem to sync if you create a new file, but the other one does not seem to sync at all anymore (beyond the initial sync).

 

I would like to add that everything has been amazingly easy to set up, and I am really excited about this product. It appears that it will require a culture change (with the Mac users), and at least currently a lot of effort to get what could be thousands of files or folders renamed to something proper. 

1 Reply

It has been awhile since I posted this, but I noticed that I never posted our resolution.  Basically, the special characters were the entire root of the problem.

 

I had found an incredible PowerShell script that helped me find and rename the files.  There are actually two that I used. One just files the files and folders with the special characters, the other one will actually rename the file changing the special character to its closest latin equivalent.  However, sometimes it even runs into characters it cannot change, so I use the first script to find and manually fix (luckily only a handful so far).

To find the files with the special characters

gci -recurse . | where {$_.Name -match "[^\u0000-\u007F]"}

The following searches for the special characters and renames using a valid character: So far this has worked beautifully. 

function Convert-ToLatinCharacters {
param([string]$inputString)
  [Text.Encoding]::ASCII.GetString([Text.Encoding]::GetEncoding("Cyrillic").GetBytes($inputString))
}

$files = gci -recurse | where {$_.Name -match "[^\u0020-\u007F]"}
$files | ForEach-Object {
  $newname = Convert-ToLatinCharacters $_.Name
  $newname = $newname.replace('?','_')
  if ($_.Name -ne $newname) {
    $num=1
    $nextname = $_.Fullname.replace($_.Name,$newname)
    while(Test-Path -Path $nextname)
    {
      $next = ([io.fileinfo]$newname).basename + " ($num)" + ([io.fileinfo]$newname).Extension
      $nextname = $_.Fullname.replace($_.Name,$next)
      $num+=1
    }
    echo $nextname
    ren $_.Fullname $nextname
  }
}

This was found here: https://superuser.com/questions/636247/how-do-i-remove-non-ascii-characters-from-filenames

It took a few days, and things didn't show as correct immediately, or even in several days. But needless to say after a long weekend I looked at everything on a Tuesday I believe and everything was synced properly.