Forum Discussion
OneDrive Client, Files on Demand and Syncing large libraries
dustintadamI'm seeking feedback on a recent mass file migration I attempted from OneDrive to Sharepoint. I'm a video creator for my organization. I moved a large amount of media files - videos/photos - from my company OneDrive to a company Sharepoint site. My colleague (also a video creator) and I both sync Sharepoint back to our OneDrives, so this was an attempt to create a more collaborative, cloud-based workflow - rather than each of us having our work live in silo-ed OneDrive folders. I'm now realizing I may have overtaxed the platform, and it's trying to slowly play catch up. Because, a week later, my OneDrive client is still "preparing to upload" and certain files that I now try to add to OneDrive are stuck in a perpetual "Syncing" state.
The other byproduct of this attempt is that I now see a large number of duplicate files with a suffix ID in the filename based on my device's name.
I did not consult this site or others like it for advice before making this migration - obviously. Can anyone with a similar experience offer advice?
M1Storyteller I had a similar issue when migrating a lot of data or re-distributing data over my 5 accounts since 1TB is not enough in todays time. (I had finally uploaded 6TB over my slow connection to my old provider when they announced the end of the service... highly annoying).
There seems to be no clear way.
If there is a suffix ID it is always a duplication, so these files can usually be deleted. The platform detects a change in timestamp or file size and duplicates it. This could be due to a auto save function in the software you are using or simply a problem when copying from one file system onto another.
It is highly annoying that this does not work as it supposed to.
First option:
Try resetting the OneDrive database (there are a number of tutorials online) and see if this fixes the issue. Let the computer online for several days and see if this resolves it, don't make an file changes.Sometimes it works, sometime not...
If not, I would do the following:
- Make backup of all files, make sure you have a "Master" where you know you have ALL files. Clean up duplicate files
- Disconnect the OneDrive service on you PC
- Delete all files in your local OneDrive
- Delete all files in the Online OneDrive
- Copy only one file or folder after another. Always wait until syncing has finished. In this time, I would not use it as a working directory (I know this is annoying...).
I actually have done it with a work around as well: I left the files where they where and synced to a empty folder. Now place a Symlink to the first folder at the original place. Let it sync. Now do the next one and so on.
If you want, you can replace the Symlink with the actual folders. Just make sure you and Onedrive is offline when you do the moving around (Remove symlink, place original folder inside Sync folder)
What I found is simple: OneDrive does not like too many files changed at once and it seems to have more of an issue with plenty of small files than with fewer large files.
Also there is a path length restriction, so I placed the Onedrive folder on the top level of a drive letter I created specially for OneDrive. Since I have multiple account, I need to be at least one folder deep, the OneDrive folder is in a one letter folder 🙂
Like: W:\M\Onedrive
Where "Onedrive" is the automatically created folder by OneDrive
Other Accounts follow the same rule with other letters. I have multiple log-ons on the computer with each Account syncing its own folder.
Again, with Symlinks, I pull the various places/ accounts into one folder as a working folder on my PC. Having multiple account (Family) will help to have smaller databases on each account which so far works for me.So I have all the Software and Backup on my Onedrive account, my Wife has all the Pictures and private files. My kids have all my music and videos.
Each have a folder with all data pulled in one folder.
Since I had one account left, I created one OneDrive account for my relatives into which we have all common pictures from all the family gatherings or holidays or trips we made together. And is also used as exchange folder to share larger files by e-mail. In this way I do not need to have a share in my private files (location).
One thing I forgot:
I almost never switch off m PC nowadays, otherwise Onedrive seems to be unable to catch up... From a security point of view, this is of course a terrible practice...
