Forum Discussion
OneDrive Client, Files on Demand and Syncing large libraries
I thought I'd post some observations regarding the OneDrive sync client we've observed that aren't documented anywhere but we needed to figure out when planning a massive move to SharePoint from on-p...
Its likely the rate of change that's causing you the most problems. Every time something changes (especially in a file copy scenario), OneDrive has to re-scan everything that's on disk to ensure that it hasn't lost a file somewhere (i.e. "Was this thing renamed? or was it moved? or both?"). When multiple machines are moving a lot around, it exponentially increases the local demand on each individual machine. The only way to alleviate this is to break up the content across multiple libraries. Do all 100,000+ files need to all be in the same library? do they have to all be brought down locally? if you are lucky enough to answer no to both of those questions, the solution is simple: break up the content into smaller libraries and make stale data online-only (disable sync through in the library settings).
Unfortunately, this isn't exactly OneDrive's fault, Windows still has no mechanism where an app could simply subscribe to a subsystem and say "tell me anytime something changes on disk, including all the context". That leaves it with no other alternative than to literally go searching the disk every time anything updates to make sure it hasn't lost anything... but if it gets overwhelmed with too many changes it simply cant keep up.
Ok thanks. That was the direction I was heading by either choosing folders/files not to sync or archiving/moving files into a different Document Library to cut down on the files that sync.
- Splitting content in multiple library could help if you think about active/inactive content or archive content. But if you split 100 000 in 10 library of 10k doc each and sync all 10 library you will face the same issues. Unfortunately this problem still présent years after years and Microsoft has not done anything to fix that. Your only real solution is to train user use Office Apps or Web interface to work with documents.
