Forum Discussion
OneDrive Client, Files on Demand and Syncing large libraries
There are a lot of variables involved that can impact sync performance. One thing to note however:
The OneDrive Team has been working on a method to sync individual folders that will be more performant, its related to the notion of "Sync Root". Essentially, SharePoint currently only allows the "Sync Root" to be the root of a Document Library. How this manifests in practice is that the Sync Client will download ALL the metadata regarding the entire library, even if you're only syncing a single subfolder. When they do manage to finish the work to allow SharePoint to set the Sync Root to a different level, you'll see improved performance when synchronizing a single subfolder.
However, another thing to keep in mind (that my organization learned the hard way), is that large libraries place additional pressure on the SQL infrastructure behind the scenes. Even if everything is working, you'll notice that all operations against a larger library will be slower across the board.
We are in the process of breaking up our larger libraries into smaller ones to ensure that we can maintain better performance in web browsing, sync, etc.
dustintadam I am wondering if you could share any updates on your experience with using OneDrive for Business Sync for large libraries. We have a use case where the company is attempting to replace certain network drives used by many users. i.e. For example, a drive with 5,000+ Folders and 7,500+files
Would you be able to offer any thoughts or input on such plan looking forward into 2020? Any input would be greatly appreciated!
- We also work with Microsoft on the issue and here is the last insight I can share.
Microsoft Support engineer answer:
“I understand that the OneDrive sync has performance issues if we try to sync large libraries due to which we recommend syncing no more than 300,000 files across all document libraries. Also, Performance issues can occur if you have 300,000 items or more across all libraries that you are syncing, even if you are not syncing all items within those libraries”
The second sentence make OneDrive sync useless if we are working with large amount of data. You have to think more then twice how you wan to organise your library... “ex: active vs archive contents” where archived contents cannot be sync.
We personally have issue syncing 1 folder containing 1 document in a library containing 250k documents. Sync never end or take forever wich result updated document get re-uploaded/downloaded after 1hour or more.A couple things to look for and consider:
One of the things we've discovered that isn't really documented anywhere is that the more content you shove into a single Document Library, the worse that library performs. Adding additional Indexes to the Library manually can help, but in general, the fewer items you put into a library the better. This becomes apparent even when browsing the library via the UI: a library with fewer total objects browses faster than one with hundreds of thousands. The Sync client will ultimately be affected by that increased overhead as well: when it makes API calls to detect or replicate changes, it's going to take longer to complete. We learned this the hard way ourselves and are actively working on breaking up our libraries. If you haven't migrated data yet, find a way to break up your content into as many libraries as possible to reduce the total volume. Also bear in mind that as is the nature of all file storage, it never gets smaller, nobody ever deletes anything, if you start with an overly large library, your experience will never get better from that point.
I know that the eventual goal is to get the OD Client to gracefully handle syncing up to a million objects, but that hasn't been publicly communicated and there is no timeline for when that might be realized.
