OneDrive Client, Files on Demand and Syncing large libraries
I thought I'd post some observations regarding the OneDrive sync client we've observed that aren't documented anywhere but we needed to figure out when planning a massive move to SharePoint from on-premise file servers: Limits: Microsoft documents that you shouldn't sync more than 300,000 files across all libraries that the client is connected to, but there was no documentation about Files on Demand limits, and we have observed the following: The OneDrive client will fail when the dat file that stores object metadata reaches exactly 2GB in size (%localappdata%\Microsoft\OneDrive\settings\Business1). Now, while Microsoft says you shouldn't sync more than 300,000 files, you can connect using files on demand to libraries that contain more than this. The trick here is that in this case, the total number of files and folders matter, lets call them collectively "objects". (Interestingly, when you first connect to a library and the client says "Process changes" and gives you a count, "changes" is the total number of objects in the library that it's bringing down using files on demand and storing in the dat file.) My suspicion is that since the OneDrive client is still 32bit, it's still subject to certain 32bit process restrictions, but I don't really know. What matters in this case is that up until build 19.033.0218.0009 (19.033.0218.0006 insiders build), the client would fill up the dat file and reach the 2GB limit after about 700-800,000 objects. After build 19.033.0218.0009, it appears that the client has been optimized and no longer needs to store quite as much metadata about each object, "increasing" the upper limit of files on demand. (It seems that in general, each object takes up just over 1KB of data in the dat file, putting the limit somewhere just under 2 million objects). Keep in mind, this is not per library, this is across all libraries, including OneDrive for Business (personal storage), SharePoint Document Libraries, etc. Performance: The client has made some significant improvements in performance quickly as they refine each new build, but there are some things to be aware of before you start connecting clients to large libraries: It. takes. forever. The more objects in a library, the longer it's going to take for the client to build it's local cache of files on demand copies of all the items in the library. It seems that in general, the client can process about 50 objects per second, if you were connecting to a library or multiple libraries that had 1.4 million objects, it will take around 8 hours before the client is "caught up". During the time that the content is being built out locally, Windows processes will also consume a large quantity of system resources. Specifically, explorer.exe and the Search Indexer will consume a lot of CPU and disk as they process the data that the client is building out. The more resources you have, the better this experience will be. On a moderately powered brand new Latitude with an i5, 8GB of Memory and an SSD OS Drive, the machine's CPU was pretty heavily taxed (over 80% CPU) for over 8 hours connecting to libraries with around 1.5 million objects. On a much more powerful PC with an i7 and 16GB of memory, the strain was closer to 30% CPU, which wouldn't cripple an end user while they wait for the client and Windows to finish processing data. But, most organizations don't deploy $2000 computers to everyone, so be mindful when planning your Team-Site automount policies. Restarts can be painful. when the OS boots back up OneDrive has to figure out what changed in the libraries in the cloud and compare that to it's local cache. I've seen this process take anywhere from 15 minutes to over an hour after restarts, depending on how many objects are in the cache. Also, if you're connected to a large number of objects in the local cache, you can expect OneDrive to routinely use about a third of CPU on an i5 processor trying to keep itself up to date. This doesn't appear to interfere with the overall performance of the client, but it's an expensive process. Hopefully over time this will continue to improve, especially as more organizations like mine move massive amounts of data up into SharePoint and retire on premise file servers. If I had to make a design suggestion or two: - If SharePoint could pre-build a generic metadata file that a client could download on first connection, it would significantly reduce the time it takes to set up a client initially. - Roll the Activity Log into an API that would allow the client to poll for changes since the last restart (this could also significantly improve the performance of migration products, as they wouldn't have to scan every object in a library when performing delta syncs, and would reduce the load on Microsoft's API endpoints when organizations perform mass migrations) - Windows to the best of my knowledge doesn't have a mechanism to track changes on disk, i.e. "what recursively changed in this directory tree in the last x timeframe", if it were possible to do this, Windows and SharePoint could eliminate most of the overhead that the OneDrive client has to shoulder on it's own to keep itself up to date. Speaking to OneDrive engineers at Ignite last year, support for larger libraries is high on their radar, and it's apparent in this latest production release that they are keeping their word on prioritizing iterative improvements for large libraries. If you haven't yet started mass data migrations into SharePoint, I can't stress enough the importance of deeply analyzing your data and understanding what people need access to and structuring your libraries and permissions accordingly. We used PowerBI to analyze our file server content and it was an invaluable tool in our planning. Happy to chat with anyone struggling with similar issues and share what we did to resolve them. Happy SharePointing! P.S., shoutout to the OneDrive Product Team, you guys are doing great, love what you've done with the OneDrive client, but for IT Pros struggling with competing product limits and business requirements, documenting behind the scenes technical data and sharing more of the roadmap would be incredibly valuable in helping our companies adopt or plan to adopt OneDrive and SharePoint.
Urgent Complaint – OneDrive Disrupted My Local Files and Workflow
Dear Microsoft Support Team, I am writing to formally express my deep frustration with how OneDrive disrupted my workflow and caused significant inconvenience. I never expected a simple cloud storage service to override my local files and create such a chaotic situation without my explicit permission. The Problem Without my consent, OneDrive moved my local files into the cloud, making them unavailable offline and disrupting the entire file structure on my computer. I had no idea where my documents were stored, leading to confusion and anxiety. Additionally, this unexpected synchronization slowed down my computer considerably, making it almost unusable while it processed thousands of files. This was an unnecessary burden, especially considering that I never intended for OneDrive to take control over my local data. Time and Financial Loss The impact of this issue has been severe: Countless hours wasted trying to understand what had happened to my files. Increased anxiety due to the fear of losing critical documents. Technical support costs incurred to troubleshoot and restore my system. Productivity loss while waiting for the computer to finish syncing thousands of files I never asked to be moved. Request for Change A service like OneDrive should never move local files without explicit user approval. The default behavior should be copying files to the cloud, not transferring them away from the local device. I urge Microsoft to: Ensure OneDrive does not alter local files without user confirmation. Provide a clearer setup process that allows users to choose exactly how their files are managed. Make the reversal of such changes easier, as restoring files to their original location is currently a tedious process. This issue has caused me considerable stress, financial loss, and wasted time, and I expect Microsoft to address this matter seriously. I appreciate your time in reviewing this complaint and would like to hear how Microsoft plans to improve this aspect of OneDrive. Sincerely, Leonardo
Sync performance and the future of OneDrive sync
Many of us have experienced significant performance issues with OneDrive when syncing large volumes of files (300,000+). This has been a long-standing problem. Has there been any communication from Microsoft regarding the future of OneDrive sync and plans to resolve this issue? We do not encounter this level of performance degradation in competing products, and it has been a major headache for our customers. I understand that the recommended approach is to use OneDrive and SharePoint through the web interface. However, we do not always have control over how our customers choose to use their data. In many cases, our customers prefer to utilize the Sync feature for syncing hundreds of thousands of files or more. What actions are being taken on the development side to address this issue?
Can't empty recycle bin in onedrive
I recently have started having OneDrive storage space issues. I have spent hours scouring message boards but none of them apply to my problem. The problem is that I have tried emptying the recycle bin using "advice" from other posts but there is NO button in my onedrive that says "empty recycle bin", only "restore". it says i am over my limit by 195% i know i'm not always the brightest star in the sky but unless i am a complete idiot, there is something wrong. please help, this is very frustrating update: I was finally able to find an "empty recycle button" but it was only available in the section where it said my account had been frozen due to lack of space. I clicked on it and it has been "retrieving the files" for 10 minutes now. something is not right.
Student account storage update?
Since Monday, 2/19/2024, I have been receiving emails on Outlook regarding my OneDrive storage. This is a student OneDrive account, which should provide 1TB of cloud storage access for a lifetime. Have there been any updates to the policy that might have changed it to 100GB, or is this possibly just a glitch in the system?GPO to sync SharePoint library, Online files Only?
I need to sync a document library to all 2000 PC/Mac devices in a company. The library will hold <30 documents so not a large library. I don't want to setup 2000 devices manually. Now Microsoft have recently published a way to configure The OneDrive Client to automatically sync a SharePoint Library via GPO. I look at the documentation for the GPO and it states: "This setting allows you to specify SharePoint team site libraries to sync automatically the next time users signs in to the OneDrive sync client (OneDrive.exe). It may take up to 8 hours after a users signs in before the library begins to sync. To use the setting, you must enable OneDrive Files On-Demand, and the setting applies only for users on computers running Windows 10 Fall Creators Update or later. Do not enable this setting for the same library to more than 1,000 devices. This feature is not enabled for on-premises SharePoint sites. If you enable this setting, the OneDrive sync client will automatically sync the contents of the libraries you specified as online-only files the next time the user signs in. The user won't be able to stop syncing the libraries. If you disable this setting, team site libraries that you've specified won't be automatically synced for new users. Existing users can choose to stop syncing the libraries, but the libraries won't stop syncing automatically. To configure the setting, in the Options box, click Show, and then enter the library ID in the Valuefield and a friendly name to identify the library in the Value Name field. To find the library ID, sign in as a global or SharePoint admin in Office 365, browse to the library, and click the Sync button. In the "Starting sync" dialog box, click the Copy library ID link. Enabling this setting sets the following registry key: [HKCU\Software\Policies\Microsoft\OneDrive\TenantAutoMount]"LibraryName"="LibraryID" Now, Is there a way to sync the library so that the files are available offline too? What is the reason behind the 1000 devices limit?
OneDrive for Business NGSC limits for SharePoint Sync - when will these be raised?
Guys n Gals, looking for some roadmap assistance here if possible. Been asking for this for a while now in relation to a client requirement. Pleased to see that the OneDrive for Business NGSC updates are now gonna provide the ability to sync with SharePoint Document Libraries. Great stuff. What I need to know however is when the limit is going to be raised on the 20,000 files that can be synched? I have a client with something in the region of 35,000 files which are currently stored using a competitors online storage and sync facility - total space is negligible - but the limit on number of files to sync is currently a deal breaker that prevents us migrating them to using SharePoint as the central file store and using OneDrive for Business to sync the files to the client machines. We need a solution in place *prior* to next Spring. We're hoping for confirmation that the arbitrary limit that's currently there will be gone with a [hopefully near!] future incarnation of the NGSC. Any updates from those in the know? Nothing we can see at present on the roadmap... David Elders Hosted IT Edinburgh, Great Britain
