"OneDrive Is Updating Files" never ends and has high CPU

%3CLINGO-SUB%20id%3D%22lingo-sub-1564936%22%20slang%3D%22en-US%22%3E%22OneDrive%20Is%20Updating%20Files%22%20never%20ends%20and%20has%20high%20CPU%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1564936%22%20slang%3D%22en-US%22%3E%3CP%3EI%20have%20two%20machines%20where%20one%20has%20a%20large%20bunch%20of%20files%20(ca.%20360%20GB)%20while%20the%20other%20machine%20has%20only%20a%20small%20fraction%20of%20them%20downloaded.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAfter%20reaching%20some%20point%20OneDrive%20does%20stop%20syncing%20anything.%20When%20looking%20at%20the%20machine%20with%26nbsp%3B%20high%20CPU%20I%20find%20that%20most%20time%20is%20spent%20in%26nbsp%3BKernelBase.dll!LCMapStringW%20by%20the%20sync%20engine.%26nbsp%3B%3C%2FP%3E%3CP%3EProcmon%20and%20ETW%20shows%20what%20happens%3A%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3COL%3E%3CLI%3EEnumerate%20a%20file%20on%20my%20drive%3C%2FLI%3E%3CLI%3ECall%26nbsp%3BLCMapStringW%20with%20the%20file%20name%3C%2FLI%3E%3CLI%3Eread%2Fwrite%20SyncEngine-2020-08-03.2209.25372.179.aodl%26nbsp%3B%3C%2FLI%3E%3CLI%3EGoto%201%3C%2FLI%3E%3C%2FOL%3E%3CP%3EBetween%20each%20file%20enumeration%20ca.%202.5s!%20are%20spent%20in%26nbsp%3BLCMapStringW%20and%20sync%20engine.%20That%20is%20an%20insane%20amount%20of%20CPU%20to%20walk%20through%20the%20file%20system.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20aodl%20file%20seems%20to%20be%20some%20encrypted%20SQL%20database%20which%20I%20cannot%20read.%20Has%20OneDrive%20for%20business%20scalability%20issues%20with%20many%20small%20(ca.%2010K)%20files%3F%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1564936%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EPerformance%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1564992%22%20slang%3D%22en-US%22%3ERe%3A%20%22OneDrive%20Is%20Updating%20Files%22%20never%20ends%20and%20has%20high%20CPU%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1564992%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F748890%22%20target%3D%22_blank%22%3E%40AloisKraus%3C%2FA%3E%26nbsp%3BI've%20seen%20performance%20issues%20when%20tracking%20tens%20of%20thousands%20of%20constantly%20changing%20files.%20However%20I%20have%20found%20that%20a%20number%20of%20the%20computers%20with%20problems%20had%20quite%20old%20versions%20of%20OneDrive%20that%20were%20perhaps%20not%20as%20well%20optimised.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20raw%20size%20of%20the%20files%20is%20probably%20less%20interesting%20than%20the%20number%20of%20files%20and%20the%20rate%20of%20change.%20However%20I%20think%20there%20is%20definitely%20a%20long%20way%20to%20go%20with%20OneDrive%20efficiencies%20e.g.%20I've%20noticed%20my%20OneDrive%20tracking%20thousands%20of%20changes%20in%20a%20folder%20%3CEM%3Ethat%20is%20not%20synced%20to%20that%20computer.%3C%2FEM%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20synchronization%20limits%20are%20easily%20found%20via%20web%20search%20e.g.%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Foffice365%2Fservicedescriptions%2Fsharepoint-online-service-description%2Fsharepoint-online-limits%23%3A~%3Atext%3DYou%2520can%2520sync%2520up%2520to%252020%252C000%2520items%2520in%2520a%2520OneDrive%2CThis%2520includes%2520folders%2520and%2520files.%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Foffice365%2Fservicedescriptions%2Fsharepoint-online-service-description%2Fsharepoint-online-limits%23%3A~%3Atext%3DYou%2520can%2520sync%2520up%2520to%252020%252C000%2520items%2520in%2520a%2520OneDrive%2CThis%2520includes%2520folders%2520and%2520files.%3C%2FA%3E%26nbsp%3B%20but%20YMMV%20according%20to%20software%20versions%20etc.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1565214%22%20slang%3D%22en-US%22%3ERe%3A%20%22OneDrive%20Is%20Updating%20Files%22%20never%20ends%20and%20has%20high%20CPU%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1565214%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F235411%22%20target%3D%22_blank%22%3E%40Mike%20Williams%3C%2FA%3E%3A%20300K%20files%20is%20not%20that%20much.%20My%20main%20workload%20are%20memory%20dumps%20and%20ETL%20files%20which%20comes%20with%20thousands%20of%20pdbs%20per%20zip.%20The%20previous%20product%20I%20used%20was%20Synplicity%20which%20allowed%20me%20to%20skip%20.pdb%2C%20.dmp%2C%20.etl%20from%20syncing.%20I%20always%20get%20compressed%20files%20so%20I%20can%20easily%20sync%20the%20compressed%20files%20and%20unzip%20them%20locally.%26nbsp%3B%3C%2FP%3E%3CP%3EWith%20OneDrive%20For%20Business%20the%20world%20was%20simplified.%20Now%20everything%20is%20synced%20and%20I%20cannot%20prevent%20it.%20I%20did%20ask%20our%20IT%20and%20they%20said%20that%20feature%20is%20there%20but%20only%20at%20tennant%20level.%26nbsp%3B%3C%2FP%3E%3CP%3EIn%20the%20new%20world%20it%20looks%20like%20I%20need%20to%20copy%20the%20data%20again%20somewhere%20else%20and%20uncompress%20them%20before%20OneDrive%20gets%20a%20chance%20to%20overflow.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EHow%20can%20I%20check%20with%20OneDrive%20how%20many%20files%20I%20have%20in%20the%20cloud%3F%20The%20Storage%20explorer%20only%20shows%20files%20sizes%20but%20not%20counts.%26nbsp%3B%3C%2FP%3E%3CP%3EIf%20I%20look%20at%20the%26nbsp%3BObfuscationStringMap.txt%20file%20I%20find%20174647%20lines%20which%20should%20mean%20that%20I%20have%20at%20least%20so%20many%20files.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1566085%22%20slang%3D%22en-US%22%3ERe%3A%20%22OneDrive%20Is%20Updating%20Files%22%20never%20ends%20and%20has%20high%20CPU%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1566085%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F748890%22%20target%3D%22_blank%22%3E%40AloisKraus%3C%2FA%3E%26nbsp%3Bwrote%3C%2FP%3E%3CP%3E%22%3CSPAN%3ENow%20everything%20is%20synced%20and%20I%20cannot%20prevent%20it.%20I%20did%20ask%20our%20IT%20and%20they%20said%20that%20feature%20is%20there%20but%20only%20at%20tennant%20level.%20%22%20You%20can%20choose%20which%20folders%20to%20sync%20at%20the%20client%20level.%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%3E%22How%20can%20I%20check%20with%20OneDrive%20how%20many%20files%20I%20have%20in%20the%20cloud%3F%26nbsp%3B%22%20Sure%2C%20I'll%20google%20that%20for%20you%20too%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fanswers.microsoft.com%2Fen-us%2Fmsoffice%2Fforum%2Fall%2Fonedrive-for-business-how-count-number-of-files%2Fa1c508ff-6eba-422a-8c04-efcc2b285573%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fanswers.microsoft.com%2Fen-us%2Fmsoffice%2Fforum%2Fall%2Fonedrive-for-business-how-count-number-of-files%2Fa1c508ff-6eba-422a-8c04-efcc2b285573%3C%2FA%3E%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1566666%22%20slang%3D%22en-US%22%3ERe%3A%20%22OneDrive%20Is%20Updating%20Files%22%20never%20ends%20and%20has%20high%20CPU%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1566666%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F235411%22%20target%3D%22_blank%22%3E%40Mike%20Williams%3C%2FA%3E%3A%20I%20think%20you%20underestimate%20the%20mental%20capabilities%20of%20paying%20customers.%20The%20googled%20linked%20answer%20was%20valid%20until%20the%20site%20design%20was%20changed.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESee%20attached%20screenshot.%20The%20%22new%22%20way%20is%20to%20open%20the%20context%20menu%20to%20show%20the%20Manage%20Storage%20view%20but%20this%20shows%20only%20file%20sizes%20but%20not%20the%20file%20counts.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EDo%20you%20have%20an%20answer%20to%20my%20question%20which%20is%20not%20outdated%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1566678%22%20slang%3D%22en-US%22%3ERe%3A%20%22OneDrive%20Is%20Updating%20Files%22%20never%20ends%20and%20has%20high%20CPU%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1566678%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F748890%22%20target%3D%22_blank%22%3E%40AloisKraus%3C%2FA%3E%26nbsp%3BYou're%20on%20a%20public%20forum%20and%20just%20get%20other%20members%20of%20the%20public%20like%20me%20who%20don't%20have%20a%20crossword%20puzzle%20within%20reach%20during%20a%20coffee%20break.%20If%20you%20want%20someone%20to%20pay%20attention%20to%20you%20%3CEM%3Eas%20a%20paying%20customer%3C%2FEM%3E%20then%20you%20need%20to%20contact%20Microsoft%20Support.%3CBR%20%2F%3E%3CBR%20%2F%3EI%20found%20the%20answer%20from%20the%20link%20I%20provided%20worked%20quite%20well%20on%20both%20my%20OfB%20account%20and%20personal%20OneDrive.%20The%20Site%20Contents%20does%20show%20%23files%20for%20each%20Doclib.%20It%20is%20not%20the%20same%20page%20as%20the%20site%20Storage%20Metrics.%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
New Contributor

I have two machines where one has a large bunch of files (ca. 360 GB) while the other machine has only a small fraction of them downloaded. 

 

After reaching some point OneDrive does stop syncing anything. When looking at the machine with  high CPU I find that most time is spent in KernelBase.dll!LCMapStringW by the sync engine. 

Procmon and ETW shows what happens:

 

  1. Enumerate a file on my drive
  2. Call LCMapStringW with the file name
  3. read/write SyncEngine-2020-08-03.2209.25372.179.aodl 
  4. Goto 1

Between each file enumeration ca. 2.5s! are spent in LCMapStringW and sync engine. That is an insane amount of CPU to walk through the file system.

 

The aodl file seems to be some encrypted SQL database which I cannot read. Has OneDrive for business scalability issues with many small (ca. 10K) files? 

 

5 Replies
Highlighted

@AloisKraus I've seen performance issues when tracking tens of thousands of constantly changing files. However I have found that a number of the computers with problems had quite old versions of OneDrive that were perhaps not as well optimised.

 

The raw size of the files is probably less interesting than the number of files and the rate of change. However I think there is definitely a long way to go with OneDrive efficiencies e.g. I've noticed my OneDrive tracking thousands of changes in a folder that is not synced to that computer.

 

The synchronization limits are easily found via web search e.g. https://docs.microsoft.com/en-us/office365/servicedescriptions/sharepoint-online-service-description...  but YMMV according to software versions etc.

 

 

Highlighted

@Mike Williams: 300K files is not that much. My main workload are memory dumps and ETL files which comes with thousands of pdbs per zip. The previous product I used was Synplicity which allowed me to skip .pdb, .dmp, .etl from syncing. I always get compressed files so I can easily sync the compressed files and unzip them locally. 

With OneDrive For Business the world was simplified. Now everything is synced and I cannot prevent it. I did ask our IT and they said that feature is there but only at tennant level. 

In the new world it looks like I need to copy the data again somewhere else and uncompress them before OneDrive gets a chance to overflow. 

 

How can I check with OneDrive how many files I have in the cloud? The Storage explorer only shows files sizes but not counts. 

If I look at the ObfuscationStringMap.txt file I find 174647 lines which should mean that I have at least so many files. 

 

 

Highlighted

@AloisKraus wrote

"Now everything is synced and I cannot prevent it. I did ask our IT and they said that feature is there but only at tennant level. " You can choose which folders to sync at the client level.

 

"How can I check with OneDrive how many files I have in the cloud? " Sure, I'll google that for you too: https://answers.microsoft.com/en-us/msoffice/forum/all/onedrive-for-business-how-count-number-of-fil...

Highlighted

@Mike Williams: I think you underestimate the mental capabilities of paying customers. The googled linked answer was valid until the site design was changed. 

 

See attached screenshot. The "new" way is to open the context menu to show the Manage Storage view but this shows only file sizes but not the file counts. 

 

Do you have an answer to my question which is not outdated?

 

Highlighted

@AloisKraus You're on a public forum and just get other members of the public like me who don't have a crossword puzzle within reach during a coffee break. If you want someone to pay attention to you as a paying customer then you need to contact Microsoft Support.

I found the answer from the link I provided worked quite well on both my OfB account and personal OneDrive. The Site Contents does show #files for each Doclib. It is not the same page as the site Storage Metrics.