Migrated files much larger in SharePoint Online than on disk

%3CLINGO-SUB%20id%3D%22lingo-sub-80483%22%20slang%3D%22en-US%22%3EMigrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-80483%22%20slang%3D%22en-US%22%3E%3CP%3EI%20am%20doing%20some%20migration%20testing%20and%20I%20am%20surprised%26nbsp%3Bhow%20much%20larger%20files%20are%20when%20uploaded%20to%20SharePoint%20Online.%20I%20uploaded%20roughly%2080GB%20of%20office%20files%2C%20which%20resulted%20in%20maxing%20out%20the%20storage%20limit%20I%20had%20set%20for%20the%20sitecollection%20at%20200GB.%20Some%20files%20are%20triple%20their%20original%20size%20and%20I%20have%20checked%20there%20is%20only%20one%20version.%20The%20%22storman.aspx%22%20also%20reports%20larger%20size%20than%20the%20Version%20history.%20The%20files%20were%20uploaded%20using%20a%26nbsp%3BSPOMigrationPackage.%20Any%20ideas%20why%20the%20files%20are%20so%20much%20larger%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-80483%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAdmin%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EDocument%20Library%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EFiles%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESharePoint%20Online%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-241200%22%20slang%3D%22en-US%22%3ERe%3A%20Migrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-241200%22%20slang%3D%22en-US%22%3E%3CP%3EI%20have%20discovered%20the%20exact%20same%20issue...%20200kB%20word%20docs%20have%20grown%20after%20sharepoint%20upload%20to%201.4MB%20files!%20There%20are%20no%20extra%20'versions'%20that%20I%20can%20see%2C%20just%20the%20document%20itself%3B%20It%20is%20exactly%20as%20you%20have%20shown.%20I%20uploaded%20simply%20by%20syncing%20a%20folder%20on%20my%20desktop%20to%20the%20site%20library.%3C%2FP%3E%3CP%3EDid%20you%20happen%20to%20find%20out%20what%20is%20going%20on%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-81110%22%20slang%3D%22en-US%22%3ERe%3A%20Migrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-81110%22%20slang%3D%22en-US%22%3E%3CP%3ESmall%20increases%20(e.g.%2010%20kB%20for%20msg%20files)%20are%20expected%20but%20this%20is%20extreme.%3CBR%20%2F%3EI%20have%20only%20seen%20this%20increase%20for%20compressed%20tiff%20files%20where%20the%20size%20sometimes%20increases%205%20folder%20upon%20uploading%20(SharePoint%20uncompresses%20the%20tiff%2C%20inserts%20metadata%20and%20then%20applies%20a%20much%20less%20efficient%20compression%20mechanism).%3CBR%20%2F%3EI%20would%20compare%20the%20contents%20of%20the%20pptx%20before%20and%20after%20uploading.%20rename%20the%20extension%20to%20zip%20and%20then%20unpack%20the%20file.%20This%20should%20provide%20information%20what%20is%20going%20on.%3C%2FP%3E%3CP%3EPaul%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-81097%22%20slang%3D%22en-US%22%3ERe%3A%20Migrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-81097%22%20slang%3D%22en-US%22%3E%3CP%3EYes%20I%20have%20checked%20that%20the%20nr%20of%20files%20and%20folders%20match.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-81068%22%20slang%3D%22en-US%22%3ERe%3A%20Migrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-81068%22%20slang%3D%22en-US%22%3EHave%20you%20also%20checked%20the%20number%20of%20files%20in%20the%20Doc.%20Library%20is%20the%20same%20number%20you%20have%20in%20your%20local%20disk%3F%20I%20would%20also%20review%20if%20there%20are%20any%20checked%20out%20files%20in%20the%20doc.%20libary...as%20you%20said%2C%20it%20seems%20each%20files%20has%20only%20a%20version%20but%20you%20have%20many%20files%20there%20so%20not%20sure%20if%20this%20is%20true%20for%20all%20the%20files%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-81057%22%20slang%3D%22en-US%22%3ERe%3A%20Migrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-81057%22%20slang%3D%22en-US%22%3E%3CP%3ENo%20it%20was%20just%20standard%20test%20office%20files.%20I%20used%20Invoke-SPOMigrationEncryptUploadSubmit%20to%20migrate%20the%20files.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EHere%20are%20some%20examples%3A%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F16239i582A3CEC7A1F1494%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%222017-06-22_07-34-27.png%22%20title%3D%222017-06-22_07-34-27.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20362px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F16240i436F72B5908315BD%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%222017-06-22_07-47-00.png%22%20title%3D%222017-06-22_07-47-00.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20399px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F16241i3B5EA1561150A73D%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%222017-06-22_07-41-52.png%22%20title%3D%222017-06-22_07-41-52.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20552px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F16242i3583E600DD02765D%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%222017-06-22_07-45-46.png%22%20title%3D%222017-06-22_07-45-46.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20900px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F16243iBA0A7C5D07303F17%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%222017-06-22_07-41-25.png%22%20title%3D%222017-06-22_07-41-25.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-81005%22%20slang%3D%22en-US%22%3ERe%3A%20Migrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-81005%22%20slang%3D%22en-US%22%3E%3CP%3EI%20have%20seen%20file%20sizes%20change%20(%3CA%20href%3D%22http%3A%2F%2Ftechtrainingnotes.blogspot.com%2F2010%2F11%2Fsharepoint-changes-files-as-they-are.html%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttp%3A%2F%2Ftechtrainingnotes.blogspot.com%2F2010%2F11%2Fsharepoint-changes-files-as-they-are.html%3C%2FA%3E)%2C%20but%20nothing%20like%20that.%20Any%20unusual%20files%20types%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20wonder%20if%20you%20had%20some%20ZIP%20files%20unpacked%20in%20the%20move%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-80936%22%20slang%3D%22en-US%22%3ERe%3A%20Migrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-80936%22%20slang%3D%22en-US%22%3E%3CP%3EYes%20it%20is%20from%20the%20local%20file%20system.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-80579%22%20slang%3D%22en-US%22%3ERe%3A%20Migrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-80579%22%20slang%3D%22en-US%22%3EFrom%20where%20are%20you%20uploading%20your%20files%3A%20File%20System%3F%20Of%20course%20this%20is%20not%20normal%20behavior%20and%20it's%20totally%20unexpected%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2061865%22%20slang%3D%22en-US%22%3ERe%3A%20Migrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2061865%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F58215%22%20target%3D%22_blank%22%3E%40H%C3%A5vard%20Meling%3C%2FA%3E%26nbsp%3Bdid%20you%20ever%20find%20a%20solution%20to%20this%20issue%3F%26nbsp%3B%20I%20am%20having%20exactly%20the%20same%20problem%20and%20can't%20find%20a%20solution.%26nbsp%3B%20I%20have%20just%20migrated%20all%20our%20data%20across%20to%20SharePoint%20-%20just%20normal%20Office%20files%20in%20Word%20and%20Excel%2C%20PDF%20files%20and%20some%20images.%26nbsp%3B%20Almost%20every%20file%20I%20uploaded%20has%20a%20much%20bigger%20file%20size%20online%20than%20it%20does%20on%20the%20local%20disk.%26nbsp%3B%20I%20have%20a%20backup%2Fsynch%20application%20that%20I%20wanted%20to%20use%20to%20keep%20an%20offline%20backup%20but%20when%20I%20analyse%20changes%20between%20my%20locally%20synched%20SharePoint%20sites%20with%20the%20original%20files%20on%20a%20hard%20disk%2C%20they%20are%20all%20same%20dates%2Ftimes%2Fnames%20-%20but%20totally%20different%20file%20sizes.%26nbsp%3B%20Any%20pointers%20you%20can%20give%20to%20an%20explanation%20greatly%20appreciated.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2063409%22%20slang%3D%22en-US%22%3ERe%3A%20Migrated%20files%20much%20larger%20in%20SharePoint%20Online%20than%20on%20disk%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2063409%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F929515%22%20target%3D%22_blank%22%3E%40chrisvanz%3C%2FA%3E%26nbsp%3B%3CBR%20%2F%3Edid%20you%20compare%20a%20file%20before%20and%20after%20uploading%3F%3CBR%20%2F%3EFor%20Office%20files%20I%20would%20simply%20open%20the%20file%20with%20a%20zip%20tool%20to%20see%20the%20different%20components%20(e.g.%20xml%20files%2C%20images%2C%20...).%20For%20binary%20files%20I%20would%20suggest%20to%20use%20the%20online%20tool%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fhexed.it%2F%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fhexed.it%2F%3C%2FA%3E%3C%2FP%3E%3CP%3ESome%20small%20changes%20are%20expected%20for%20msg%20and%20Office%20files%20but%20they%20should%20be%20in%20the%20order%20of%2010%20kB's.%3C%2FP%3E%3CP%3EPaul%20%7C%20SLIM%20Applications%3C%2FP%3E%3C%2FLINGO-BODY%3E
New Contributor

I am doing some migration testing and I am surprised how much larger files are when uploaded to SharePoint Online. I uploaded roughly 80GB of office files, which resulted in maxing out the storage limit I had set for the sitecollection at 200GB. Some files are triple their original size and I have checked there is only one version. The "storman.aspx" also reports larger size than the Version history. The files were uploaded using a SPOMigrationPackage. Any ideas why the files are so much larger?

10 Replies
From where are you uploading your files: File System? Of course this is not normal behavior and it's totally unexpected

Yes it is from the local file system.

I have seen file sizes change (http://techtrainingnotes.blogspot.com/2010/11/sharepoint-changes-files-as-they-are.html), but nothing like that. Any unusual files types?

 

I wonder if you had some ZIP files unpacked in the move?

 

No it was just standard test office files. I used Invoke-SPOMigrationEncryptUploadSubmit to migrate the files. 

 

Here are some examples:

2017-06-22_07-34-27.png

 

2017-06-22_07-47-00.png

 

2017-06-22_07-41-52.png

2017-06-22_07-45-46.png

 

2017-06-22_07-41-25.png

 

Have you also checked the number of files in the Doc. Library is the same number you have in your local disk? I would also review if there are any checked out files in the doc. libary...as you said, it seems each files has only a version but you have many files there so not sure if this is true for all the files

Yes I have checked that the nr of files and folders match.

Small increases (e.g. 10 kB for msg files) are expected but this is extreme.
I have only seen this increase for compressed tiff files where the size sometimes increases 5 folder upon uploading (SharePoint uncompresses the tiff, inserts metadata and then applies a much less efficient compression mechanism).
I would compare the contents of the pptx before and after uploading. rename the extension to zip and then unpack the file. This should provide information what is going on.

Paul

I have discovered the exact same issue... 200kB word docs have grown after sharepoint upload to 1.4MB files! There are no extra 'versions' that I can see, just the document itself; It is exactly as you have shown. I uploaded simply by syncing a folder on my desktop to the site library.

Did you happen to find out what is going on?

@Håvard Meling did you ever find a solution to this issue?  I am having exactly the same problem and can't find a solution.  I have just migrated all our data across to SharePoint - just normal Office files in Word and Excel, PDF files and some images.  Almost every file I uploaded has a much bigger file size online than it does on the local disk.  I have a backup/synch application that I wanted to use to keep an offline backup but when I analyse changes between my locally synched SharePoint sites with the original files on a hard disk, they are all same dates/times/names - but totally different file sizes.  Any pointers you can give to an explanation greatly appreciated.

@chrisvanz 
did you compare a file before and after uploading?
For Office files I would simply open the file with a zip tool to see the different components (e.g. xml files, images, ...). For binary files I would suggest to use the online tool https://hexed.it/

Some small changes are expected for msg and Office files but they should be in the order of 10 kB's.

Paul | SLIM Applications