Forum Discussion
Roger Seekell
Nov 23, 2021Brass Contributor
What's the best way to transfer 4TB of files to SharePoint Online site?
I have nearly 4TB of assorted files on a drive to upload to a SharePoint Online site. What's the best/fastest way to upload all these files? (Hopefully, it doesn't involve a paid third-party produc...
- Nov 23, 2021You can use SharePoint Migration Tool (free): https://spmt.sharepointonline.com/install/default.htm
Ben Leach
Nov 24, 2021Copper Contributor
I'd like to second this particular point. I worked with a customer that had uploaded 4TB into a single document library in a single site. They are having so many problems as a result, particularly with syncing and they have no easy way of restructuring this architecture now the data has already been uploaded.
Whatever you do, make sure you plan out your SharePoint structure before uploading. Don't upload multiple department's files into a SharePoint Site. Create sites for each department. If a single department has approaching 100,000 files, then create multiple document libraries.
Once you've then mapped out the correct structure in your set of SharePoint Sites, create multiple migration jobs using CSV and load them into the SharePoint Migration Tool.
I have noticed that migration into SharePoint over the weekend is significantly faster than during the week, as I believe that ingestion to SharePoint is throttled during the working week.
Also, the migration tool allows you to create an initial bulk import job, then setup some incremental jobs to pick up any changes once the bulk of data has been imported.
To address RobElliott's point, I disagree. The SharePoint migration tool is very fast (subject to Microsoft throttling) as it uses the approach of uploading to Azure blob storage behind the scenes, then linking into SharePoint libraries. I've done a tonne of migrations and have been impressed with how good the out of the box SharePoint Migration Tool actually is nowadays
Whatever you do, make sure you plan out your SharePoint structure before uploading. Don't upload multiple department's files into a SharePoint Site. Create sites for each department. If a single department has approaching 100,000 files, then create multiple document libraries.
Once you've then mapped out the correct structure in your set of SharePoint Sites, create multiple migration jobs using CSV and load them into the SharePoint Migration Tool.
I have noticed that migration into SharePoint over the weekend is significantly faster than during the week, as I believe that ingestion to SharePoint is throttled during the working week.
Also, the migration tool allows you to create an initial bulk import job, then setup some incremental jobs to pick up any changes once the bulk of data has been imported.
To address RobElliott's point, I disagree. The SharePoint migration tool is very fast (subject to Microsoft throttling) as it uses the approach of uploading to Azure blob storage behind the scenes, then linking into SharePoint libraries. I've done a tonne of migrations and have been impressed with how good the out of the box SharePoint Migration Tool actually is nowadays
Maggan Wåhlin
Nov 29, 2021Iron Contributor
This Is basically what I wrote earlier, and I do agree!