SOLVED

Moving or Copying Files: Painfully Slow, Loss of Data

Copper Contributor

Hello All! 

 

Our company transitioned over to SharePoint Online about a year ago.

 

One of the largest complaints I have to date, is the time it takes to move or copy files. In cases of one or two files, the time is minimal. But in cases where larger file moves are taking place - the pace of transfer is painfully slow. 

 

For example, today it took approx. 30 mins to transfer only 45.8 MB. 

 

Earlier this month, an employee spent the entire day waiting for files to transfer, only for her computer to stop responding, and have to re-start the entire process the next day. On our previous server based environment, the same transfer would take less than 30 minutes. 

 

Additionally, in some of these larger file moves, users have reported large losses of data. The files were moved, and confirmed, only to come in the next day and they have disappeared. 

 

This issue is decreasing adoption and causing our IT team multiple headaches.

 

Any suggestions or helpful feedback?

55 Replies
Why don't they use the option to move files using the ODFB Sync client? I think the problem you are facing is due to the fact the Copy and Move function are not intended for doing massive Copy/Move operations

Is this something which has only just start occurring? If so it could be local network/connection issues, or you can never rule out that it's something that's being worked on cloud-side.

 

Alternatively, if this is part of a migration effort then you could be being throttled, especially if you're firing a lot of content up at the same time.

 

I'd be thinking along the lines of @Juan Carlos González Martín in suggesting that your users exploit the OneDrive for Business Sync Client, so that the files can be uploading in the background while your users go about their daily routine.

Hi Hillary,

I am assuming you are uploading files to SharePoint. In that case 46 MB in 30 mins is very slow.

Do you upload files via the browser or via explorer view?
Are the files very small (e.g. emails of a few kB each)?
Are you uploading files to a remote location (e.g. other continent) where network latency may negatively impact throughput?
What is a large file? is that 100 MB or several GB's?

ODFB is an option but it means you will have a local copy of the files which needs to be removed (in the proper way) at some point in time. In case you just want to upload files this is cumbersome.

There are plenty of tools to help out. See e.g. https://directory.collab365.community/office365-sharepoint/office-365-migration-software/

My company has developed a browser-based solution (listed as SLIM Companion Migration Manager) to address upload issues by using parallel processes.
Paul (SLIM Applications)

Hi @Matt Weston, No this has been ongoing since we transitionted to SharePoint online - not a new problem. 

Hi @Paul de Jong,

 

We actually arent uploading files to SharePoint - the users are moving files within SharePoint.

 

Currently our users prefer the "open in explorer" functionality - so they are copying files from one location to another in this setting. This is where all the slowness is occuring. I have tried to copy and move from the browser - same problem, slow pace.

 

File size varies, some of these data moves have hundreds of files with various sizes - it varies alot.

 

All files are being moved to the same company location (Vancouver, Canada) - not external. 

 

Large file - good question - every data move is different, but on average the total size ranges from 45 MB  to 256 MB and the largest I have seen is 1.86 GB (this size is extremely rare).

 

How does the one drive sync work exactly?

best response confirmed by Hillary Barter (Copper Contributor)
Solution

If you're using "Open in Explorer" I think that may be part of the issue, and I'm assuming that you're using Internet Explorer. From personal experience I never really got on with the piece of functionality. It worked well back in 2007 when I first used it, but now there are much better options available.

 

Option 1 - Move To

When you visit a document library in the modern experience, you can click on the ellipsis against the document and select move to (see attachment "Modern-MoveTo.docx". In Figure 1, you can see the option to move to, and then in Figure 2 you can see that you have a number of options to either move it to OneDrive for Business or to another SharePoint site. When I tested this with a document of approx 1mb it took about 5 seconds to get itself warmed up and then do the move.

 

Option 2 - OneDrive Sync

The alternative option which has been mentioned before is to use sync (see attachment Modern-Sync.docx). In Figure 1, you can see the ability to Sync your library to your local file system using the OneDrive Sync Client.  By hitting the sync button, you'll see it connect to your sync client (Figure 2), and then it will be available from your Windows Explorer windows (Figure 3). Once it's been synced, you can copy and paste documents between synced libraries in the same way as you would with normal files. Again this took only a few seconds to copy, paste and sync between the libraries.

 

I hope this approaches are useful and that they work. If not, please let us know and we'll see what else we can come up with.

Hi @Matt Weston

 

thanks for the help, when i attempted to complete the "sync" option, the following error pops up and doesnt allow it to happen.

 

Any ideas?

Hi @Hillary Barter I think I've made an incorrect assumption that you're working in SharePoint Online rather than using the modern lists and libraries in SharePoint 2016. The ODFB Sync client unfortunately doesn't yet support syncing of on-prem libraries. Were you successful when trying the "move to" option?

Hi @Matt Weston,

 

We are using SharePoint Online - we are not using on premise services. 

 

You were not incorrect, we are using the online SP, and we should be able to sync, no?

I dont know why it is saying the "on-prem" error, it makes no sense!

This is quite strange and I'm starting to run short on ideas. The only thing I can think of is that your Sync Client isn't up to date?

 

https://onedrive.live.com/about/download/

 

You could always try grabbing the latest version and see if that clears it.

Hi Matt,

 

So I figured out the problem. When I was attempting to sync it was running from my "administrator" account, rather than my personal - so the sharepoint site to onedrive accounts did not match up - if that makes sense.

 

When i accessed the data from my personal sharepoint account, and hit sync - it worked perfectly.

 

I am curious to know - what is best practice for working with the sync'd library instead of the SP Online browser experience. Is this recommended so users can have the familar windows explorer view? Or is it better used as simply a sync client for large copys or moves as previously discussed? I'd be keen to know the main issues around using the one drive sync client exclusively (and just syncing all the main document libraries you use on a regular basis).

 

thanks for all of your help

It's difficult to suggest a best practice as it comes down to the ways in which your users like to work. In my own business unit I have some who like to only work in the browser, and someone who only like to work with synced folders. You can explain to your users how they can do it, and then they find their own way. If I'm working with complete O365 novices, then I like the synced folder approach as it's very very similar to what they've always done.

 

That's one of the things which I like about O365 so much, each user can find their own way of working which is most comfortable and most efficient for them.

thanks for all of your help!

Hi

 

I'm still looking for the optimal way of transferring the content of a document library between sharepoint sites.

I'm using Sharepoint Online (Office 365). The originating document library is a modern experience library and belongs to a modern experience team site which is a subsite of a modern experience communication site collection. The receiving document is also a modern experience library belonging to a modern experience teams site, a subsite of a modern experience team site collection.  The content consists of multiple files and folders (numbers in thousands but always less than 5000 per folder). Individual files can be very large (a few gigabites) and the total size of the library is in tens of gigabites.

 

Using the OneDrive sync client requires a download (and then upload) of a local copy of all content of the document library. This is not a good option as it requires puts demands on local resources such as bandwidth and computer.  Also the sync does not move the version history, only the newest version. The sync takes a long time and sometimes nothing happens for a long time with the message "sync delays caused by service load". So basically not an option.

 

The "Move To" feature found in the library takes a very long time and worse, when selecting all folders at the top level of the library and selecting "Move To" it actually does not move everything. After 24hrs. of processing (this was not a very big library) some content was left behind in the old library. So basically not an option.

 

Using a 3rd party tool (assuming there is such a tool that can move content of very large document libraries between site collections) is not an option due to cost.

 

Any other options you can point out?

 

Regards,

Birgir

Recently I attempted to move around some files from the document library of my Team Site to a sub-site with some unique permissions (which is our goal, to create a better secured document structure). While the "copy to" option works fine (yea, takes longer than I expected), but the "move to" option moved the files to the recycle bin of the Team Site, and I had to spend a whole day retrieving all those files. I am confused now, it is supposed to happen? How does this work? 

 

Thanks,

 

Suparna 

@Juan Carlos González Martín regardless of the intent from a programmatic point of view, there will always be cases where large numbers of files need to be moved as one action due to customer need.  

 

I also have similar issues with files missing from time to time and would be interested to know if it is a common experience?  How many times do files seem to disappear leaving only a folder tree?

 

R

@RussellTMR Yes, had the same issue. One of my customer was moving  between 2000, 3000 files from OneDrive to SharePoint and just some folders and a couple of files made it. Everything else was deleted. We needed to recover the files.

 

The copy/move functionnality is very weak and not serious for real business workload. Very disapointed.

Workaround (using OneDrive client) is not the answer. If the functionnality is not always working they must remove it or fix it. Period.

@Juan Carlos González Martín have you any updates about this discussion? It's really painfully when user copies documents and lost them. You say it's "Copy and Move function are not intended for doing massive Copy/Move operations" - i know the limits from https://docs.microsoft.com/en-us/office365/servicedescriptions/sharepoint-online-service-description... , but we have the problem even with fewer and smaller documents. 

1 best response

Accepted Solutions
best response confirmed by Hillary Barter (Copper Contributor)
Solution

If you're using "Open in Explorer" I think that may be part of the issue, and I'm assuming that you're using Internet Explorer. From personal experience I never really got on with the piece of functionality. It worked well back in 2007 when I first used it, but now there are much better options available.

 

Option 1 - Move To

When you visit a document library in the modern experience, you can click on the ellipsis against the document and select move to (see attachment "Modern-MoveTo.docx". In Figure 1, you can see the option to move to, and then in Figure 2 you can see that you have a number of options to either move it to OneDrive for Business or to another SharePoint site. When I tested this with a document of approx 1mb it took about 5 seconds to get itself warmed up and then do the move.

 

Option 2 - OneDrive Sync

The alternative option which has been mentioned before is to use sync (see attachment Modern-Sync.docx). In Figure 1, you can see the ability to Sync your library to your local file system using the OneDrive Sync Client.  By hitting the sync button, you'll see it connect to your sync client (Figure 2), and then it will be available from your Windows Explorer windows (Figure 3). Once it's been synced, you can copy and paste documents between synced libraries in the same way as you would with normal files. Again this took only a few seconds to copy, paste and sync between the libraries.

 

I hope this approaches are useful and that they work. If not, please let us know and we'll see what else we can come up with.

View solution in original post