Forum Discussion
Moving large list from one site to another one in Sharepoint Online
Hello,
I have a list with 11k records. I can't turn it into a template because it is too big and the system won't let me select to add the content of the list.
Is there a way to move large lists from one place to another?
Thanks in advanced!
- There are a few choices:
Do it programmatically using PowerShell and CSOM to go through the source list and add items to destination.
Flow based on a timer start and select the list as a collection of items and then for each write to the destination list.
Use a tool such as Sharegate to do all the work for you.
3 Replies
Another option here is using desktop Office: Access, Excel & Outlook to pull down offline, then push up into new list. This is one approach recommended in https://support.office.com/en-us/article/Manage-large-lists-and-libraries-in-SharePoint-b8588dae-9387-48c2-9248-c24122f07c59 for managing large lists.
- Outlook for reading Contacts, Tasks & Calendar
- Excel - a common link-up. No blocking of rows >5,000
- Access - I've used Access to remarkable effect for years: one-offs; standing queries; reporting (poor man's SSRS / Crystal Reports). Since ver 2007
- Microsoft doing a good job of maintaining optimized Access/SharePoint drivers
- Office 2016: External Data --> More --> SharePoint List
- counterintuitive trick: DISABLE "Use the cache format that is compatible with Microsof Access 2010 and later" in File --> Options --> Current Database. This form of caching kills performance when working with lists over many sessions. Can even throw off sync
- helpful to check/clean the Access/SharePoint caches from time to time (especially if you forgot to untick caching option)
- %APPDATA%\Roaming\Microsoft\Access\ (delete AcessCache.accdb
- %APPDATA%\Roaming\Microsoft\Web Server Extensions\Cache (delete all files in the Cache subfolder)
- automatically discovers and opens related lists (lookups)
- maintains lookup column GUIDs and creation, modified dates
- especially helpful if you want to apply data transformations such as
- cleanup leading and trailing spaces
- cleanup non-printable characters - often a result of end user copy/paste from web sources. Can really scrub email addresses using this and stripping extra spaces (Google for syntax)
- combine or split columns
- identify duplicates and sparse rows
- head off garbage in/garbage out!
- Once linked, even though the list(s) appear in Access's list of tables ready to open, best to to interact via queries instead. Even if querying every column. Driver optimized for explicit SQL queries - easy to build in the query builder GUI (similar to SQL Server Management Studio)
- Faster than one might think once query built and after 1st open. Don't have to worry about indexes: the driver automatically batches. Especially important feature when you've exceeded 5,000 item limit and prevented from creating indexs as work-around (the catch 22)!
- Microsoft doing a good job of maintaining optimized Access/SharePoint drivers
You can try to create indexed columns in the list and divide the content into views using filters, this might allow you to move them from SPO UI. The other way will be using powershell csom which can use CAML query with maximum threashold value set to return all your content. The easiest option is using a migration tool Sharegate or AvePoint.
- There are a few choices:
Do it programmatically using PowerShell and CSOM to go through the source list and add items to destination.
Flow based on a timer start and select the list as a collection of items and then for each write to the destination list.
Use a tool such as Sharegate to do all the work for you.
