What you need to think about when migrating files to SharePoint Online

[Haniel Croitoru]

Migrating your organizational data to SharePoint Online requires careful planning to be successful.  The purpose of this article is to provide you with some high-level points to consider as you’re embarking on this journey.

Sources and Destinations

For “lift and shift” migrations, the grouping of files at the destination will closely resemble the source.  You can think of groups as being specific drives or folders.  These types of migrations don’t require you to do a lot of mapping of files and you can migrate your files group by group.

On the other hand, when you are reorganizing the content into different locations and structure, you need to carefully plan out the structure and consider the Volumes and File and Folder Names points listed below.

Volume

Today, the limits in SharePoint Online are quite high, allowing you to store up to 30 million files and folders in a library.  In many cases, this limit will suffice for storing the files.  However, as the number of documents grows so does the time it will take to render a document library view that displays them.  Therefore, you should carefully plan out how you will view these files online.  In cases, where you expect to have over several hundred files in a single view, you should consider adopting some techniques for managing large libraries.

When you’re migrating your files, try to break down the process into smaller batches that will be quicker to migrate and verify within a given time.

File Sizes

Each file you upload can be up to 15GB in size.  Those numbers are not uncommon when dealing with large architectural drawings, geospatial mappings, or creative files.  Depending on the Office 365 plan that your organization is using, the amount of available space will vary.  For example, for an Office 365 Mid-size Business account, the you can store up to 1TB per site collection.  So before moving those large files into Office 365, you want to connect with you IT department to find out what your specific limit is.

File Types

In the past, there were restrictions on the types of files that could be added to a SharePoint Online document library.  However, recent enhancements made by Microsoft have removed these limitations.  There are currently no known file types restrictions.

File and Folder Names

There are also restrictions on the characters, names, and length that you can use in a file or folder names.  There are certain legacy names used in Windows systems, which cannot be used as either file or folder names.  These names are: AUX, PRN, NUL, CON, COM0, COM1, COM2, COM3, COM4, COM5, COM6, COM7, COM8, COM9, LPT0, LPT1, LPT2, LPT3, LPT4, LPT5, LPT6, LPT7, LPT8, LPT9.  As well, names cannot begin with a tilde (~), which is typically used to indicate a temporary file.

The characters that you cannot include are also blocked by Windows as they have a special meaning, such as “\” for a path and “:” for a drive.  Therefore, the following characters cannot be included in any file or folder names: " * : < > ? / \ | .  In addition, # and % may or may not be supported in file names by your organization. 

When you are migrating your files and folders to SharePoint Online, you need to also consider the overall file name and path lengths.  The overall path, including the file name must be no more than 400 characters in length.  One strategy to reduce file path is to map some of the paths to custom metadata that can be associated with the files.

Metadata

One of the many advantages that SharePoint offers is the ability to associate additional metadata with each file.  If you are looking to add custom metadata to your files as part of the migration, then you need to think about where this metadata will be captured and how it will get applied to the files.

There are different ways to achieve this, such as via PowerShell scripting or 3^rd party tools.  Make sure that you identify all required metadata prior to the migration to avoid situations, where migrations fail due to missing metadata.

Scheduling migrations

There are several factors that can impact the migration speeds that you may experience.  These include:

• Parallelizing Migration – by using several VM’s, networks, or physical computers, you can increase the migration speed as each environment will have some limits applied to it. So, if one machine can upload 1TB/day, then running 5 VM’s in parallel could result in 5TB/day uploads.
• Metadata complexity – uploading files is directly impacted by the number and complexity of metadata fields required for each file. The more metadata, the slower the upload speeds
• Limiting package sizes – try uploading groups of no more than 1000 files and less than 100MB per group at a time.
• Leveraging Azure – leveraging Azure as a temporary storage for the files provides better overall speeds even though an extra step is added to the process. This stems from the fact that uploading to Azure is much faster than uploading to SharePoint Online directly.  You can even select the datacenter you wish to use for the migration.  So, you can choose wisely and select the same datacenter location for your Azure and Office 365 locations.

Given all these factors, it would be a good idea to do a few test migrations to understand what type of throughput you are getting for your specific scenario before performing the actual migrations.  Once you have a good idea, you should also work with the business users to establish a schedule that will enable you to confidently migrate complete groups of files (whether it’s by department or another logical unit).   

Training

No matter if your migration will be a simple “lift and shift”, or reorganize all the files into different sites with lots of metadata, you should plan for training for your end users so they understand not just where the files are located in the new environment (SharePoint Online), but also understand the rationale behind any changes made and how to work within the new confines.

Conclusion

A successful mile migration to SharePoint Online requires careful planning.  There are several parameters that you must consider for increasing the changes of success.  This article provided a high-level overview of some of the key considerations.

References

• SharePoint Online limits
• Types of files that cannot be added to a list or library
• Manage large lists and libraries in SharePoint
• Invalid file names and file types in OneDrive, OneDrive for Business, and SharePoint
• New support for # and % in SharePoint Online and OneDrive for Business
• Avoid getting throttled or blocked in SharePoint Online
• SharePoint Online and OneDrive Migration Speed