Forum Discussion
Solved
SharePoint Online Limits: Should we distribute large document libraries across multiple sites?
Dear all,
we're currently planning on moving more and more files and folders from classic file servers to document libraries in SharePoint online.
The website about SharePoint online limits states that i.e. a single site can hold up to 25TB of data. In case we have to create multiple large libraries, - for example 10 libraries with 2TB of data and tens of thousands of files each - : Can we easily create all libraries below a single SharePoint site or should we distribute the libraries across multiple sites?
Background: We would like to create a single "Public" document library for each division (about 10) of our company. I'm currently drawn between two ways to go:
a) Creating the document library as additional library within each divisions Team (MS Teams)
b) Creating a dedicated SharePoint site and adding a document library for each division to it
Any suggestions or recommendations are highly appreciated!
Thanks
Michael
- The limit is a site collection limit, so you can put many document libraries in the same site just fine. The biggest issue you will want to concern yourself with besides number of files etc. will be security. After 100k files into a parent folder you can no longer break permission inheritance and thus you cannot share files in those folders unless you can path down enough to get under that limit. So make sure if you have a directory with over 100k Items, you set the permissions on the parent folder prior to moving the files in or you won't be able to. If everything should never have custom permission you should be fine, there really isn't anything preventing using one site with multiple libraries.
The reasons you would use multiple libraries or sites would be settings. For example, retention policies are site level so you would want different sites if different "Libraries" needed different retention periods. Do libraries can have search flags turned off, or have various other settings, and same for sites you would want to consider, but mostly, outside of policies, single site is fine there.
Breaking libraries up helps prevent someone from syncing the "Entire File library" by trying to sync at the root of a library. By having multiples this prevents that goof. Site's have no bearing here in that regard since sync isn't site based.
I know I'm just rambling here just kind of typing as it comes in, you seem to have the idea around splitting into libraries, but for sites, it mainly boils down to another permission level and or policies that can be applied only at site level so if having all libraries in that same container would never be different, then one site would ultimately be fine.
5 Replies
Dear Michael,
Your question is normal in that kind of Situation "Strategy to Move to Cloud", and Office 365 SPO is one of the option you can use for File Server Move.
But I would suggest to add one criteria into your plan is to add the freshness of data you want to load.
Because SPO has a certain number of limitations and load a huge volume without any real usage will create a lot of frustrations from user point of view.
Take also care about native usage from File Server difficult to retrieve with SPO: Windows Explorer mode
The only supported solution from MS to benefit that feature will be via OneDrive SyncApp, but this module is based on sync module not immediate.
Fab
Dear Fabrice,
thanks for your quick reply. We are already aware of the limitations on OneDrive syncing (issues when syncing more than 100.000/300.000 items even with Files on Demand) and are also thinking about a guideline for our users regarding which files should be placed in O365 and which not.
Because of the current limits and the preference of most users to work with files in Explorer we will not perform a "Lift and Shift" migration (just move everything to SharePoint document libraries). Instead I am currently thinking of recommending to place only files in O365 for which collaboration and/or external access is needed. And also specify limits - i.e. not placing extremely large amounts of data there - at least speaking to IT in case this is requested.
Regarding distribution of files across multiple document libraries and sites: We are now just creating the foundation for placing documents in the cloud and I would like to avoid misconceptions here so we aren't creating a structure that runs into well known limits very quickly.
I think distributing files and folders across multiple document libraries definitely makes sense (since creating a structure within just one library will hit limits quickly). However, I am still unsure regarding my initial question: Could we create multiple document libraries within one site or should we rather distribute the libraries across multiple SharePoint sites?
From what I can see the 25TB data amount limit applies to a single site which is sufficient. However, the OneDrive sync limit officially is 300.000 items and in reality problems already start at a much lower amount of files. So I'm not sure if the 25TB limit of a single site is actually realistic.Best Regards
Michael- The limit is a site collection limit, so you can put many document libraries in the same site just fine. The biggest issue you will want to concern yourself with besides number of files etc. will be security. After 100k files into a parent folder you can no longer break permission inheritance and thus you cannot share files in those folders unless you can path down enough to get under that limit. So make sure if you have a directory with over 100k Items, you set the permissions on the parent folder prior to moving the files in or you won't be able to. If everything should never have custom permission you should be fine, there really isn't anything preventing using one site with multiple libraries.
The reasons you would use multiple libraries or sites would be settings. For example, retention policies are site level so you would want different sites if different "Libraries" needed different retention periods. Do libraries can have search flags turned off, or have various other settings, and same for sites you would want to consider, but mostly, outside of policies, single site is fine there.
Breaking libraries up helps prevent someone from syncing the "Entire File library" by trying to sync at the root of a library. By having multiples this prevents that goof. Site's have no bearing here in that regard since sync isn't site based.
I know I'm just rambling here just kind of typing as it comes in, you seem to have the idea around splitting into libraries, but for sites, it mainly boils down to another permission level and or policies that can be applied only at site level so if having all libraries in that same container would never be different, then one site would ultimately be fine.
