Forum Discussion

DanielClaassen's avatar
DanielClaassen
Brass Contributor
Sep 21, 2018

How many documents is to much for a Document Set and Record center?

Hi   Hoping someone can advise on a large Correction: document set  Document center/record centers. How many documents are to much?   I am architecting a solution for a client and just want to t...
document center
Document Set
sharepoint server
Brent Ellis's avatar
Brent Ellis
Silver Contributor
Sep 21, 2018
In my experience, a document library with document sets is easily doable for this volume.

We do this in SharePoint Online, and have a single library with hundreds of thousands of documents, with root-level document sets, standard folders within each document set, and then the subsequent documents inside of those folders. And a few other similar implementations, with tens of thousands of docs.

The keys are this:
1) Make sure you define any indexes you need on the front end. Use all 20 of them, and if you can't fill them up, consider creating some placeholder names just in case a new requirement in the future drives the need for a new one

2) Know how to script (in powershell) and how to iterate documents, in case you need to make mass changes

3) Leverage Search and Refiners as the primary interface

4) Make sure your root level document set has views to avoid threshold limits, and whatever folder structures you have underneath do the same. I know on-prem you can change the threshold values, in the cloud you can't, but if you do it right it may not matter. I'm not sure any recent updates on prem that might make it easier. If you are gonna automatically archive stuff, consider using subfolder by year or some other type of value.

You could move things to a record center if you want, but technically you dont HAVE to if you dont have a good reason to. A well architected solution will allow you continue to grow.

Technically, it will work, just takes some thought on the front end.
  • DanielClaassen's avatar
    DanielClaassen
    Brass Contributor
    Sep 25, 2018

    Thanks Brent for the great reply. Sorry for the late reply, had a public holiday yesterday. 

     

    Going to keep that in mind...Going to have a read through the other replies. 
    So...Indexes and views really really important. 

    Still, it is possible but is it recommended?

     

  • Sderix's avatar
    Sderix
    Brass Contributor
    Sep 21, 2018

    Hi,

    first off Brents reply is really good. Second it looks like you did your homework.

    I've been working with several IMHO rather large environments e.g.:
    - 5.000 new documents every month containing millions of documents
    - collaboration environements with tens of millions of documents

    What I would like to emphasize is: Know your limits! This is what a foundation is for a building. If you mess that up... #DOH.
    Since you didn't mention the version of SharePoint you are using they are:
    SharePoint Online - https://technet.microsoft.com/en-us/library/mt842345.aspx
    SharePoint 2016 - https://docs.microsoft.com/nl-nl/SharePoint/install/software-boundaries-and-limits-0
    SharePoint 2013 - https://docs.microsoft.com/nl-nl/SharePoint/install/software-boundaries-and-limits

    Your story isn't clear about your information architecture.
    Is het perhaps possible to create multiple archives?
    Let's say you need to keep documents of a certain type for 7 years.
    Perhaps in the first 2 years there is (e.g) a 15% change that you actually need it.
    When the document is 3-7 years old the chances you need it deminish.


    Some last ramblings:
    Why are you using Document Sets. If you have a good reason: perfect. These could give you nightmares. Suppose you need to restore 1 document from a document set?
    Are you allowed to modify the document set because you need 1 document?

    To elaborate on the answer Brent gave. Indexes are a very good idea (if you are in SPO you don't need to worry about them, columns are already indexed).
    Make you sure SP automaticly creates (sub) folders for archived documents. Then make views based on your metadata (and use the indexed columns).
     
    My 2 cents: Check the limits and create an information architecte (including a document life cycle plan)
    Maybe these worksheets will help:
    http://www.sharepointgeoff.com/planning-worksheets/

    • DanielClaassen's avatar
      DanielClaassen
      Brass Contributor
      Sep 25, 2018

      Hi Sander

      Thanks for the detailed response. Yes, need to do your research, this is my bread and butter. (:


      The client is running SharePoint 2016 Standard Edition. Sorry, mentioned it in my first post but somehow that got marked as Spam so had to redo the post.
      It is a new clean environment. We did not implement it so still have to verify the documentation we received.

      Can we have multiple archives? Yes, we are designing it and if that is the best way we will recommend it to the clients. 

       

      "Perhaps in the first 2 years there is (e.g) a 15% chance that you actually need it."
      So maybe I must do the following:

      Create document Library, Index metadata fields.

      Create views

      (Just realized that I actually did mention document set in the original post, suppose to be a document center. )

      Leave invoices in the document library for 2 years then set a retention policy to move documents to a Document center on a different Content Database. Not sure if want a Content-Type Syndication hub as well so maybe just keep in one DB... 

       
      Or must I forget about it and just leave in one document library?


      So then the number of documents in Document Library will be: 
      74 000+  probably increasing a little every year
      450 000+ documents in the document center? (Separate DB hopefully)


      So good to know that the document library can handle the volume but is it the best way since im designing the solution at the moment?

      Thanks

Resources