Sharepoint document library over 100k files

[andrab3008]

We use SharePoint Online (microsoft 365)  as a file storage solution leveraging metadata, we anticipate managing over 100,000 files within a single library. Our approach involves employing various metadata fields to facilitate document retrieval, including document type, company, date, among others. We intend to index multiple columns to optimize search performance. However, there are some key considerations we've learned from our research on web:

1. Indexing Limitations: once you cross 5k

2. Inheritance Restrictions: Once the file count exceeds 100,000, breaking inheritance becomes unable.

Given these constraints, we seek insights from those experienced with managing large file volumes. Have you encountered challenges with search, sorting, or filtering? What strategies do you recommend for efficient document retrieval?

Furthermore, we've installed PnP Modern Search and plan to leverage its capabilities for enhanced search functionality. Any advice or best practices regarding its implementation would be greatly appreciated.

Thank you for your insights and guidance.

[#2]

Hello,

Do you have any specific reason to store 100k documents within a single library? What are the reasons for not separating them?

[#3]

@ArefHalmstrand 

Hi.

We will have a document library for each year. Within this library, we will store files, anticipating 100k+.

This library will serve as an 'archive' where we scan and upload documents, adding metadata such as date, company, and other relevant information. Subsequently, we would like to search for files using metadata as well as keywords from file content.

Did you encounter any issues when working with large document library in your company? (searching, filtering)

thank you.

[#4]

We have faced issues with large libraries especially if you employ a complex folder structure, I would suggest creating a library template that you can re-use to breakdown the files, you could then create a custom solution (classic search page, Power app, etc.) To search all your documents and configure in a way that works for your organisation although with some tweaks the built in search can be very robust. If using on prem, you may want to consider provisioning a separate content database aswell