LVT suddenly more picky: several views suddenly not working anymore

[Francis Laurin]

Hi,

I have a client that has ~40k documents in a document library and this has been the case for years. Suddenly, two days ago, several flat views just stopped working with the dreaded "The attempted operation is prohibited because it exceeds the list view threshold" error message. I deleted 5k test documents to see if some sort of hidden threshold had been hit, but this didn't change anything. 

Something changed in the infrastructure, I believe. I also try to create routine views on other libraries with more than 5000 documents that always worked and don't anymore.

I don't really want to go through the details of each affected views and the possible work arounds, but does anyone know if something changed lately in the SPO infrastructure about the LVT?

[#2]

More precisely, this looks to affect views that sort (or group by) on text or managed metadata columns.

[#3]

@Francis Laurin , I am having the same issue. Has anyone found a way to manage this? The List View is important to the users managing this Library. Having less than 5000 items is not an option in this case. 

[#4]

@shawncklsmartin

Good to see that I am not alone, but I got no help or acknowledgement from Microsoft support. They won't touch anything with more than 5000 items. Support don't understand it themselves.

What I find is that the issue only happens on flat views on a library with document sets. This may explain why it is a niche case and not enough seem to experiment the issue. 

[#5]

Would it be possible to differentiate the documents into multiple content types? Have you tried to index certain columns and limit the view?

I would gladly try to help through a MS Teams meeting if interested.

[#6]

Hi @ArefHalmstrand! Thanks so much for reaching out. I have not considered separate content types. I have tried to limit views and index columns. I would be interested in a MS Teams call if you have the time. Please private message me for contact details. 

Cc @Francis Laurin , thanks for your reply yesterday. I appreciate your follow up. 

[#7]

In my case, I have content types for documents and document sets, my index are properly set up and my views only filter and sort through the indexed columns. I showed it all to Marc Anderson and he confirmed the issue but was not able to get the attention of Microsoft with it. You are very welcome to take a look. I have several setup like this at different clients: One doc lib with doc sets and flat views on documents that don't work anymore.

[#8]

I spoke to Shawn recently. My recommendation is to separate the data into multiple document libraries. If you still want to access them all from a single view, it is doable by either create a page with short links to the different document libraries. Or you could include them in the same library, by adding them as a shortcut link.

The advantage of having multiple document libraries is that you could actually handle them in a more complex way. Exclude some of them from search, set specific retention policies or even hide certain document libraries from end-users.

[#9]

There are several issues that I find with this idea of splitting into multiple libraries:
-There may not be an obvious criteria to split the library, then having multiple is a nuisance on the user point of view. I have several real-life scenarios that I can share: HR employee files (one doc set per employee) where one would like to filter all the performance reviews, several suppliers (one doc set per supplier) on a big project where one would like to filter some document types, Contract projects where lawyers would want to filter the documents assigned to them transversally.
-The split-library solution gets a lot more complex to assemble: site content types, library templates, some tricks to aggregate unified views... Compare that to a single library which can be assembled by a power user or for significantly less consulting costs.
-From the user point of view, working in a single library interface may also be much simpler: all the views they need, direct interactions with documents and metadata. For some existing clients, I was forced to recreate key views using the PNP Modern Search but it is much less usable than a basic view in this scenario.
-The deal with Microsoft has always been that a library supports 30M documents and if you do your indexes correctly, you won't have any issue. I did that and it worked well until it didn't in 2022 for libraries with doc sets. Several clients with which I did not even work anymore suddenly had their libraries failed.

For new projects where I need to implement the pattern, I resorted to create a separate "archive" library to move all the inactive document sets. If volume allows, it leaves less than 5000 active documents and doc sets in the "active" library and I created a Power Automate flow to move the doc sets once they are completed. It adds complexity to my ideal pre-2022 scenario, but keep a lot of advantages.

[#10]

Good point, I understand your dilemma and I unfortunately cannot give a quick and perfect answer.
However, If I would try to split this into different subcategories, I would start with the HR employee files.

I do not know how big the company is, but in my experience, having one doc set per employee sounds a bit "risky/messy" for me.
I would recommend perhaps using OneDrive. If the employee has a HR folder, and that is shared with a specific HR security group. Working from that employees OneDrive might be a better way to handle employee specific data. At the same time... The user experience for the HR will be a bit tougher, you will no longer have the option to just see all performance review in one place (by default). This can however be configured or created through an SPFx solution. I am not saying this is the "right way", but definitely an approach I might ask to consider.

When it comes to big projects (lawyers), I would recommend using MS Teams and have a channel setup between standard channels, private and shared channels. All documents that are "working documents" on a live project, should in my experience be in MS Teams channels.

Having an archive library is awesome! I would almost recommend creating an Archive Hub! Splitting it into multiple sites and then document libraries to really have a good and clean structure on what the organization has decided to save as archived. Unfortunately, in my experience, archives mostly become a black hole that no one wants to clean.

I am not 100% sure I have given you anything good to use haha, but if I can help in any way, I am available for a Teams call. Also looking forward to hearing what others might have to say.