May 03 2020 03:56 PM
Hi there,
I am trying to get some information from a very big document library. Just under 600,000 files.
I would like to restructure the content into several libraries but I am having a heck of a time understanding how it is structured.
I tried running this PowerShell script, but I got an error as it exceeded the list view threshold.
#Connect to SPO
Connect-PnPOnline -Url "https://<TENANT-NAME>.sharepoint.com/sites/<YOUR-SITE>"
#My list target
$myList = "/Shared Documents"
#Store the results
$results = @()
foreach ($list in $myList) {
$allItems = Get-PnPListItem -List $list -Fields "FileLeafRef", "SMTotalFileStreamSize", "FileDirRef", "FolderChildCount", "ItemChildCount"
foreach ($item in $allItems) {
#Narrow down to folder type only
if (($item.FileSystemObjectType) -eq "Folder") {
$results += New-Object psobject -Property @{
FileType = $item.FileSystemObjectType
RootFolder = $item["FileDirRef"]
LibraryName = $list
FolderName = $item["FileLeafRef"]
FullPath = $item["FileRef"]
FolderSizeInMB = ($item["SMTotalFileStreamSize"] / 1MB).ToString("N")
NbOfNestedFolders = $item["FolderChildCount"]
NbOfFiles = $item["ItemChildCount"]
}
}
}
}
#Export the results
$results | Export-Csv -Path "C:\Users\$env:USERNAME\Desktop\NestedFoldersForONEdoclib.csv" -NoTypeInformation
So I am kind of at a loss, then I found something called a file plan report from this link ...
But I don't have that option ... does anyone else have any suggestions?
Thanks!
May 03 2020 11:58 PM
@Mike Dumka
Do you need a CSV file with the details of all items or do you want to have a File Explorer like view which shows the total size, number of documents, number of folders within each folder?
Do you also need to get information on the document versions present, encrypted files and checked out items?
Paul | SLIM Applications
May 04 2020 07:17 AM
@Paul de Jong Just need how many items are in each folder.
As strange as it may sound I don't even need to know the size of the files/folders. The reason I need the count, is because users are trying to sync the 600,000 files and there is a huge performance issue. The OneDrive sync client has a 300,000 per computer so that is why I need to spit this up.
Thanks!
May 04 2020 08:04 AM
Splitting the library is indeed a good way forward to avoid using OneDrive to synchronise such a large data set. Having LVT problems only complicates things further. I am not sure how well PowerShell scripts can cope with the LVT. There must be other solutions out there but we have developed a solution that uses REST API and can cope with LVT limits. You can use the solution to browse through the structure or generate a detailed report (or csv list). You can use the trial version so there are no costs.
Paul
May 04 2020 09:33 AM
@Paul de Jong Thanks buddy! :)
I will check that out today!
Jan 12 2021 08:11 PM
Hey Paul, How are you ? Im facing the same issue here; LARGE DOCUMENT LIBRARY impacting on the Onedrive Sync; I need to create a plan to move/migrate to new document library but I need to check how many items have in each folder; Could you please help me with that ? Which product (TRIAL) should I use ?
Tks buddy!
Claudio Santini - csantini@bracta.com
Jan 12 2021 10:37 PM
@Claudio Santini
Use the trial for the Explorer tool. See https://www.slimapplications.com/request-trial/
Paul