Get the number of folders and files from a document library

%3CLINGO-SUB%20id%3D%22lingo-sub-1356539%22%20slang%3D%22en-US%22%3EGet%20the%20number%20of%20folders%20and%20files%20from%20a%20document%20library%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1356539%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20there%2C%26nbsp%3B%3C%2FP%3E%3CP%3EI%20am%20trying%20to%20get%20some%20information%20from%20a%20very%20big%20document%20library.%20Just%20under%20600%2C000%20files.%26nbsp%3B%3C%2FP%3E%3CP%3EI%20would%20like%20to%20restructure%20the%20content%20into%20several%20libraries%20but%20I%20am%20having%20a%20heck%20of%20a%20time%20understanding%20how%20it%20is%20structured.%26nbsp%3B%3C%2FP%3E%3CP%3EI%20tried%20running%20this%20PowerShell%20script%2C%20but%20I%20got%20an%20error%20as%20it%20exceeded%20the%20list%20view%20threshold.%26nbsp%3B%3C%2FP%3E%3CPRE%20class%3D%22lia-code-sample%20language-powershell%22%3E%3CCODE%3E%23Connect%20to%20SPO%0AConnect-PnPOnline%20-Url%20%22https%3A%2F%2F%3CTENANT-NAME%3E.sharepoint.com%2Fsites%2F%3CYOUR-SITE%3E%22%0A%0A%23My%20list%20target%20%0A%24myList%20%3D%20%22%2FShared%20Documents%22%0A%23Store%20the%20results%0A%24results%20%3D%20%40()%0Aforeach%20(%24list%20in%20%24myList)%20%7B%0A%20%20%20%20%24allItems%20%3D%20Get-PnPListItem%20-List%20%24list%20-Fields%20%22FileLeafRef%22%2C%20%22SMTotalFileStreamSize%22%2C%20%22FileDirRef%22%2C%20%22FolderChildCount%22%2C%20%22ItemChildCount%22%0A%20%20%20%20%0A%20%20%20%20foreach%20(%24item%20in%20%24allItems)%20%7B%0A%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20%20%20%23Narrow%20down%20to%20folder%20type%20only%0A%20%20%20%20%20%20%20%20if%20((%24item.FileSystemObjectType)%20-eq%20%22Folder%22)%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%24results%20%2B%3D%20New-Object%20psobject%20-Property%20%40%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20FileType%20%20%20%20%20%20%20%20%20%20%3D%20%24item.FileSystemObjectType%20%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20RootFolder%20%20%20%20%20%20%20%20%3D%20%24item%5B%22FileDirRef%22%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20LibraryName%20%20%20%20%20%20%20%3D%20%24list%20%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20FolderName%20%20%20%20%20%20%20%20%3D%20%24item%5B%22FileLeafRef%22%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20FullPath%20%20%20%20%20%20%20%20%20%20%3D%20%24item%5B%22FileRef%22%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20FolderSizeInMB%20%20%20%20%3D%20(%24item%5B%22SMTotalFileStreamSize%22%5D%20%2F%201MB).ToString(%22N%22)%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20NbOfNestedFolders%20%3D%20%24item%5B%22FolderChildCount%22%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20NbOfFiles%20%20%20%20%20%20%20%20%20%3D%20%24item%5B%22ItemChildCount%22%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%7D%0A%23Export%20the%20results%0A%24results%20%7C%20Export-Csv%20-Path%20%22C%3A%5CUsers%5C%24env%3AUSERNAME%5CDesktop%5CNestedFoldersForONEdoclib.csv%22%20-NoTypeInformation%3C%2FYOUR-SITE%3E%3C%2FTENANT-NAME%3E%3C%2FCODE%3E%3C%2FPRE%3E%3CP%3ESo%20I%20am%20kind%20of%20at%20a%20loss%2C%20then%20I%20found%20something%20called%20a%20file%20plan%20report%20from%20this%20link%20...%26nbsp%3B%3C%2FP%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fwww.c-sharpcorner.com%2FUploadFile%2F644e54%2Fgenerate-file-plan-report-in-sharepoint-2013-and-office-365%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3EFile%20Plan%20Report%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3EBut%20I%20don't%20have%20that%20option%20...%20does%20anyone%20else%20have%20any%20suggestions%3F%3C%2FP%3E%3CP%3EThanks!%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1356539%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ESharePoint%20Online%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1356992%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20the%20number%20of%20folders%20and%20files%20from%20a%20document%20library%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1356992%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F28866%22%20target%3D%22_blank%22%3E%40Mike%20Dumka%3C%2FA%3E%26nbsp%3B%3CBR%20%2F%3EDo%20you%20need%20a%20CSV%20file%20with%20the%20details%20of%20all%20items%20or%20do%20you%20want%20to%20have%20a%20File%20Explorer%20like%20view%20which%20shows%20the%20total%20size%2C%20number%20of%20documents%2C%20number%20of%20folders%20within%20each%20folder%3F%3CBR%20%2F%3EDo%20you%20also%20need%20to%20get%20information%20on%20the%20document%20versions%20present%2C%20encrypted%20files%20and%20checked%20out%20items%3F%3C%2FP%3E%3CP%3EPaul%20%7C%20SLIM%20Applications%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1357786%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20the%20number%20of%20folders%20and%20files%20from%20a%20document%20library%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1357786%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F4831%22%20target%3D%22_blank%22%3E%40Paul%20de%20Jong%3C%2FA%3E%26nbsp%3BJust%20need%20how%20many%20items%20are%20in%20each%20folder.%26nbsp%3B%3C%2FP%3E%3CP%3EAs%20strange%20as%20it%20may%20sound%20I%20don't%20even%20need%20to%20know%20the%20size%20of%20the%20files%2Ffolders.%20The%20reason%20I%20need%20the%20count%2C%20is%20because%20users%20are%20trying%20to%20sync%20the%20600%2C000%20files%20and%20there%20is%20a%20huge%20performance%20issue.%20The%20OneDrive%20sync%20client%20has%20a%20300%2C000%20per%20computer%20so%20that%20is%20why%20I%20need%20to%20spit%20this%20up.%26nbsp%3B%3C%2FP%3E%3CP%3EThanks!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1357965%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20the%20number%20of%20folders%20and%20files%20from%20a%20document%20library%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1357965%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F28866%22%20target%3D%22_blank%22%3E%40Mike%20Dumka%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESplitting%20the%20library%20is%20indeed%20a%20good%20way%20forward%20to%20avoid%20using%20OneDrive%20to%20synchronise%20such%20a%20large%20data%20set.%20Having%20LVT%20problems%20only%20complicates%20things%20further.%20I%20am%20not%20sure%20how%20well%26nbsp%3BPowerShell%20scripts%20can%20cope%20with%20the%20LVT.%20There%20must%20be%20other%20solutions%20out%20there%20but%20we%20have%20developed%20a%20%3CA%20href%3D%22https%3A%2F%2Fwww.slimapplications.com%2Fproduct%2Fexplorer%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Esolution%3C%2FA%3E%20that%20uses%20REST%20API%20and%20can%20cope%20with%20LVT%20limits.%20You%20can%20use%20the%20solution%20to%20browse%20through%20the%20structure%20or%20generate%20a%20detailed%20report%20(or%20csv%20list).%20You%20can%20use%20the%20trial%20version%20so%20there%20are%20no%20costs.%3CBR%20%2F%3EPaul%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1358147%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20the%20number%20of%20folders%20and%20files%20from%20a%20document%20library%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1358147%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F4831%22%20target%3D%22_blank%22%3E%40Paul%20de%20Jong%3C%2FA%3E%26nbsp%3BThanks%20buddy!%20%3A)%3C%2Fimg%3E%3C%2FP%3E%3CP%3EI%20will%20check%20that%20out%20today!%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
Contributor

Hi there, 

I am trying to get some information from a very big document library. Just under 600,000 files. 

I would like to restructure the content into several libraries but I am having a heck of a time understanding how it is structured. 

I tried running this PowerShell script, but I got an error as it exceeded the list view threshold. 

#Connect to SPO
Connect-PnPOnline -Url "https://<TENANT-NAME>.sharepoint.com/sites/<YOUR-SITE>"

#My list target 
$myList = "/Shared Documents"
#Store the results
$results = @()
foreach ($list in $myList) {
    $allItems = Get-PnPListItem -List $list -Fields "FileLeafRef", "SMTotalFileStreamSize", "FileDirRef", "FolderChildCount", "ItemChildCount"
    
    foreach ($item in $allItems) {
        
        #Narrow down to folder type only
        if (($item.FileSystemObjectType) -eq "Folder") {
            $results += New-Object psobject -Property @{
                FileType          = $item.FileSystemObjectType 
                RootFolder        = $item["FileDirRef"]
                LibraryName       = $list 
                FolderName        = $item["FileLeafRef"]
                FullPath          = $item["FileRef"]
                FolderSizeInMB    = ($item["SMTotalFileStreamSize"] / 1MB).ToString("N")
                NbOfNestedFolders = $item["FolderChildCount"]
                NbOfFiles         = $item["ItemChildCount"]
            }
        }
    }
}
#Export the results
$results | Export-Csv -Path "C:\Users\$env:USERNAME\Desktop\NestedFoldersForONEdoclib.csv" -NoTypeInformation

So I am kind of at a loss, then I found something called a file plan report from this link ... 

File Plan Report 

But I don't have that option ... does anyone else have any suggestions?

Thanks! 

4 Replies
Highlighted

@Mike Dumka 
Do you need a CSV file with the details of all items or do you want to have a File Explorer like view which shows the total size, number of documents, number of folders within each folder?
Do you also need to get information on the document versions present, encrypted files and checked out items?

Paul | SLIM Applications

Highlighted

@Paul de Jong Just need how many items are in each folder. 

As strange as it may sound I don't even need to know the size of the files/folders. The reason I need the count, is because users are trying to sync the 600,000 files and there is a huge performance issue. The OneDrive sync client has a 300,000 per computer so that is why I need to spit this up. 

Thanks!

Highlighted

@Mike Dumka 

Splitting the library is indeed a good way forward to avoid using OneDrive to synchronise such a large data set. Having LVT problems only complicates things further. I am not sure how well PowerShell scripts can cope with the LVT. There must be other solutions out there but we have developed a solution that uses REST API and can cope with LVT limits. You can use the solution to browse through the structure or generate a detailed report (or csv list). You can use the trial version so there are no costs.
Paul

Highlighted

@Paul de Jong Thanks buddy! :)

I will check that out today!