Forum Discussion
Beckie_White
Jun 10, 2024Copper Contributor
Unable to use power query to combine files in folder synced from SharePoint due to system file
I have a library synced locally so that I can analyze portions of the file contents based on a query. Currently there are around 5,000 files in the library, and there will be closer to 10,000 by the ...
- Jun 10, 2024
How do I work around this system file issue?
When you query a folder path you get something like:
where the [Attributes] column contains a Record per file. Each Record has a number of Fields, one of them being 'System'. The latter tells you if the file is marked as a System file
To filter out System files you can do:
let Source = Folder.Files("To_Be_Replaced_With_Path_To_Folder"), FilteredOutSystemFiles = Table.SelectRows(Source, each [Attributes]?[System]? <> true) in FilteredOutSystemFiles
similarly with Hidden files (if a file is open when you query a folder its name starts with ~$ and it's marked as Hidden:
to get rid of Hidden files:
let Source = Folder.Files("To_Be_Replaced_With_Path_To_Folder"), FilteredOutHiddenFiles = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true) in FilteredOutHiddenFiles
To filter out System + Hidden files:
let Source = Folder.Files("To_Be_Replaced_With_Path_To_Folder"), FilteredOutSystemAndHiddenFiles = Table.SelectRows(Source, each ([Attributes]?[System]? <> true) and ([Attributes]?[Hidden]? <> true) ) in FilteredOutSystemAndHiddenFiles
SGGusau
Jun 10, 2024Copper Contributor
You probably have tried filtering out the system file, if not, right click the cell and choose DOES NOT EQUAL. Alternatively in the Extension column, filter to only the file extension you wish to keep, for eg. xlsx. If this doesn’t resolve it, you might need to do the filtering higher up in the path.
Beckie_White
Jun 10, 2024Copper Contributor
Thanks, unfortunately I cannot even get to the part where I would be able to select a filter.