Forum Discussion

Beckie_White's avatar
Beckie_White
Copper Contributor
Jun 10, 2024
Solved

Unable to use power query to combine files in folder synced from SharePoint due to system file

I have a library synced locally so that I can analyze portions of the file contents based on a query. Currently there are around 5,000 files in the library, and there will be closer to 10,000 by the ...
  • Lorenzo's avatar
    Jun 10, 2024

    Beckie_White 

     

    How do I work around this system file issue?

    When you query a folder path you get something like:

     

    where the [Attributes] column contains a Record per file. Each Record has a number of Fields, one of them being 'System'. The latter tells you if the file is marked as a System file

     

    To filter out System files you can do:

    let
        Source = Folder.Files("To_Be_Replaced_With_Path_To_Folder"),
        FilteredOutSystemFiles = Table.SelectRows(Source, each [Attributes]?[System]? <> true)
    in
        FilteredOutSystemFiles

     

    similarly with Hidden files (if a file is open when you query a folder its name starts with ~$ and it's marked as Hidden:

     

    to get rid of Hidden files:

    let
        Source = Folder.Files("To_Be_Replaced_With_Path_To_Folder"),
        FilteredOutHiddenFiles = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true)
    in
        FilteredOutHiddenFiles

     

    To filter out System + Hidden files:

    let
        Source = Folder.Files("To_Be_Replaced_With_Path_To_Folder"),
        FilteredOutSystemAndHiddenFiles = Table.SelectRows(Source, each
            ([Attributes]?[System]? <> true) and
            ([Attributes]?[Hidden]? <> true)
        )
    in
        FilteredOutSystemAndHiddenFiles

     

Resources