Event banner
Event details
Copilot seems to have very limited knowledge and understanding of the structure or the filesystem itself. I.e., folder and file organization, relative locations, even what "contains" means. I have the sense this isn't really something in the "nature" of LLMs, but here goes:
e.g.,
- query:"are there any ps1 files in the workbench folder in my onedrive?"
- reply:"Yes, there is at least one ps1 file in your workbench folder in your OneDrive. The file refreshXMLAUser.ps1 was mentioned in the bpcm-2181_deploymen_essbase_filest.txt file1 Is there anything else you would like to know?"
There wasn't any such file there. But there was another file that had content *describing* it. So again it is all content based. No "cognizance" of structure of filesystems.
However, if i limited it to questions like this:
- query:"in my onedrive folder, are there any folders with the word "project" in their names?"
- query:"in my onedrive are there any folders named "crypt"?"
Results seemed accurate and useful. And yet, this resulted in a "sorry don't know":
query:"of those folders, how many contain files with the filename extension "SVG"?
underneath the special projects folder, and recursively descending, find files with the filename extension "SVG" "
Are there prompting or configuration techniques that will give Copilot "awareness" of file-and-folder structures useful for querying? And/or is this something that is on the roadmap?
Hey Justin, it may be that your user data is still being indexed and therefore Copilot may not have a reference to it yet. If you're an admin, you can check the status of indexing through the Microsoft 365 admin center. Check out these articles for more https://learn.microsoft.com/en-us/MicrosoftSearch/semantic-index-for-copilot#enablement
as well https://learn.microsoft.com/en-us/MicrosoftSearch/semantic-index-for-copilot#enablement
Otherwise, the index isn't so much about file structure or system, more so about the relevancy and context of the data.
- Understood, thank Gabe. I guess my feedback on that point is this: many businesses will actually capture a lot of context and "metadata" in the filesystems. E.g., folder structure is regional hierarchy state/district/county/city/office and then under that maybe a couple of special folders for "YET_TO_BE_PROCESSED". And even if it is best practices to have rich detailed "document properties" type metadata that also captures this, there are many filetypes that do not offer such metadata (even if I could get the budget to have people go back across all the files and insert and update metadata like that).
- Appreciate that feedback! I'll take that back to the team. We're continuing to invest in Search, Copilot being a critical beneficiary of that so I'm sure they'll appreciate the insight here.