Forum Discussion
Errors in source file retrieval from the knowledge base in copilot agents?
Hi all,
I understand that copilot agents are connected to MS Graph, which maps the relationships between all the data stored in your MS 365 tenancy (sharepoint, onedrive files, emails etc).
Recently, I created an agent and assigned a specific folder to the knowledge base and turned off the "use web content" toggle, because I wanted the responses to be very directly tailored to my folder (inclu. sub-folders with multiple files).
I then tested if/how well the agent retrieved specific files using this prompt:
"Can you please tell me how many files are in this folder and list the files in the folder? [Insert link to sub-folder in from the main folder in the knowledge base]"
The agent responded with (1) an incorrect count and (2) listed a few files that were not in the sub-folder but in another part of the knowledge base.
As I understand it, it is a counting error in (1) and retrieval+indexing error in (2). I'm more concerned about (2) because I'm worried the agent isn't retrieving (and therefore, using the info in) all the files in an important folder (when specifically linked to it even).
Questions:
(a) Where is this error happening in the indexing process within MS graph? Am I misunderstanding where the error lies? Any ideas on why an agent is naming the wrong files in a folder within its own knowledge base??
(b) Do agents created within the copilot agents web interface use Azure AI Search for semantic indexing or is that only for more custom RAG solutions created "from scratch" using foundry, SDK, etc? Do copilot agents use Microsoft Search to query and index files used in a response?
Thanks!
1 Reply
- Aggarwalatul
Microsoft
Hey mv2026 - what you are expeienceing is expected behaviour. Copilot agents use semantic retrieval, not deterministic file or folder enumeration. Agents cannot reliably count files or restrict responses strictly to a sub‑folder, even when web content is disabled. Files mentioned from outside the folder likely matched the semantic intent and exist within the broader indexed knowledge base. Copilot agents created via the UI use Microsoft Search / Microsoft Graph–based indexing, not Azure AI Search. Azure AI Search is only used in custom RAG solutions built with Foundry or SDKs.
For validation, test with content‑specific questions rather than file counts or folder listings.