Copilot Studio Knowledge Source Limitation When Iterating Over Multiple SharePoint Documents
Hi, I’m looking for clarification on a limitation we’re currently encountering in Copilot Studio that is blocking some of our use case. Example Scenario (Policy Agent) We have a SharePoint document library containing ~100 policy documents. A Copilot Studio agent is configured with this library as a knowledge source. The agent performs well for typical question-answering scenarios where responses can be derived from a subset of documents. For example: “How much annual leave can I take?” correctly returns answers sourced from multiple relevant policies. Issue When the question requires the agent to evaluate all documents individually, the results are incomplete. Example prompt: “Review each policy document and return the review date.” In this scenario: The agent only processes the first ~10 documents. It then stops, without indicating that the response is partial or that a limit has been reached. The remaining documents in the library are not evaluated. During a recent Microsoft-led course, we were advised that this behaviour is expected due to platform limitations. Specifically: While it will reside over all documents to genereate the most suitable response, the agent is not designed to self‑iterate across all items in a large knowledge source for individual document responses. Asking it to “review each document” effectively requires iteration, which is constrained. The suggested workaround was to: Create a trigger-based flow Implement a loop to process the documents in batches We were able to make this approach work, but it feels like a heavy and brittle workaround for what seems like a common enterprise requirement. We’ve Tried Both available SharePoint knowledge source connection methods Allowing sufficient time for indexing and refresh Rephrasing prompts to encourage broader coverage None of these approaches changed the outcome, the agent consistently returns results for only the first subset of documents. Is this behaviour a documented or known limitation of Copilot Studio knowledge sources? Are there recommended design patterns for scenarios that require document-by-document evaluation at scale? Is there a more native or supported approach planned to avoid custom looping logic for this kind of use case? Any guidance or confirmation would be appreciated. Thanks.
Designing a Governed RTO Compliance Agent Using Copilot Studio and Databricks Genie
Enterprise AI adoption in HR scenarios comes with a unique challenge: how do you deliver actionable insights without compromising privacy, trust, or policy boundaries? In this blog, I’ll share how we built an RTO (Return‑to‑Office) Compliance Agent using Microsoft Copilot Studio and Databricks Genie, focusing on governance‑first design, controlled data access, and real‑world enterprise constraints. This solution was developed as part of an HRLT proof‑of‑value initiative and is designed to support people managers with clear, aggregated compliance insights, delivered conversationally inside Microsoft Teams. The Problem We Were Solving As hybrid work models mature, organizations need a reliable way to answer questions such as: How compliant is my team with RTO expectations? Are there trends across regions or time periods? Traditional dashboards often fall short because they: Require manual interpretation Expose too much granular data Are difficult to govern at scale Our objective was to create an AI‑powered conversational interface that provides: Only manager‑authorized, aggregated insights Zero visibility into individual‑level behavior Built‑in enforcement of HR and privacy policies Architecture Overview The solution integrates Copilot Studio with Databricks Genie, backed by curated data sources. (Image: High-level Copilot Studio and Databricks Genie architecture) Key Components Copilot Studio – Conversational orchestration, policy enforcement, and Teams deployment Databricks Genie – Governed natural-language interface to curated datasets RokFusion Platform – Trusted HR and badge-swipe data This layered approach ensures governance is applied before data is ever queried. Controlled End-to-End Data Flow The interaction pattern follows a strict, auditable flow: A manager asks a question in Copilot Studio Copilot forwards the request to Genie with instruction constraints Genie executes logic only on curated, approved tables Calculations are performed at team or manager level only Copilot formats and returns compliant responses (text, tables, or charts) At no point are employee IDs, badge events, or individual metrics exposed. Using Genie as a Governance Layer, Not Just a Query Tool One of the most critical decisions was to treat Databricks Genie as a policy‑enforcement layer, not merely a natural‑language SQL generator. (Image: Genie instruction configuration enforcing compliance rules) What We Configured in Genie Synonyms and NL mappings for HR terminology Strict filtering logic for employee categories Population threshold enforcement (minimum count) Explicit rejection of sensitive attributes such as gender, race, religion, or age Prevention of formula or row‑level data exposure This approach ensured that even malformed or risky prompts could not bypass policy constraints. Compliance Scenarios Supported The agent supports multiple business‑aligned interpretations of RTO compliance: Hybrid Compliance Hybrid employees counted only on eligible hybrid days Onsite Compliance Onsite employees counted across standard working days All Employees View Weighted aggregation combining hybrid and onsite logic These scenarios are embedded into the agent’s instruction logic, not dynamically inferred at runtime—ensuring consistency and auditability. Why We Chose Conversational AI Over Dashboards A key insight early on was that managers don’t want spreadsheets—they want answers. Instead of navigating filters and charts, managers can ask: “What was my team’s compliance last week?” “Show me a comparison across regions.” When required, the agent can also render simple visual outputs. (Image: Sample Microsoft Teams output with compliance visualization) Importantly, visuals follow the same governance rules as text responses. Publishing and Validation in Microsoft Teams Once configured, the agent was published directly from Copilot Studio to Microsoft Teams, making adoption frictionless. (Image: Publishing Copilot Studio agent to Microsoft Teams) End‑to‑end testing validated: Authorization boundaries Population rules Safe handling of incomplete or ambiguous queries Key Engineering Learnings Governance must be instruction‑driven Relying on frontend filtering alone is insufficient for HR data. Natural language needs strong guardrails Enterprise AI benefits from being constrained, not free‑form. Aggregation builds trust Managers are more comfortable with insights when they know individual visibility is impossible. Copilot Studio accelerates enterprise delivery Security, deployment, and integration stay within the Microsoft ecosystem. Closing Thoughts This RTO Compliance Agent demonstrates how Copilot Studio and Databricks Genie can be used to build governed, enterprise‑ready AI solutions—especially in sensitive domains like HR. By embedding policy into architecture, instructions, and data access, we were able to deliver: Useful insights Strong privacy guarantees High user trust This pattern is extensible well beyond RTO—opening the door for future HR intelligence use cases built on the same foundation.
