Is "uncertainty" the feedback signal Copilot Studio agents are actually missing?
At today's M365 Platform Weekly session, we were asked for our input on what feedback we wish we could pull beyond thumbs up/down and verbatims. Is it trends over time, sentiment themes, the response-to-triage loop, etc. Here's an angle: What if the primitive itself is wrong? Thumbs up/down measures satisfaction after the fact. What if we measured confidence instead? How often an agent actually knows it's on shaky ground, and whether the user's reaction matches that? If an agent flags its own uncertainty at the point of response instead of a static thumbs up/down, a feedback prompt gets generated from whatever's trending in that uncertainty instead of the same generic question every time, and "trends over time" becomes "did this agent get more confident or less confident since the last update" rather than a flat satisfaction line. It might also solve the silence problem that most users rarely click anything. A reaction that's actually specific ("you caught something the agent flagged as shaky") seems easier to engage with than a binary good or bad. To be clear, confidence signals already exist in adjacent forms. Copilot Studio and most conversational AI platforms already use a confidence score internally to decide whether to answer directly, ask a clarifying question, or escalate to a human. GitHub Copilot has used a confidence score since its earliest versions too, ranking code suggestions and defaulting to the highest-scoring one. None of that is new. So rather than "add a percentage next to the answer", what if there is a specific flag pointing at the exact claim or step the agent is unsure about, feeding into the feedback loop? Curious if anyone else building in Copilot Studio has run into this: Do you ever wish your agent had hedged when it didn't? What would you actually do with an uncertainty score if you had one? And would just love others thoughts on this :)
Copilot Studio agents problems connecting to Sharepoint knowledge source
Hello, Since last week users in my tenant are experimenting issues regarding the connection between copilot studio agents and Sharepoint. The agents are not able to extract information from Sharepoint sites, printing there is not information in the Sharepoint site regarding the user's question when that information is in Sharepoint. These agents used to work well till last week. Does have been any update in Microsoft 365 services that can be affecting these agents ability to retrieve information from Sharepoint?how to create a globally Shared ServiceNow Connector Connection in Copilot Studio
Hello, I have configured Microsoft entra ID oauth using certificate and shared this connection with everyone in my company since this is the only shared connection on the platform. https://learn.microsoft.com/en-us/connectors/service-now/#microsoft-entra-id-oauth-using-certificate. But, whey user's ( end users, agent maker, environment maker basically any user in copilot environment) are trying to use any servicenow tools ( e.g create record) which is using this shared connection in copilot studio/ teams, they are getting below error. https://learn.microsoft.com/en-us/connectors/service-now/ How to create a shared connection which can be shared across all enterprise users in my org for copilot AI agents which are using servicenow connector? Regards, SachinError Connecting Whatsapp Number (Copilot Studio to Azure ACS)
Please help. I created an AI Agent in Copilot Studio for connecting to WhatsApp. I created an Azure Communication Services (ACS) and connected my personal Facebook page to it. I created a WhatsApp Business number, I got the code, and it connected. However, when I return to Copilot studio to connect using my Azure Subscription and Resource, it says- Error connecting to WhatsApp phone number. Please try again. How do I resolve this?
White paper: Choosing between the GitHub Copilot and Standard harnesses in Copilot Studio
The Standard harness and GitHub Copilot harness are two authoring and runtime options within Microsoft Copilot Studio. A harness is the operating layer between the model and the agent’s configuration. It determines how the model receives context, uses instructions and tools, interprets results, and moves through a task toward completion. Put simply, the model provides the reasoning capability, while the harness equips and directs it. Both harnesses are built for task-based, multi-step agents that create real business value, but they are suited to different scenarios. The Standard harness supports consistent, reliable execution of bounded business processes, while the GitHub Copilot harness extends these capabilities to longer-running, coordination-heavy, and reasoning-intensive work at a larger scale. This paper explains the key differences, tradeoffs, and scenarios to help you choose which harness is right for your business process. Read the full paper by clicking the PDF attachment below.
What is the best file format for an AI agent knowledge base?
This is a best practice sharing the best format for an agent and show you why you should convert your PPT, PDF, WORD into a TXT markdown. I had an issue with my agent, time taken to answer was too long, and usually we spend a lot of time asking: What is the best prompt? Why is my agent slow? Why does retrieval sometimes work and sometimes fail? How can I improve answer quality? But I realised I was asking another question much less often: What is actually the best file format for the knowledge base? PDF? Raw text? Markdown? Pre-chunked text? Semantic sections? Context-enriched text? And more importantly: How much does the format alone affect agent performance? I tried to find a quantified benchmark answering this specific question, with the same agent, same source knowledge and same questions, but different knowledge representations. I couldn't find one that really answered what I wanted to measure. So I decided to run the experiment myself on a real case. My first exploratory tests were already surprising: depending on the representation, the agent could be significantly faster and more accurate, despite working from the exact same source information. So I decided to push the test further. My objective I want to identify, without assumptions and based on actual evaluation data, how a long document should be prepared for an LLM knowledge base so that the agent can retrieve, understand, ground and answer from it as reliably as possible. I focused on five dimensions: Answer quality Retrieval reliability Source grounding / citations Execution time Robustness across single-turn and multi-turn questions The broader question I'm trying to answer is: How should we structure knowledge so that an LLM can retrieve and use it as reliably as possible? The test case I deliberately chose a document that isn't particularly friendly for RAG: a 46-page European regulation, https://eur-lex.europa.eu/eli/reg/2011/1169/oj?locale=fr, on the provision of food information to consumers. The information is distributed across articles, definitions, exceptions, annexes, tables, numerical thresholds and cross-references. That makes it useful for testing retrieval: answering correctly often requires finding a very specific piece of information while preserving enough context to understand how it applies. I used the native PDF as the baseline and created 6 additional knowledge-base representations of the same document: Raw TXT Markdown Chunk-ready TXT RAG-oriented units Semantic TXT Contextual TXT One rule: same knowledge, same agent, same instructions, same questions. Only the knowledge representation changes. The benchmark I used two evaluation sets: 42 single-turn questions testing broad coverage of the document: direct facts, thresholds, exceptions, annexes, lists and cross-references. 5 multi-turn conversations containing 13 questions, to see what happens when a user asks a question and then follows up with things like: "And in this case?" "What are the exceptions?" "And for dietary fibre?" This gave me: 47 evaluated test cases / 55 actual questions per format Across all 7 formats: 329 evaluated conversations 385 user questions executed First results Metric Native PDF Best structured representation Overall pass rate 66.0% 85.1% - Contextual TXT Best single-turn score 69.0% 88.1% - Chunk-ready TXT Multi-turn benchmark 40% 80% - Contextual TXT Multi-turn execution time 14m24 5m54 Total benchmark time 44m31 24m49 The quality gap was already substantial: 66.0% → 85.1% That's +19.1 percentage points while keeping the underlying knowledge unchanged. I also saw a major difference in execution time. On the multi-turn test: 14m24 → 5m54 That's approximately 2.4× faster. Across the complete benchmark: 44m31 → 24m49 Around 44% less execution time. These timings represent the complete agent evaluation pipeline, so they shouldn't be interpreted as pure LLM inference latency. But the difference under identical test conditions is large enough that I want to understand it better. Findings There wasn't one format dominating every benchmark. Chunk-ready TXT scored highest on independent questions: 88.1%, while Contextual TXT performed better across multi-turn conversations and finished with the highest overall score. That may suggest that the way we optimise a document for isolated retrieval isn't exactly the same as the way we should prepare it for conversational retrieval. In the contextual version, I tried to make every section understandable when retrieved independently by keeping useful information around it: Source references Section context Retrieval cues Relevant cross-references For regulatory documents, this seems particularly important. A numerical value retrieved alone can be meaningless without knowing which rule it belongs to, under which conditions it applies, and whether another article contains an exception. Where I am now This remains an exploratory benchmark: One document One domain One agent setup One evaluation framework One run per configuration There are plenty of things I still want to test: repeated runs, retrieval-level evaluation, token consumption, larger knowledge bases, other document types, chunk sizes, overlap, contextual headers, and more. But these first results already convinced me that the preparation of the knowledge base deserves much more attention when evaluating an agent. We often spend hours refining instructions while the same information may behave very differently depending on how it reaches the retrieval layer. Next step I'll share the prompts, knowledge-base formats and evaluation methodology on GitHub so the experiment can be reproduced and challenged. I'll keep enriching the repository as I test new formats, improve the evaluation set and add new results. If people here have ideas, edge cases or formats worth testing, I'd genuinely like to include some of them in the next iteration. What would you test next?
How to Integrate Copilot Studio Agent with a Website Using API?
Hello Team, I have created a Copilot Agent using Copilot Studio and published it on our public website using an iframe. Currently, the agent does not have any authentication configured because we want it to be publicly accessible. However, our website development team has raised a security concern with this approach, as embedding the Copilot Agent directly using an iframe may not be the most secure or recommended approach. We are looking for an alternative integration approach. For example: Is it possible to expose the Copilot Studio Agent through an API or another secure endpoint? Can our website development team call the agent through an API, receive the response, and build their own custom UI instead of embedding the agent using an iframe? Is there any recommended architecture or Microsoft-supported approach for securely integrating a Copilot Studio Agent with a public-facing website? If anyone has implemented a similar solution or has any recommendations, I would appreciate your suggestions.Unanswered Questions on GitHub Copilot Harness in Copilot Studio
We're piloting the GitHub Copilot harness in Copilot Studio (GA August 2026) and several operational and architectural details remain undocumented in the GA FAQ, Microsoft Learn, or licensing guides. Looking for official answers or PM contacts on: Architecture & Execution – When the harness breaks tasks into subtasks, does it use internal sub-agents or only skills/connected agents, what are the exact timeout/retry/max-execution-duration limits for long-running workflows, and are planning/context-retrieval/orchestration internals documented anywhere or is the orchestrator a black box? Model Selection – Can individual skills within one agent use different models or is selection strictly agent-level, how are models chosen internally when multiple skills execute, are any internal models developer-configurable, and what's the roadmap for models being added/retired/deprecated plus the lag between public release and Copilot Studio availability? Cost & Token Optimization – How exactly is the ~45% token reduction achieved, how much control do makers have over context/caching/retrieval/tool calls, what are per-model credit consumption characteristics, which models are most cost-effective for specific workloads, and what's the minimum credit cost for trivial interactions? Memory Management – What are retention periods for session/working/agent memory beyond the documented 28-day user-memory expiry, is true long-term memory supported, and what changed versus earlier implementations? Knowledge Retrieval – Can skills or system instructions influence retrieval strategy/document selection/prioritization/filtering, can planning stages perform conflict/duplicate/version detection before retrieval, and how does the harness decide which sources to search? Apps Feature – What is the "Apps (preview)" capability for, when does it GA, and how does it differ from workflows/skills/adaptive cards/agents? Billing & Credit Sizing – Is there a framework to classify users/agents by expected consumption and size credit allocation per group (citizen vs pro developers), and what's the minimum/typical consumption for simple/medium/heavy interactions? Governance & Admin – Can usage limits be set at user level (not just environment/agent), is there an API/IaC path for large-scale credit assignment, can non-admins view their own consumption/remaining allocation, and is there a self-service request-more-credits dashboard? ALM & Environments – What's the recommended path to move harness agents across Dev/Test/UAT/Prod (Solutions/ALM "setup differs" per parity chart—how?), does GitHub integration replace or complement solution-based deployment, are there recommended AgentOps practices for source control/releases/versioning, and what baseline credits and onboarding model are suggested for citizen developers under usage billing—any enterprise reference implementations?
How to preserve document format and structure with Copilot Agent
Hi everyone, I was building a copilot agent that should be able to accept a document and then make changes to it before returning the new version of that document, which has the same format and structure as the original. What is the best way to accomplish this in copilot studio? So far, I have tried using a power automate flow and external tools but it seems that there is no free included tool that can achieve this task. All the document writing tools seem to be paid services requiring API keys. Is there an alternate free way of achieving this task? Thank you
