Agentic Mentor: A Specification-Driven, Multi-Agent Learning Tool
Project Overview Rather than relying on the model to know when the student understands, we built a pipeline where progress itself is gated: a student cannot move forward until they can demonstrate they understood what was just built. And rather than depending on commercial, token-billed APIs to make this teaching possible, the system is designed to run on locally hosted or free-tier models, keeping it accessible to students and institutions alike. As our contribution to Microsoft, we've handed over the full public repository behind Agentic Mentor. Given how central token-based services have become to software development, we see real potential for this approach in initiatives like GitHub Education, where affordable, agentic tools for students matter. In this post, we'll walk through what motivated the project, how Agentic Mentor works under the hood, and what we learned putting it to the test on a real piece of university coursework. The Project Journey This project was completed over three months. The first few weeks were spent on background research and requirements elicitation. We reviewed the literature on AI-assisted learning, specification-driven development and prompt ambiguity, elicited functional and non-functional requirements from our client supervisor at Microsoft, Lee Stott, and our academic supervisors, and broke the work into components we could build and test independently. This told us what was realistically achievable in the time we had, and which features were core to the tool rather than desirable extras. Implementation was coordinated through GitHub, with the team meeting daily to keep parallel work in sync and weekly meetings with both our supervisors and our client to report progress and check we were still building the right thing. We worked this way because the shape of the tool was still settling once implementation began, and that frequency of contact meant a wrong assumption surfaced in days rather than weeks. The most consequential decision of the project came out of those conversations. Agentic Mentor had been proposed as an assessment tool, but the AI does not write the assessment brief, so it cannot be relied on to read it as the academic intended, and grading on top of that reading would carry the model's misunderstanding into a student's mark. We pivoted to a learning tool, and the design of the pipeline followed from there. The final month went on completing both interfaces and evaluating the system, first against SpecBench as a correctness check, then against a real master's-level coursework. Two problems surfaced. Our GPU infrastructure went offline, so we moved to free-tier cloud models and scheduled runs around quota resets. And one of the questions the system generated turned out to be subtly wrong, and we had read it and accepted it without question, which is the automation bias we had spent the project writing about. Technical Details Agentic Mentor is built as a multi-agent, specification-driven pipeline. It has four agents; Research, Ingestion, Mentoring, and Viva. Each agent was implemented using the Microsoft Agent Framework. We chose to give each phase its own dedicated agent so that every stage could be equipped with the specific tools its task required, rather than relying on one general-purpose agent to handle the whole assignment. The agents run linearly, with each one writing its output to disk and the orchestrator passing the resulting file paths to the next stage. This design lets a session pause and resume. An overview of Agentic Mentor's architecture is seen in the figure below. Research Agent. This stage grounds the pipeline in external context. It uses GitHub and arXiv MCP servers to gather relevant literature and existing implementations. MCP gave the agent a uniform interface to both sources, meaning further ones could be added later simply by connecting another server, without reworking the agent itself. Ingestion Agent. Built around GitHub SpecKit, an open-source toolkit for spec-driven development, this agent converts the assignment brief and research context into structured specification files. Rather than letting the model resolve ambiguities in the brief on its own, SpecKit's clarification step surfaces unclear points directly to the student, who must answer before the pipeline continues. This choice keeps the student engaged in design decisions rather than letting the model make them silently. Mentoring Agent. This is where the student and the model actually build the project together, phase by phase, with the agent writing code and explaining its reasoning as it goes. Progress is gated: the student cannot move to the next phase until every task is complete, and a short multiple-choice checkpoint has been passed. The checkpoint tests understanding of what was just built rather than simply advancing on request. Viva Agent. Once implementation is complete, this agent interviews the student. Answers are checked against key points prepared in advance, with feedback given after each response and a full transcript saved for later review. How Microsoft's tools shaped the project. The Microsoft Agent Framework was the backbone that made the multi-agent design practical: it let us build four functionally distinct agents that could each carry their own tools and responsibilities while still communicating cleanly through a shared orchestration layer. Together with GitHub SpecKit's structured approach to specification-driven development, these tools gave Agentic Mentor a technical foundation that would have been considerably harder to assemble from scratch, and were central to making the pipeline's phase-gated, specification-first design actually work in practice rather than remaining a concept on paper. Demo This demo video shows an end-to-end demonstration of the VS Code extension of Agentic Mentor. Results and outcomes We evaluated Agentic Mentor in two phases: a benchmark check for basic correctness, followed by a real master's-level coursework that better matched the system's intended use. Phase 1: Code Correctness We used SpecBench to evaluate Agentic Mentor’s code correctness. SpecBench is a benchmark of 30 systems-level programming tasks with pre-existing test suites. We ran six of them using a locally served Qwen3.6-27B model. Our results show that tasks with common structural logic scored well, such as json_parser achieving a 97.7% pass rate. However, more complex tasks performed worse. On crypto_primitives, hallucinations prevented a testable solution being produced at all. This result reflects the limits of a small local model such as Qwen3.6-27B. However, the success of some tasks show that smaller local models have promise. Phase 2: Student Understanding The second phase evaluated the system against a real academic assignment on Test-Driven Development, which the team had previously completed without Agentic Mentor. Claude Sonnet 5 was used to reduce hallucinations and improve reasoning. We combined our own assessment as students with an interview with Jens Krinke, the module leader who set and assessed the coursework. From our perspective, we found that Agentic Mentor made a large, loosely specified project easier to approach, and the questions at each stage required active recall which helped us understand the assignment better. It improved on our original attempt by constructing a synthetic repository with known test–production pairs as concrete acceptance criteria, linking test and production files by co-occurrence in commit history rather than naming conventions, and producing unit test coverage within a properly separated project structure. The assessment of its understanding was less favourable. Jens rated its comprehension as comparable to a typical student's, but no better: it treated commit history as capable of confirming the presence of TDD, when it can only reveal the degree of its absence, and it missed an implicit hint regarding commit size. Lessons Learned Building Agentic Mentor taught our team a great deal about coding agents, local models, and the practical challenges of applying AI in education. These lessons shaped both the tool itself and how we think about deploying AI in learning contexts. Knowing where the tool fits. One of our clearest takeaways was that Agentic Mentor works best on well-specified, undergraduate-level assignments rather than open-ended, research-style coursework. Our evaluation on SpecBench's json_parser task, where the tool achieved a 97.7% pass rate, showed just how effective it can be on concrete, well-bounded problems. Our Test-Driven Development case study showed the same tool struggling once the task demanded interpreting ambiguous or partially hidden objectives. Even so, our own experience using the tool showed genuine improvements in our learning outcomes and even surfaced implementation ideas we hadn't considered ourselves, which reinforced our belief in the promise of agentic learning tools when applied to the right kind of problem. Navigating the local model trade-off. Our extensive experimentation with locally hosted models taught us that there is a trade-off between accessibility and quality. Local models performed well on smaller, simpler tasks, but as task complexity grew, code correctness declined and hallucinations became more frequent. This was a valuable lesson in engineering trade-offs: a fully local, cost-free deployment is achievable in principle, but a genuinely effective one currently depends on access to larger, cloud-hosted models. Rethinking what the tool should be for. Perhaps our biggest shift in thinking came from an idea we abandoned. We originally envisioned Agentic Mentor as an assessment tool, capable of evaluating a student's understanding for official grading. Working through the implementation made clear why that vision doesn't hold up: an AI system that didn't generate the assessment brief itself cannot be guaranteed to interpret an educator's intent correctly. This is a limitation of natural language interpretation that is well documented in the literature we reviewed. Recognizing this early enough to change course was itself a valuable exercise in engineering judgment, and it led us to reposition Agentic Mentor as a learning tool focused on helping students engage with agentic AI, rather than an assessment tool that asks AI to make judgments it isn't equipped to make reliably. Implications for Educators Our experience building and evaluating Agentic Mentor highlighted three critical takeaways for educators looking to integrate AI into their classrooms. Watch out for automation bias. Our academic evaluation revealed that the AI agents occasionally held technical misconceptions, which raised up the risk of automation bias; one of the biggest risks in AI-assisted education. When an AI speaks with absolute confidence, it’s incredibly easy to believe it. If students trust the model's outputs without sufficient questioning, they bypass the critical thinking the coursework was meant to provoke. Educational AI needs built-in friction to force the AI to highlight its own uncertainties and encourage students to question the output. Be careful using AI as an Assessment tool. The discovery of the AI’s technical misconceptions also suggests caution using AI for grading or formal assessment. Because the model can misunderstand core concepts the exact same way a student might, it cannot reliably evaluate student comprehension. This also led us to pivot Agentic Mentor strictly to a learning tool. AI is an excellent tool for supporting student learning through phased tasks, but it is not currently reliable enough to act as an autonomous judge of a student's underlying understanding. Agentic Development is reshaping software engineering. As agentic development reshapes software engineering, educators must shift their focus from teaching students how to write code manually, to teaching them how to prompt coding agents. This includes using practices such as specification-driven development, which reduces AI errors in code by preventing ambiguities in natural language prompts. Future Development The limitations we encountered while building Agentic Mentor point to two clear directions for further development. Both would make the system a more reliable learning tool for students. Validation agent Agentic Mentor currently interprets a project with the same gaps and misconceptions a typical student might bring to it. Because the system presents its output with confidence, those gaps can be passed on unchallenged. We propose extending the pipeline with a validation agent. This component would be dedicated to interrogating the questions and conclusions the system produces. Its purpose would be to introduce a deliberate layer of hesitation, surfacing assumptions explicitly rather than allowing them to reach the student as established fact. The intended outcome is a reduction in automation bias, encouraging students to engage critically with the system's output rather than accepting it uncritically. Integration with academic platforms As long as the system's understanding of an assignment is constrained in the same ways a student's is, it cannot be relied upon to identify a task's critical elements. A future version could accept explicit input from educators, specifying those critical elements in advance along with guidance on how students should be directed through them. This would require Agentic Mentor to move beyond a standalone local tool and integrate with the institution's existing education platform. Such integration would give educators direct control over the system's behaviour, making it both more trustworthy and more productive as a learning aid. Conclusion Agentic Mentor was built to help students learn from their coding coursework rather than simply complete it, embedding the learning process into the structure of the system instead of relying on vibe coding. We found that a model cannot be trusted to be consistently correct. Running on smaller local models trades capability for accessibility. And measuring student understanding is difficult without a large user study. Agentic Mentor nonetheless represents a step towards integrating AI-assisted development into education while preserving student understanding. With adequate guardrails against automation bias, it can serve as an accessible tool that adapts the learning process to the changes AI has brought to software development. Call to Action The challenge of "one-click" AI code generation in education is here to stay, but with Agentic Mentor, we aim to keep students actively engaged in problem-solving rather than bypassing it. We invite you to explore our work and help us build the future of AI in education. Explore the Code: Visit the Agentic Mentor GitHub Repository Run It Your Way: Execute the orchestrator via the CLI (see the main README), or use our custom GUI by running the VS Code Extension (located in the agentic-mentor-extension folder). Tools: Our pipeline is built on the Microsoft Agent Framework for multi-agent routing and GitHub SpecKit to enforce strict Specification-Driven Development. Team Our team involved in developing this project included 6 members. All of us are Masters students at UCL, studying either Software Systems Engineering or Artificial Intelligence and Data Engineering. Mark Connor – Team Leader – Software Engineer GitHub URL: https://github.com/markjconnor LinkedIn URL: http://www.linkedin.com/in/mark-connor2003 Alexander Filippov – Software Engineer GitHub URL: https://github.com/ucabavf LinkedIn URL: https://www.linkedin.com/in/alexander-f-003a5721b Weeraya Hew – Software Engineer GitHub URL: https://github.com/tingwry LinkedIn URL: https://www.linkedin.com/in/weeraya-hew-924a19261 Tanishka Jaikrishnia – Software Engineer GitHub URL: https://github.com/tanishkajaikrishnia LinkedIn URL: https://www.linkedin.com/in/tanishka-jaikrishnia-96b652274/ Pranav Kannan – Software Engineer GitHub URL: https://github.com/pranavk295 LinkedIn URL: https://www.linkedin.com/in/pranav-kannan-0b2a11221 Gabriel Mardakhaev – Software Engineer GitHub URL: https://github.com/gabmardakhaev LinkedIn URL: https://www.linkedin.com/in/gabriel-mardakhaev Special Thanks to Contributors We want to express our deepest gratitude to the following contributors, whose ongoing support and dedication led to the success of this project. Lee Stott, Principal Cloud Advocate Microsoft He Ye, Academic Supervisor, UCL Jens Krinke, Senior Lecturer and Academic Supervisor, UCL
Securing AI Agent Tool Calls in Azure: Identity, Authorization, and Verified Execution
When an agent can use tools, untrusted content can influence requests to access sensitive data or change external systems. A retrieved document might tell the agent to send a file to an external address. An attacker-controlled log field might suggest disabling a security setting. A tool result might redirect the next action. The application must establish who is acting, authorize each requested operation, constrain execution, and verify what happened. System instructions, content filters, and model evaluations help reduce unsafe proposals. Token validation, retrieval permissions, business authorization, execution limits, and disclosure checks govern what the application actually permits. This article uses an Agent Control Loop as a design-review framework for placing those checks at five observable handoffs: Input → Context → Tool Request → Execution → Output Tool results return to Context because they can influence another model decision. They need the same attention to source permissions, provenance, and untrusted content as retrieved documents. The framework should fit the workload. A read-only assistant needs strong retrieval and disclosure checks when it handles sensitive information. A routine transaction may need only service-side authorization and safe retry handling. A delayed, high-impact operation may also need approval, expiry, current-state checks, and recovery procedures. Choose safeguards according to data sensitivity, action scope, reversibility, delay, and the consequence of failure. In this article: Input | Context | Tool Request | Execution | Output | Review questions Guide and test the model's proposals Model behavior remains an important part of the design: Guide the model with system instructions, task boundaries, and a limited tool set. Ground its decisions in permitted sources with provenance metadata. Review observable artifacts such as tool arguments, citations, assumptions, uncertainty, and concise decision summaries. Test behavior with offline evaluations and adversarial regression tests. These measures reduce the likelihood of a poor proposal. The receiving service must still validate its arguments and authorize the requested operation. Authorization also has limits. A request can satisfy business rules while conflicting with the user's intent. Narrow workflow scope, cumulative action limits, and review when consequences justify it help address that remaining risk. Token validation, authorization rules, and execution limits must run outside model-controlled context. Every path to a protected capability—including direct API calls, background jobs, and retries—must pass through the applicable checks. 1. Input: validate identity and enforce request limits Input includes the user's prompt, attachments, conversation state, and request metadata. For an authenticated enterprise workflow, establish the caller's identity before loading protected conversation state or retrieving enterprise data. Validate the token's signature, issuer, audience, and lifetime. Check the expected tenant, client application, and required claims. Derive the acting principal and tenant from validated identity information, and resolve application permissions through trusted server logic. Keep this authorization context separate from model-editable content. A tenant name in a prompt or a role asserted in an attachment must not change the caller's permissions. Apply request-size limits, attachment-type restrictions, rate limits, and relevant data-handling rules before content reaches the model. These checks protect application capacity and reduce avoidable exposure of sensitive content. Microsoft Entra ID can authenticate users and workloads. Azure API Management can enforce token-validation policies, required claims, quotas, and rate limits. Azure AI Content Safety Prompt Shields can add detection for direct prompt attacks. A detection result may trigger rejection, isolation, or additional review. Passing that check does not authorize a later tool call. A missed attack must still encounter the tool service's argument validation and authorization checks. Authentication establishes who submitted the request. It does not establish that the request is accurate, appropriate, or within the caller's business authority. 2. Context: enforce retrieval permissions and preserve provenance Context can contain system instructions, retrieved documents, database records, conversation history, memory, and tool results. These sources have different trust properties. For example, an Azure log record may have authenticated provenance while containing a filename, URI, process argument, or user-agent string chosen by an attacker. The logging service can accurately preserve that value without making it a valid instruction. A registered tool can return the same kind of content. Its response may include attacker-controlled text, unrelated sensitive information, or evidence that contradicts another source. Enforce source permissions before content enters model context. Scope retrieval to the authorized tenant and permitted documents or records. Limit tables, indexes, fields, result counts, and time windows. Construct security filters from trusted identity information, and prevent model-generated queries from removing mandatory restrictions. For Azure AI Search, use supported document-level access controls or security filters enforced by the application. Authentication to the search service alone does not establish permission to read every document in an index. Microsoft documents this distinction in its security-filter guidance. For Azure Monitor and other query tools, constrain the accessible resources and query scope through service permissions and application logic. Query templates should parameterize permitted inputs while preserving mandatory tenant and resource restrictions. Preserve source identifiers, timestamps, classification, and tenant metadata. Keep application instructions separate from retrieved evidence, and label externally influenced text as untrusted data. These labels help the model interpret content; retrieval authorization remains the service's responsibility. Require citations when conclusions depend on retrieved evidence. For consequential decisions, verify that the cited material supports the conclusion and corroborate it against another source or current system state where practical. Prompt Shields can add detection for indirect prompt attacks in document content. Microsoft Purview classification can inform filtering and disclosure decisions where classification is available and integrated into the application. Retrieved content may inform a proposed action. It must not modify the caller's identity, broaden retrieval permissions, or grant authority to another tool. 3. Tool Request: validate arguments and authorize the exact action The model produces an observable request: a tool name, target, and arguments. The receiving service must decide whether that exact request is permitted. Use a narrow tool definition. For example, a refund tool can accept an order identifier and an amount without accepting an arbitrary payment destination or caller-selected authorization limit. The following function-definition fragment describes a workflow for USD refunds. Amounts use integer cents: JSON | Refund tool definition { "name": "issue_refund", "description": "Request a USD refund to the order's original payment method.", "strict": true, "parameters": { "type": "object", "additionalProperties": false, "required": ["order_id", "amount_minor"], "properties": { "order_id": { "type": "string", "description": "The order identifier." }, "amount_minor": { "type": "integer", "description": "The refund amount in USD cents." } } } } The surrounding API format and supported schema constraints depend on the provider and model. Azure's structured-output guidance and OpenAI's guidance document different support for some constraints. Confirm compatibility with the deployment, and enforce all required validation in the receiving service. Here is an illustrative Pydantic v2 validation and authorization fragment. It assumes that principal comes from trusted authentication and permission resolution, and that order is loaded from the authoritative store through a tenant-scoped lookup. Python | Request validation and refund authorization from pydantic import BaseModel, ConfigDict, Field class RefundRequest(BaseModel): model_config = ConfigDict(extra="forbid", strict=True) order_id: str = Field(pattern=r"^ord_[A-Za-z0-9]{12}$") amount_minor: int = Field(gt=0, le=50_000) # USD cents: at most $500 class AuthorizationError(Exception): pass def authorize_refund(request: RefundRequest, principal, order) -> None: if order.tenant_id != principal.tenant_id or order.id != request.order_id: raise AuthorizationError("order is outside the requested scope") if not principal.can_issue_refunds: raise AuthorizationError("caller cannot issue refunds") if order.customer_id != principal.customer_id: raise AuthorizationError("caller does not own the order") if order.currency != "USD": raise AuthorizationError("this workflow supports USD orders only") if not order.is_refundable or order.account_fraud_blocked: raise AuthorizationError("order is not eligible for a refund") if request.amount_minor > principal.refund_limit_usd_minor: raise AuthorizationError("amount exceeds caller authority") if request.amount_minor > order.remaining_refundable_minor: raise AuthorizationError("amount exceeds the refundable balance") The service must parse the incoming arguments with RefundRequest.model_validate(...) before calling this function. The example assumes a customer refund workflow; a support-agent workflow would use its own delegated permissions and order-access rules. The payment destination comes from the authoritative payment record and must satisfy the refund policy. It is not selected from model-generated arguments. These checks do not reserve funds or prevent concurrent refunds. The service must couple authorization to an atomic transaction or conditional reservation so competing requests cannot consume the same refundable balance. For systems that allow partial refunds, the available balance should account for both completed refunds and active reservations. Derive the acting principal and allowed scope from trusted server context. If a tool accepts a target user or tenant, treat that value as an untrusted target selector and authorize it independently. Authenticating a workload also does not establish the authority of the user it represents. A domain service must enforce either the represented user's permissions or an explicitly defined service workflow's authority. An Azure Function, App Service API, or another domain service can perform these checks. An external policy engine can help when rules need independent ownership or consistent reuse. Azure RBAC separately limits which Azure operations the executor's identity can perform. 4. Execution: bind approval, recheck state, and constrain privileges An authorized request can become stale before execution. The caller's permissions may change, a refund balance may shrink, or a resource may be modified while a request waits for approval. For an immediate operation within one transactional system, authorize and apply the change in the same transaction where possible. An external API call introduces a separate failure boundary and may require idempotency and reconciliation. When human approval is required, store an approval record that binds the decision to the exact operation, canonical target, parameters, requester, tenant, and expiry. Validate the approver's authority, and require a new decision if the approved action changes. Before execution, check approval validity where applicable, re-evaluate current authorization, and re-read the state relevant to the operation. Use conditional writes when the target API supports the required concurrency condition. A single-use approval claim prevents approval reuse. It does not by itself prevent duplicate external effects. A worker can submit a payment and crash before recording the result. Where supported, bind a downstream idempotency key to the authorized action and reuse that key for retries. If the outcome is uncertain, reconcile with the downstream system before submitting another operation. Use an executor identity with only the resource scope and API permissions the workflow requires. A separate managed identity is useful when execution needs privileges that the model-facing component should not hold. Restrict outbound destinations and the data that may be sent to each destination. Example: verify an Azure Storage configuration change The following execution fragment changes one Storage account property. It assumes that the service has already loaded a trusted execution record, validated any required approval, and authorized the exact subscription and resource target. The expected state comes from that record. Python | Azure Storage execution and verification import os from azure.identity import ManagedIdentityCredential from azure.mgmt.storage import StorageManagementClient from azure.mgmt.storage.models import StorageAccountUpdateParameters def disable_public_network_access(command): client = StorageManagementClient( credential=ManagedIdentityCredential( client_id=os.environ["EXECUTOR_MANAGED_IDENTITY_CLIENT_ID"] ), subscription_id=command.subscription_id, ) before = client.storage_accounts.get_properties( command.resource_group, command.account_name, ) if before.public_network_access != command.expected_public_network_access: raise RuntimeError("resource state changed after authorization") client.storage_accounts.update( command.resource_group, command.account_name, StorageAccountUpdateParameters(public_network_access="Disabled"), ) after = client.storage_accounts.get_properties( command.resource_group, command.account_name, ) return { "operation": "storage.disable_public_network_access", "target": after.id, "status": ( "configuration_verified" if after.public_network_access == "Disabled" else "verification_pending" ), "observed_public_network_access": after.public_network_access, } This verifies the property reported by the management API. It does not establish that every access path is blocked. Azure documents that previously configured trusted-service and resource-instance exceptions can remain effective, and that Storage firewall restrictions apply to data-plane operations. If the objective is containment, verification must also account for the relevant exceptions and access paths. See Storage network-security limitations. The Storage Accounts Update reference for API version 2025-08-01 does not document an If-Match header. The initial read can detect a difference from the expected state, but another writer can still change the resource between that read and the update. The example does not provide an atomic compare-and-update operation. If verification remains pending or a network failure obscures the result, retain the action record and reconcile current state. Do not report verified completion or blindly resubmit a new action. The Azure services involved each have a specific role: Managed identities provide credentials without embedding application secrets. Azure RBAC limits resource scope and permitted operations. A broad resource-write permission may still allow more property changes than this function exposes. Azure Policy deny rules can reject covered resource changes that violate configured policies. Private endpoints and egress restrictions constrain connectivity; application logic must still authorize destinations and payloads. Durable Functions can coordinate an approval wait and execution workflow. Cosmos DB conditional writes can protect an approval claim or action record against competing updates. The domain service remains responsible for deciding whether the business action is permitted. Workflow coordination and restricted credentials do not replace that decision. 5. Output: verify outcomes, authorize disclosure, and record evidence Output includes the tool result returned to the model, the final user response, and operational evidence. Each has a different audience and disclosure policy. Keep separate records of what the model requested, what the service authorized, what a human approved when required, what the downstream API returned, and what verification subsequently observed. A successful API response may describe an intermediate state. A refund may be accepted but not settled. A message may be queued but not delivered. A resource property may be updated while the broader containment objective remains unverified. Define the postcondition for the workflow and report the state actually observed. Use explicit statuses such as submitted, pending verification, completed, failed, or outcome unknown. For delayed outcomes, retain the action identifier and update its status as evidence becomes available. Authorize disclosure before returning data to the model or user. Apply equivalent checks before tools transmit data to another service. A response filter cannot recover information already sent in an outbound tool request. Remove secrets and unrelated personal or tenant data from tool responses, final answers, and telemetry. Collect the identifiers, decision reasons, timestamps, and verification results needed to investigate the action without retaining unrestricted prompts or credentials. Read target state through the relevant API and correlate the result with Azure Monitor or Application Insights telemetry. Microsoft Sentinel can consume the evidence when the workflow supports security operations. Use access-controlled or immutable records where audit requirements justify them. Recovery must reflect the consequence of the action. Retrying an idempotent evidence write can be appropriate. Automatically reversing a containment action because a later annotation failed can create a new security problem. Preserve partial failures and uncertain outcomes so operators can make an informed recovery decision. Logs record observations. They do not establish every aspect of the real-world outcome. State what was verified, when it was observed, and what remains unresolved. Review the five handoffs For an agentic application, ask: Input: Which token checks establish the caller, and which request limits apply? Context: Which retrieval permissions restrict the evidence, and how are its source and trust represented? Tool Request: Which argument checks and business rules authorize this operation, target, amount, and acting principal? Execution: Are approval and authorization still valid, how are competing actions and retries handled, and what can the executor change or transmit? Output: Which postcondition was observed, who may receive the result, and what evidence remains available? Continue improving the model's instructions, grounding, and evaluations. At each handoff, make the responsible service and its checks explicit. The model proposes; trusted services validate requests, authorize actions, constrain execution, and verify outcomes.Copilot, Microsoft 365 & Power Platform product updates call
💡Copilot, Microsoft 365 & Power Platform product updates call concentrates on the different use cases and features within the Microsoft 365 and in Power Platform. Call includes topics like Microsoft 365 Copilot, Copilot Studio, Microsoft Teams, Power Platform, Microsoft Graph, Microsoft Viva, Microsoft Search, Microsoft Lists, SharePoint, Power Automate, Power Apps and more. 👏 Weekly Tuesday call is for all community members to see Microsoft PMs, engineering and Cloud Advocates showcasing the art of possible with Microsoft 365 and Power Platform. 📅 On the 15th of September we'll have following agenda: News and updates from Microsoft Together mode group photo Rémi Dyon – Building an Agent with GitHub Harness in Copilot Studio Steve Pucelik + Marc Windle – Latest on SharePoint Embedded Vesa Juvonen – Surfacing your business apps in Copilot canvas – IT concierge scenario 📞 & 📺 Join the Microsoft Teams meeting live at https://aka.ms/community/ms-speakers-call-join 🗓️ Download recurrent invite for this weekly call from https://aka.ms/community/ms-speakers-call-invite 👋 See you in the call! 💡 Building something cool for Microsoft 365 or Power Platform (Copilot, SharePoint, Power Apps, etc)? We are always looking for presenters - Volunteer for a community call demo at https://aka.ms/community/request/demo 📖 Resources: Previous community call recordings and demos from the Microsoft Community Learning YouTube channel at https://aka.ms/community/youtube Microsoft 365 & Power Platform samples from Microsoft and community - https://aka.ms/community/samples Microsoft 365 & Power Platform community details - https://aka.ms/community/home 🧡 Sharing is caring!Microsoft Power Platform community call - September 2026
💡 Power Platform monthly community call focuses on different extensibility options for builders, makers and developers within the Power Platform. Typically demos are from our awesome community members who showcase the art of possible within the Power Platform capabilities. 👏 Looking to catch up on the latest news and updates, including cool community demos, this call is for you! 📅 On 16th of September we'll have following agenda: Power Platform Updates & Events Latest on Power Platform samples John Liu - How to easily to convert markdown documents to PDF with Power Automate Sanjiv Venkatram - Using Azure AI video analyzer + Functions + Storage & Power Platform to analyze echocardiograms Billur Şamdancıoğlu - Copilot Cowork for Dynamics 365 Finance and Operations Elliot Margot - Stop Clicking Through Run History: Build Your Own MCP to Debug Power Automate Flows 📅 Download recurrent invite from https://aka.ms/powerplatformcommunitycall 📞 & 📺 Join the Microsoft Teams meeting live at https://aka.ms/PowerPlatformMonthlyCall 💡 Building something cool for Microsoft 365 or Power Platform (Copilot, SharePoint, Power Apps, etc)? We are always looking for presenters - Volunteer for a community call demo at https://aka.ms/community/request/demo 👋 See you in the call! 📖 Resources: Previous community call recordings and demos from the Microsoft 365 & Power Platform community YouTube channel at https://aka.ms/community/videos Microsoft 365 & Power Platform samples from Microsoft and community - https://aka.ms/community/samples Microsoft 365 & Power Platform community details - https://aka.ms/community/homeCopilot, Microsoft 365 & Power Platform Community call
💡 Copilot, Microsoft 365 & Power Platform weekly community call focuses on different use cases and features within the Microsoft 365 and Power Platform - across Microsoft 365 Copilot, Copilot Studio, SharePoint, Power Apps and more. Demos in this call are presented by the community members. 👏 Looking to catch up on the latest news and updates, including cool community demos, this call is for you! 📅 On 17th of September we'll have following agenda: Latest on SharePoint Framework (SPFx) Latest on Copilot prompt of the week PnPjs CLI for Microsoft 365 Dev Proxy Reusable Controls for SPFx SPFx Toolkit VS Code extension PnP Search Solution Demos this time Josh Bray – Managing Power Platform Environments from SharePoint with an SPFx Web Part Yves Habersaat – Agent-Driven Power Apps: Canvas Authoring MCP Server with GitHub Copilot CLI Mohammed Amer – Build Interactive Component Components UX for Copilot canvas for Microsoft 365 Roadmap Features 📅 Download recurrent invite from https://aka.ms/community/m365-powerplat-dev-call-invite 📞 & 📺 Join the Microsoft Teams meeting live at https://aka.ms/community/m365-powerplat-dev-call-join 💡 Building something cool for Microsoft 365 or Power Platform (Copilot, SharePoint, Power Apps, etc)? We are always looking for presenters - Volunteer for a community call demo at https://aka.ms/community/request/demo 👋 See you in the call! 📖 Resources: Previous community call recordings and demos from the Microsoft Community Learning YouTube channel at https://aka.ms/community/youtube Microsoft 365 & Power Platform samples from Microsoft and community - https://aka.ms/community/samples Microsoft 365 & Power Platform community details - https://aka.ms/community/home 🧡 Sharing is caring!
Ingestion and replication: moving from detection in hours to detection in minutes.
It seems that currently, we only become aware of a problem once there is a visible impact on tracking or on the tables consumed by the business. In many cases, the root cause appears to occur much earlier—for instance, when a package or replication process stops running correctly, starts skipping steps, or begins to accumulate delays. Would it make sense to move toward earlier detection? I was thinking of something that periodically monitors data from `replication.package` and similar sources to identify anomalies before tracking alerts are triggered: Packages that haven't run in the last *N* minutes. Failed jobs. Recurring skips. Abnormal increases in the time between executions. Growing discrepancies between expected timestamps and actual processed timestamps. It could even evolve into an agent that not only alerts but also provides context: "Package X on server Y hasn't run for 35 minutes. The last execution failed, and the likely impact is on the BayCity tracking tables." My impression is that this could help us significantly reduce detection time, since we often end up investigating only after a delay of several hours has already built up. What do you think? Is there already a tool or monitoring system in place that addresses this need—one I might not be aware of? We are working with Azure Databricks; I was planning to implement this using an agent, but I need...Copilot, Microsoft 365 & Power Platform Community call
💡 Copilot, Microsoft 365 & Power Platform weekly community call focuses on different use cases and features within the Microsoft 365 and Power Platform - across Microsoft 365 Copilot, Copilot Studio, SharePoint, Power Apps and more. Demos in this call are presented by the community members 🙏 👏 Looking to catch up on the latest news and updates, including cool community demos, this call is for you! 📅 On 10th of September we'll have following agenda: Copilot prompt of the week CommunityDays.org update Microsoft 365 Maturity model PnP Framework and Core SDK extension PnP PowerShell Script samples Copilot pro dev samples Power Platform samples Marlon König – Document Analysis in Practice: AI Builder vs. Content Understanding in a Real-World Customer Project Sankalp Saoji – From Insights to Action: Building a Power BI AI Agent with Microsoft Copilot Studio Peter Paul Kirschner – Integrating UI elements into Copilot with SPFx 📅 Download recurrent invite from https://aka.ms/community/m365-powerplat-dev-call-invite 📞 & 📺 Join the Microsoft Teams meeting live at https://aka.ms/community/m365-powerplat-dev-call-join 💡 Building something cool for Copilot, Microsoft 365 or Power Platform (Copilot Studio, SharePoint, Power Apps, etc)? We are always looking for presenters - Volunteer for a community call demo at https://aka.ms/community/request/demo 👋 See you in the call! 📖 Resources: Previous community call recordings and demos from the Microsoft Community Learning YouTube channel at https://aka.ms/community/youtube Microsoft 365 & Power Platform samples from Microsoft and community - https://aka.ms/community/samples Microsoft 365 & Power Platform community details - https://aka.ms/community/home 🧡 Sharing is caring!Copilot, Microsoft 365 & Power Platform product updates call
💡Copilot, Microsoft 365 & Power Platform product updates call concentrates on the different use cases and features within the Microsoft 365 and in Power Platform. Call includes topics like Microsoft 365 Copilot, Copilot Studio, Microsoft Teams, Power Platform, Microsoft Graph, Microsoft Viva, Microsoft Search, Microsoft Lists, SharePoint, Power Automate, Power Apps and more. 👏 Weekly Tuesday call is for all community members to see Microsoft PMs, engineering and Cloud Advocates showcasing the art of possible with Microsoft 365 and Power Platform. 📅 On the 8th of September we'll have following agenda: News and updates from Microsoft Together mode group photo Rémi Dyon – Building Skills in the new Copilot studio experience Fabia Williams – Ask Copilot how your agents are being used - Introduction to Insights Agent Vesa Juvonen – Creating Copilot Components with SPFx: How Does It Actually Work? 📞 & 📺 Join the Microsoft Teams meeting live at https://aka.ms/community/ms-speakers-call-join 🗓️ Download recurrent invite for this weekly call from https://aka.ms/community/ms-speakers-call-invite 👋 See you in the call! 💡 Building something cool for Microsoft 365 or Power Platform (Copilot, SharePoint, Power Apps, etc)? We are always looking for presenters - Volunteer for a community call demo at https://aka.ms/community/request/demo 📖 Resources: Previous community call recordings and demos from the Microsoft Community Learning YouTube channel at https://aka.ms/community/youtube Microsoft 365 & Power Platform samples from Microsoft and community - https://aka.ms/community/samples Microsoft 365 & Power Platform community details - https://aka.ms/community/home 🧡 Sharing is caring!Introducing Inside Microsoft Foundry: Quickstart 🎬
Discover Inside Microsoft Foundry: Quickstart, a new video series for developers building AI agents. Starting with "What does it really take to ship an AI agent?", the series explores real-world challenges such as model selection, grounding agents in data, evaluation, deployment, observability, and governance. Follow along as we show how the Microsoft Foundry ecosystem helps developers move from prototype to production, with new episodes released in the coming weeks.
