Goodstack-Validated Nonprofit Rejected After Forced Resubmission — No Escalation Path
I am posting this because Microsoft Elevate frontline support told me there are no further escalation paths available, and I believe this may affect other nonprofits. Timeline of facts: - June 19, 2026: Goodstack, Microsoft's authorized validation partner, confirmed our nonprofit (Opera Verace Foundation, EIN 33-2305878, California 501(c)(3)) was approved for Microsoft nonprofit grants and discounts. - June 23, 2026: Microsoft Elevate support informed me the original application was in an unrecoverable system error state due to an unrecoverable error in their system, and they instructed me to resubmit under a different email address. I only had one email directed to my business email and asked if I could use a Yahoo email. - June 30, 2026: The same Elevate representative confirmed in email to me that using a Yahoo email address as a temporary workaround was fully supported and "the correct and supported path forward." - July 2026: A different Microsoft Validation Team (not the one I had worked with originally) rejected the resubmission (likely due to use of a yahoo.com email as my business email address) and demanded documents not listed in the published eligibility requirements, including a bank statement. They stated the original Goodstack approval cannot be reinstated. - When I pointed out that Goodstack's validation was never rescinded, that my organization's nonprofit status can be confirmed on propublica online, and that the resubmission existed solely due to Microsoft's own system error and instructions, I was told this was a determination of another team and no further escalation paths exist. The core issue: Goodstack validated our organization on Microsoft's behalf. Microsoft's own system error forced resubmission. Microsoft's own representative authorized the (incorrect) workaround in writing. The Validation Team is now treating the workaround submission as a brand new independent submission requiring extraordinary measures for re-validation, ignoring the Goodstack validation, ignoring the request to check propublica, and ignoring the fact that thier reuest for additional documents is nowhere to be found in Microsoft's own document requirements in Microsoft's published nonprofit eligibility policies. Ticket number: 2606210040000788 Is anyone at Microsoft able to escalate this to someone with authority over the Validation Team? Has anyone else encountered this after being directed to use a personal email for resubmission?
Launching DIY Data Control for Nonprofits
I want to create a library of open-source resources that helps self-taught IT champions in nonprofit organisations build fully governed, secure shared data systems. I have plenty of ideas about how to do this, but before I press ahead, I want to listen to the people I hope to support to ground those ideas in reality. For my first project, I want to draw on part of the Microsoft ecosystem, which provides some useful tools for safely building your own data systems: Microsoft Learn: Microsoft provides a wealth of free learning resources. I use them a lot and they are as good as other resources that I have happily paid for. The two big barriers to using them are finding time and working out where to start. The Common Data Model for Nonprofits: This is an open-source resource from Microsoft, it describes the tables of data that a nonprofit organisation needs, and the relationships between them. Once you add your own data it becomes a shared functioning model of your organisation. Role based column and row level security: This is the revolutionary part that lets you safely replace scattered spreadsheets with a single, comprehensive, shared model of your organisation. It lets you decide exactly which parts of which records people are allowed to see or change, so everyone who interacts with your shared model can only see and change the things that they need to. Model-driven Power Apps and Power BI: Once you have a detailed model of your organisation with governance baked into it, adding tools to interact with it becomes straightforward. Both Model-driven Power Apps and Power BI can understand the structure of your organisation from your model, and they will respect the governance policies that you built into it. In theory, these should provide all the tools you need to bring together scattered data in siloed spreadsheets into a single, secure, auditable shared source of truth. But there are gaps, and I want to find out what guides, signposts and tools would be needed to bridge them. Later I will want to explore the features that you could add with Copilot Studio if you wanted to. Once you have established a clearly defined, precisely secured data environment, Copilot Studio can provide the tools to develop and deploy AI agents safely. But for now, I will focus on the foundations of well-defined, well-governed, accessible data. If you have any thoughts about this project then I would love to hear them, and I’d especially like to find out what pain-points there are that I can target. So please do get in touch. Patrick Killeen Head and Heart CIC email address removed for privacy reasons https://www.headandheart.info/ This work is released under the MIT Licence and is available at https://github.com/head-and-heart-cic/public/blob/main/in-practice/2600707-launching-diy-data-control-for-nonprofits/README.md
Microsoft Prompt Gallery for Gov Clouds
Has anyone found a way to access Microsoft's prompt gallery with a gov cloud? Any web access link requires you to log in and brings you to the commercial endpoint. I don't see a way to access it from inside Office using the Copilot panel. We are GCCH. Also has anyone built an internal prompt gallery for your business to use and share prompts? Did you create this in a SharePoint list? Would love to hear other's ideas and even some examples of what is working great for them as far as what you are capturing and what is your repository.Hands-on webinar: Teach Module in M365 Copilot and Copilot Notebooks (available for all educators)
Join us on Wednesday, July 8th @ 8am Pacific Time for an in-depth professional development webinar on the new AI-powered "Teach" module in M365 Copilot that is fully rolled out and available to everyone. We will also be covering Copilot Notebooks and the Study Guide which is now also available to all M365 Educators and Students. This will be a 60-minute hands-on webinar where the Product Management team will walk through the new updates in detail and you can follow-along at home with your own M365 Education account. To reiterate, the Teach Module and Copilot Notebooks are available globally to all educators using Microsoft 365. And don’t worry – we’ll be recording these and posting on our Microsoft Education YouTube channel so you’ll always to able to watch later or share with others. Register here: https://msit.events.teams.microsoft.com/event/msit.ed7f065d-57a5-437c-ba23-12b0ef06a259@72f988bf-86f1-41af-91ab-2d7cd011db47 Agenda: How to use the new AI-powered "Teach" module in M365. ✅ Lesson plans and the new Keep Going feature ✅ Learning Zone integration ✅ Modifying Existing content ✅ Learning Activities ✅ Coplot Notebooks plus Study Guide We look forward to having you attend the event! Also, be sure to check out and sign up for our Microsoft Elevate Educator community. Mike Tholfsen Group Product Manager Microsoft Education team
Access issue with Copilot Studio in Microsoft Learn Applied Skills lab
Hello Microsoft Community, I am currently working on the Microsoft Learn Applied Skills exercise: “Create your customer service agent” using Copilot Studio. When attempting to sign in, I am using my personal Gmail account. However, I receive an error stating that personal Microsoft accounts are not supported, and I am unable to proceed with the lab. Is a Microsoft Entra ID work/school account required to access Copilot Studio for this exercise? Thanks in advance.Getting Started with AI and MS Copilot - English
🚀 Ready to explore AI and Microsoft Copilot in a fun, hands-on way? Join our session: “Introduction to AI and Microsoft Copilot”—designed for educators who are just getting started! ✅ Learn the fundamentals of generative AI ✅ Master the art of creating effective prompts ✅ Discover practical ways to use these tools in your classroom ✅ Access ready-to-use teaching resources ✅ Practice with 10 interactive exercises 📅 Don’t miss this opportunity to boost your teaching with AI! #MicrosoftCopilot #Educators #Innovation #TeachingTools Getting Started with AI and MS Copilot - English | Meeting-Join | Microsoft Teams📣 Getting Started with AI and MS Copilot — Português
Olá, 👋 📢 Quer explorar IA e Microsoft Copilot de forma prática para o aprendizado? Participe da sessão “Introdução à IA com o uso do MS Copilot”, pensada especialmente para docentes que estão começando a usar o Copilot. Vamos aprender os fundamentos da IA generativa, como criar boas instruções e aplicar essas ferramentas na sala de aula. 📌 Sessão com exemplos práticos, materiais para utilizar e um espaço ideal para praticar e tirar dúvidas. No horário indicado, favor realizar acesso ao link.GitHub Copilot App Canvas Is a Runtime
There is a quiet shift happening in how we build software with AI. We are moving from writing static code to orchestrating living systems where developers and AI agents co-create, observe, and evolve a solution in real time. This post is a working theory of what GitHub Copilot App Canvas is actually for, grounded in a real, runnable demo you can clone today: leestott/agent-runtime-canvas. The Agent Runtime canvas open beside the chat — control bar, activity spotlight, requirement & constraints, and the live agent roster. The headline claim, which the rest of this post defends with code: Traditional UIs are for using software. Canvas is for shaping software while it runs. 1. The misconception worth getting out of the way The first instinct most engineers have when they see Canvas is to build a UI with it a dashboard, a DevOps board, an admin panel. That is the wrong mental model, and it leads to disappointment. A Kanban board rendered in Canvas is just a worse version of a tool that already exists. Canvas is not where your users live. It is where your system becomes visible to you and to the AI while you are still figuring it out. The distinction matters: You don't build Canvas instead of your UI. You use Canvas to figure out, test, and evolve the UI and the system before and during building it. Canvas solves problems your final UI should never try to solve in a visible way agent coordination, intermediate state, test validation, failure propagation. These are observability concerns, not end-user features. Canvas is intended for test validation and the implementation of agent-driven solutions not for shipping a production control panel. A useful analogy: Figma is Human-to-Human one person designs a static artifact for another person to read. Canvas is Human-to-AI-to-System a shared surface where a human, an AI agent, and a running system all act on the same live model. Figma shows you a picture of the software. Canvas is a runtime where things actually execute. 2. The positioning, stated plainly Here is the thesis the demo is built to prove: Canvas redefines software development by shifting from writing static code to orchestrating living systems, where developers and AI co-create, observe, and evolve solutions in real time. Instead of building UIs for users, we build interactive environments for agents — turning debugging, testing, and execution into a continuous, visual feedback loop that accelerates innovation and brings ideas to production faster than ever. Read that again with the demo in mind, because the demo is not a slide, it is a working Copilot CLI extension that renders exactly this loop. 3. What we built: the Agent Runtime canvas agent-runtime-canvas is a GitHub Copilot CLI canvas extension called Agent Runtime. It turns Canvas into a runtime observability and control plane for a multi-agent software system that is being designed, tested, and evolved in real time. The canvas renders a single living SystemModel that both humans and the AI agent edit at the same time. The agent drives it through five canvas actions; the human drives it through panel controls. Every change streams to the iframe over Server-Sent Events (SSE), so the system visibly evolves through interaction. The seven panels: a system you can watch think Panel What it makes observable Requirement & constraints The feature under design plus editable policies and constraints Agents Active agents, their responsibilities, and live state (idle / working / done / error / blocked) Task Flow The dependency graph of tasks across agents, with live status Artifacts The intermediate outputs each task emits Validation Test cases, pass/fail, expected vs. actual, and the reasoning behind each verdict Live State The shared memory objects the agents read and write — directly human-editable Timeline A change-over-time log, including before→after state diffs None of these are things you would put in front of an end user. All of them are things you desperately want to see while you and an AI are co-designing an agentic system. The five agent actions The AI co-creates and evolves the system by calling five actions, declared in the canvas extension: Action Effect decompose_system Break a requirement into collaborating agents + a task-flow graph execute_workflow Coordinate agents to advance tasks ( step / run / pause / resume / reset ) validate_output Run evaluation tests, return structured pass/fail + reasoning update_system_design Modify architecture/logic: requirement, constraints, agents, tasks track_state Persist/update a shared state object, recording the diff on the timeline The critical detail is that human controls and agent actions funnel through the exact same store. There is no separate "AI view" and "human view" — one model, two kinds of participant. 4. How it actually works (the parts that matter) The extension is deliberately small and dependency-free. It uses only Node's built-in modules plus github/copilot-sdk , which the CLI auto-resolves. Three files do the work: .github/extensions/agent-runtime/ extension.mjs # wiring: loopback HTTP server, SSE, /control, 5 canvas actions store.mjs # durable SystemModel + execution engine + validation ui.mjs # iframe renderer (system view, validation, state, timeline) One shared model, broadcast on every mutation The heart of the demo is the SystemStore . It is an EventEmitter : every mutation bumps a version, appends a timeline entry, persists to disk, and broadcasts a fresh snapshot to all connected panels. This is the single line that makes "humans and AI edit the same live system" true rather than aspirational: // store.mjs — every change is versioned, logged, persisted, and broadcast. _commit(eventType, summary, detail) { this.model.version += 1; this.model.updatedAt = now(); if (eventType) { this.model.timeline.unshift({ id: uid("ev"), ts: now(), type: eventType, summary, detail: detail || null, }); this.model.timeline = this.model.timeline.slice(0, 200); } this._queueSave(); // best-effort JSON persistence under ~/.copilot this.emit("change", this.model); // fan out to every SSE client return this.model; } The agent action and the human button hit the same method In extension.mjs , the canvas action handler and the iframe's /control POST both call store.execute(...) . That symmetry is the whole point — neither the human nor the AI is privileged: // extension.mjs — a human control POST maps onto the same store method // the AI agent calls through the execute_workflow canvas action. function applyControl(store, body) { switch (body.action) { case "execute": return store.execute(body.mode || "step", body); case "validate": return store.validate(body.tests); case "decompose":return store.decompose(body.requirement, body); case "inject_failure": return store.injectFailure(body.taskKey); case "edit_state": return store.editState(body.key, body.value); // ...requirement, constraints, clear_failures, update_design } } Execution you can watch one task at a time The engine advances the task graph through a visible begin→dwell→finish lifecycle so the active agent is always observable. A ready task is one whose dependencies are all done : // store.mjs — the scheduler only starts a task when its deps are satisfied. _readyTask() { return this.model.tasks.find( (t) => t.status === "pending" && t.deps.every((d) => { const dep = this.model.tasks.find((x) => x.id === d); return dep && dep.status === "done"; }), ); } When a task finishes, its agent emits an artifact and writes to shared state; when a dependency fails, the engine walks the graph to a fixpoint and marks every downstream task blocked . That is failure propagation you can see — exactly the kind of thing a production UI would (correctly) hide, and exactly the kind of thing you need exposed while designing the system. Validation as a first-class, re-runnable citizen The default evaluation suite asserts properties of the running system, not of static code — every test returns an expected value, an actual value, and a human -readable reason: // store.mjs — tests assert properties of the live system model. _defaultTests() { const t = (name, target, assertion) => ({ id: uid("test"), name, target, assertion }); return [ t("All tasks reach a terminal state", "tasks", "no_pending"), t("No tasks failed", "tasks", "none_failed"), t("Every completed task emitted an artifact", "artifacts", "artifact_per_done"), t("Design state populated before build", "state", "design_before_build"), t("Decision recorded by Reviewer", "state", "has_decision"), ]; } This is the "continuous, visual feedback loop" from the thesis, made concrete: decompose → execute → validate → redesign → re-validate, with the Timeline recording every before→after transition. 5. Run it yourself You need a GitHub Copilot CLI / app with canvas support (the canvas-renderer capability) and this repo opened as your workspace. There is no npm install the SDK is auto-resolved and the extension uses only built-in Node modules. Clone and open the workspace. git clone https://github.com/leestott/agent-runtime-canvas.git cd agent-runtime-canvas The extension auto-discovers from .github/extensions/agent-runtime/ . Open the canvas with a requirement. Ask Copilot: Open the Agent Runtime canvas with the requirement "Add CSV export to the reports page". Walk the loop. Decompose into five agents and a six-task graph, press Run ▶, watch the spotlight track the active agent, press Run tests ✓ for 5/5 green, then Inject failure ⚡ to watch downstream tasks go blocked and validation drop to 4/5 — and recover. State persists per documentId under ~/.copilot/extensions/agent-runtime/artifacts/ , so a reload resumes exactly where you left off. The companion demoscript.md in the repo gives you a tight, timed walkthrough. 6. Why this is an observability story Once you accept that Canvas is a runtime rather than a UI, the most compelling use case becomes observability of agentic systems. Agentic software is notoriously hard to debug: the interesting behavior lives in intermediate state, coordination order, and the moments where one agent's failure cascades into another's. A production UI is designed to hide all of that. A Canvas is designed to surface it, temporarily, while you are shaping the system — and then get out of the way. This reframes Canvas alongside the broader Microsoft and GitHub agent tooling story. As teams adopt the GitHub Copilot SDK and patterns like the open Model Context Protocol to wire agents into real systems, the gap is rarely "can the agent act?" it is "can a human see what the agent did, judge it, and steer it?" Canvas is a candidate answer to that second question. When you take agents toward production on Azure with services like Microsoft Foundry, the same instinct applies: build the evaluation and observability loop first, and let it shape the system before you commit a single end-user pixel. 7. The open question: why can't Canvas be multi-user? There is an obvious next frontier, and it is worth stating as an honest open question rather than a finished feature. Everything that makes Canvas valuable also makes it a natural collaborative surface: It is a shared space. It is visual. It is collaborative. Multiple participants — human and AI — interact with the same surface. If Figma earned its place by making Human-to-Human design multiplayer, the provocative question is whether a project- or repo-scoped Canvas can make Human-to-AI-to-System development multiplayer too: several engineers and several agents shaping one running system on one surface. The demo here is single-user by design, but its architecture — one shared store, versioned, broadcast to every subscriber — is already the shape you would need. That is a genuine research direction, and worth experimenting with as licensing and access broaden. 8. Honest limitations In the spirit of building credibility rather than hype: This is a demonstration. The decomposition, artifacts, and state are synthesized to make the runtime loop legible — it models an agentic system rather than running arbitrary production agents. It is single-user and single-machine. The loopback HTTP server and per-document store are local by design; multi-user is an aspiration, not a shipped capability. Access is gated. Canvas support requires a Copilot CLI/app build with the canvas-renderer capability. Licensing and preview access are the biggest practical blockers to wider experimentation today. Persistence is best-effort. State is written to a local JSON artifact; treat it as demo durability, not a database. Key takeaways Don't build a UI in Canvas. Use Canvas to shape, test, and evolve a system — and the UI — while it runs. Traditional UIs are for using software; Canvas is for shaping software while it runs. Canvas is Human-to-AI-to-System, a runtime where things execute — not a static design surface. Its strongest use case is observability and validation of agentic systems: surface the intermediate state your production UI should hide. The shared-model architecture — one versioned store broadcast to every participant — is what makes human + AI co-editing real, and what hints at a multi-user future. Next steps Clone and run the demo: github.com/leestott/agent-runtime-canvas. Read the extension source under .github/extensions/agent-runtime/ — start with store.mjs . Explore the building blocks: the GitHub Copilot SDK, the Model Context Protocol, and Microsoft Foundry for taking agentic systems toward production. Try the multi-user thought experiment: fork the store, add a second subscriber, and ask what changes when two humans and two agents share one surface.
In a Day (XIAD) Partner Events Program - Train the Trainer Events (AgIAD, AIAD, AuIAD, PPIAD)
We invite you to attend an upcoming Train the Trainer session for Microsoft Partners to learn more about the In a Day (XIAD) Partner Events Program and how to lead workshops that empower customers to use and adopt Microsoft products. Our Train the Trainer events are designed to provide you with the knowledge and tools necessary to deliver successful Microsoft In a Day (XIAD) sessions. *Please note, participation is restricted to individuals representing a Microsoft Partner organization. You must register using your corporate email address that is associated with your Partner ID. Personal emails will not be approved. ✨ Why Attend? Expert Guidance: Learn from experienced trainers and get your questions answered. Comprehensive Resources: Access all the content and support you need to succeed. 📅 Upcoming Events: Agent in a Day Discover the basics of building agents with Microsoft Copilot Studio, including generative AI orchestration and integrating external data sources. Full day session (8 hours): Thursday, June 25 | 09:00–17:00 GMT+2 (Central European Summer Time) Trainer ready session (1.5 hours): Tuesday, June 30 | 09:00–10:30 GMT-5 (Central Daylight Time) Recordings available on-demand. Inquire at xiadevents@microsoft.com App in a Day Explore the fundamentals of building apps with Microsoft Power Apps, including creating custom business applications without writing code. Full day session (8 hours): Tuesday, June 23 | 09:00–17:00 GMT+2 (Central European Summer Time) Trainer ready session (1.5 hours): Tuesday, June 30 | 11:00–12:30 GMT-5 (Central Daylight Time) Recordings available on-demand. Inquire at xiadevents@microsoft.com Automation in a Day Explore automation solutions with Power Automate, including creating workflows and automating business processes. Trainer ready session (1.5 hours): Tuesday, June 30 | 14:00–15:30 GMT-5 (Central Daylight Time) Recordings available on-demand. Inquire at xiadevents@microsoft.com Power Pages in a Day Get hands-on building simple, secure business websites with Microsoft Power Pages. More sessions coming soon! Recordings available on-demand. Inquire at xiadevents@microsoft.com 👉 Register for all upcoming sessions at https://aka.ms/xiadTTT/pp Partner with us! Are you a Microsoft Partner interested in the opportunity to join the program and deliver In a Day (XIAD) events? 🔍 Learn more about the program and review partner eligibility criteria: https://aka.ms/xiadpartneropportunity. 📧 Contact the XIAD Program team: xiadevents@microsoft.com 🚀 Get started today: https://aka.ms/XIADGetStarted 📤 Submit requests to deliver events: https://aka.ms/xIAD/PartnerEvents
Mind the Specs: Grading formal specifications and KPIs as artefacts for LLM-driven code generation
Large language models now write code straight from a prompt, but the specification in between is never checked, and a model asked to judge its own work brings the same blind spots to the review. We built a pipeline that lifts a plain-language requirements bundle into two graded specifications (a formal Alloy model and a set of numerical KPI targets), scores both before a single line of code is written, and hands the graded result to the code generator. It starts from GitHub Spec Kit and the Azure Well-Architected Framework. Here is what we built, and what we learned from running it at scale. The problem Writing software used to be four separate activities: gathering requirements, writing a specification, verifying it, and implementing it. A language model collapses all four into a single step. Two of those activities used to give us a quality signal before any code existed: a formal specification you could inspect, and measurable targets an implementation had to hit. The prompt-to-code loop inherits neither. There is no externally observable signal, before a line of code is written, that the requirements a model received are even well-formed enough to drive a correct implementation. You might think the model could just check its own work. It cannot do so reliably. Ask a language model to check the logic it just wrote: not only will it bring the same blind spot to the review, but its stochastic nature will make it produce different answers on each run. A SAT solver does not behave this way. Its verdict is deterministic: the same specification produces the same verdict every time. The thing that historically kept formal specification out of everyday development was never its rigour, it was the cost of writing the specification by hand. And that is exactly the step a language model can now do. What we built We built an agentic pipeline that sits between the requirements and the generated code. In plain terms it takes the requirements once, turns them into two things that can be checked by a machine: a precise description of rules that the system must obey, and a set of measurable targets that the system must hit. These artefacts are both graded, and are handed to the code generator. We split the work in two and gave each half to the tool that is good at it. The language model does the creative part, turning messy prose into formal structure. Deterministic checks, not the model's own opinion, grade what it produces. From a single Spec Kit artefacts bundle the pipeline builds two graded specifications before any code exists, and then carries both into code generation. Since these grades are computed deterministically rather than just generated, you can actually trust them. The input is a GitHub Spec Kit bundle. Spec Kit is an open-source, specification-first toolkit: instead of prompting for code directly, you describe what you want to build, and it produces a set of structured artefacts, a feature specification, a data model, and a set of API contracts. Our pipeline reads that bundle and turns it into the two graded specifications in parallel. overview. Spec Kit artefacts on the left. The Alloy lifter (with SAT solver and the attack step) and the KPI agent run in parallel. Their graded outputs are merged into a verification report that feeds the guided code generator. A dashed baseline path feeds the goal alone to the generator for comparison. Lift the requirements into a formal model The first half is structural. An Alloy lifter translates the requirements into a formal model written in Alloy, a specification language whose rules a SAT solver can check exhaustively, and whose verdict is deterministic, so the grade never depends on asking an LLM what it thinks. A banking requirement like "zero balance discrepancies" becomes a precise, checkable rule: the money leaving one account and the money arriving in another must always add up to the balances you started with, so a transfer can never quietly create or destroy money. The solver searches for any scenario that would break the rule. We modified Spec Kit's templates to force the model to output functional requirements and their corresponding Alloy code blocks in a structured format. Against the stock templates, that change alone nearly doubled the Alloy code compilation rate, jumping from 40 to 74 percent. A machine-written specification cannot be trusted, though, so the lifter does more than write it: it attacks it. Each load-bearing rule is deliberately broken by clearing its body and injecting a clause that forces a violation and the solver is re-run on the broken model. If the solver fails after this mutation, the original rule genuinely caught the violation it was meant to catch. If it still passes, the rule never really constrained anything on its own. Mutation testing usually grades a test suite against a specification that is assumed correct; here the roles are reversed, and the specification itself is on trial. Turn the requirements into measurable targets The second half is measurable. A KPI agent takes the same Spec Kit bundle, retrieves the most relevant principles from the Azure Well-Architected Framework, and derives numerical targets in the Goal-Question-Metric style. Each target carries an explicit threshold, a direction, and a measurement method, the kind of target a monitoring tool could actually track. Where earlier automated approaches stopped at describing quality in words, this half emits the actual numbers an implementation has to satisfy. And the knowledge base is a setting, not a fixture: swapping the Well-Architected Framework for ISO 25010, the NIST Cybersecurity Framework, or Google's SRE workbook requires zero changes to the underlying code. Review the report before any code Both graded halves merge into one human-readable verification report: the patterns the model applied, which rules passed, the counterexamples the solver found, the attack results, and the KPI threshold table. A developer reads it first and can see exactly where the specification is weak: a rule that passed for the wrong reason, or a requirement that nothing covers. After revising the specification, they re-run the lifting phase. Because the process is cached, re-runs are cheap, allowing the developer to loop until the report looks perfect, all before any code exists. The work shifts from reviewing generated code after the fact to curating a specification and reading a report before anything is built. Carry the graded context into code generation Only then does the report do its real job. In the guided pipeline, the merged report becomes the context handed to a code generator, which is asked to implement each rule, requirement, and KPI threshold and to leave markers tracing the code back to them. A baseline generator gets only the plain-language goal. Same generator, same settings; the only difference is whether it can see the graded specification. Feeding graded artefacts, rather than raw prose, into code generation is the piece that ties the whole pipeline together. So three choices separate this from simply asking a model for a spec: the specification is attacked rather than trusted, the targets are numbers rather than prose, and what reaches the code generator is graded evidence rather than raw text. How we tested it We ran the pipeline at scale: 270 Alloy lifts and 1,930 KPI records, across three application domains chosen to differ sharply (banking, software-as-a-service, and healthcare), three levels of requirement detail, four knowledge bases, and three model tiers, with ten runs of each combination so a real effect could be told apart from noise. For the code-generation half, we generated two codes for each case, once with the graded report as context and once from the plain-language goal alone, and compared the two. What we found First, the foundation: the specifications proved gradeable. The rubric cleanly separated sound specifications from degenerate ones. Because it returned the same verdict run after run, the grades are reliable enough to act on. The three key observations are as follows: The model matters more than the prompt Of the two knobs a practitioner controls, the model you choose and the amount of detail you write, the model dominated by roughly nine to one. A weak model could not be rescued by richer requirements. But you do not need the most expensive one: a mid-tier model delivered about 98 percent of the best model's quality at under a third of the cost and about half the time. The cheapest tier was a false economy, producing a model the analyser could even load only 23 percent of the time. More detail can backfire More requirements are not always better. Sparse and standard requirements scored the same, but over-specified requirements collapsed: KPI quality fell from about 0.89 to about 0.73, and the effect held across all four knowledge bases. Pile in too much numerical detail and the pipeline starts echoing the numbers it was handed instead of deriving sound ones, which is the opposite of what more detail is supposed to buy. Graded context produces far better code This is the payoff, and it is the point of the whole pipeline. Across all nine combinations of domain and detail, code generated with the graded verification context scored about 8 out of 10, against about 1 out of 10 for the same generator given only the plain-language goal. The guided code carried the traceability back to each requirement, the named rules, and the structural patterns that a bare prompt gives us no way to know about. This part of the study is a single run per combination, so we report the size and the consistency of the gap rather than a precise average, but the gap was large and it held in every case. What this means for you Four things to take from our study into your own work: Write requirements at a standard, middle level of detail. Not sparse, and not exhaustively numerical. The middle is the sweet spot on both halves of the specification. Reach for a capable mid-tier model before you invest in heavy prompt engineering. Model choice moves quality more than requirement detail does, and the mid tier is the value leader. Give the code generator externally graded context instead of letting it specify for itself. That is where most of the quality gain came from. Treat the knowledge base as a setting worth tuning, not a fixed ingredient. Each is a recommendation that data supports under the conditions we tested, not a universal law. The limit Every grade measures structure, not meaning. A high score says the specification is well-formed, discriminating, and stable. It does not say whether the invariants are the right ones, or the thresholds are the right ones for your deployment. A specification can be perfectly well-formed and still describe the wrong system. That judgement stays with a human, which is where we think it belongs. The pipeline is built to make that judgement efficient by moving it earlier, to curating the specification and reading the report, rather than to remove it. Generated code should not be shipped end to end without human validation. Try it The full pipeline, every input, and the artefacts behind every figure are in the project repository. If you want the Microsoft tools it builds on, start here: Project repository: https://github.com/RadaanMadhan/Specification-Led-Development GitHub Spec Kit: https://github.com/github/spec-kit Azure Well-Architected Framework: https://learn.microsoft.com/en-us/azure/well-architected/ If you'd like to explore the work in more detail, we've included the full technical report in the project repository, covering the related work, methodology, pipeline design, experimental setup, and extended results. About the team This project was carried out by six students at Imperial College London: Leon Hausmann, Charlotte Maxwell, Radaan Madhan, Keshav Das, Anson Huang, and Ander Cobo, in collaboration with Microsoft and supervised by Lee Stott (Microsoft) and Max Cattafi (Imperial College London)
