rss.livelink.threads-in-node

What's new in Assignments: interactive lessons, localized content, and smarter AI tools

Leif Brenne — Wed, 17 Jun 2026 14:59:25 GMT

Assignments in Microsoft Teams and through the Microsoft 365 LTI app has shipped a set of updates that connect interactive learning content directly into the assignment workflow, improve content quality across English-speaking regions, and give educators more control over AI-generated output.

All of these features are available now for every Microsoft Education license (A1, A3, A5) at no extra cost.

Watch a quick walkthrough of all the updates: What's new in Assignments

Learning Zone interactive lessons in Assignments

Educators using a Copilot+ PC can turn any teaching idea into an interactive lesson — with slides, quizzes, and activities — using the Learning Zone app. Those lessons now connect directly into the assignment workflow.

When creating an assignment, educators see a new Learning Zone resource option. Pick a lesson, and it's attached to the assignment. Students complete the interactive lesson right inside their assignment in Teams or through the LMS, without leaving. When they finish, scores sync automatically — the student's completion percentage becomes their assignment grade.

A few things to know:

You need a Copilot+ PC to create lessons. Students can complete them on any device.
If you don't have any lessons yet, the picker will link you to download the Learning Zone app.
Students can also open the lesson in the Learning Zone app on Windows if they prefer.

Learn more: Add a Learning Zone lesson to an assignment

Student feedback view update

We've updated how students see feedback on returned assignments. The view now reorganizes to put what matters most front and center:

Feedback, feedback resources, rubric, and grades take the main space.
Instructions and resources collapse into side sections — still accessible, just not competing for attention.

Before this change, students saw everything at once — feedback mixed in with instructions, attachments, and rubrics. Many students struggled to find the feedback you spent time writing. Now it's the first thing they see when they open a returned assignment.

This is already active for everyone. Nothing to turn on.

Standards alignment across Instructions and Rubrics

Educators can now add educational standards directly to their assignment. When you use AI to enhance your instructions, it takes those standards into account. And when you generate a Rubric for the same assignment, it pulls in those same standards as input — so your instructions and rubric stay aligned without extra work.

We've also made a few other improvements to AI Instructions:

Describe changes to tweak the output. After the AI generates a suggestion, you can describe what you want changed in a text field and re-generate. Want it shorter? More scaffolded? Focused on a specific concept? Type what you'd like changed and get an updated version — no need to start over.
Shorter, more focused suggestions. When you use Add Detail, Add Sparkle, Add Hints, or other enhancements, the output stays closer to what you actually need rather than over-expanding your instructions.

Learn more: Standards in Assignments and the Teach module

English locale support across Assignments and the Teach module in Microsoft 365 Copilot

AI features across both Assignments and the Teach module in Microsoft 365 Copilot now support English (UK), English (Canada), and English (Australia) in addition to English (US).

Language is automatically selected based on your browser language settings. If your browser is set to en-GB, the AI generates content using British English. If you've used Teach before, it defaults to your previously used language. You can switch anytime from the Language dropdown when creating content.

We've also updated the age/year selection to match what's appropriate for each locale — so you'll see "Year 9" instead of "Grade 9" when using English (UK), for example.

Resources

Detecting Python Vulnerabilities with GraphCodeBERT

a_elhaag — Tue, 16 Jun 2026 07:00:00 GMT

We are nine software engineering students at the Egyptian Chinese University in Cairo. When we got our project brief, we noticed a gap that bothered us: Python is the most widely used language in AI development, yet almost every security tool out there was built for C and C++. The tools that do exist for Python rely on regex pattern matching — a technique that has not changed meaningfully in years.

So we built one ourselves.

We called it Code Security Identifier — CSI. Instead of matching patterns like existing tools, CSI understands code structure. We split the work across nine people, each owning a specific piece of the system: dataset engineering, model architecture, loss function design, adversarial training, hyperparameter optimization, and deployment. None of us had built a production security tool before. By the end, we had one running.

This post documents what we built, the decisions we made, the things that did not work, and what we learned.

GitHub Repo

The Problem: Python Security Is Underserved

Python powers 70% of AI workloads and 45% of enterprise backends. As AI-assisted code generation becomes standard practice, the volume of Python code being written and deployed is growing faster than any team can manually audit. GitHub Copilot, ChatGPT, and similar tools generate thousands of lines of Python daily. Much of it is never reviewed for security.

The tools that exist were not built for this reality. Bandit, the industry standard for Python static analysis, uses regex pattern matching. Its F1 score on vulnerability detection is approximately 0.62. That means for every 100 real vulnerabilities in a Python codebase, Bandit catches 62 and misses 38. In a production system handling user data, financial transactions, or infrastructure commands, those 38 missed vulnerabilities are exploitable.

The deeper problem is architectural. Regex-based tools flag code that looks suspicious based on token patterns. They cannot trace how data flows through a program. They cannot reason about whether an untrusted input reaches a dangerous execution point. They catch obvious cases and miss subtle ones — which are exactly the cases that matter most in real-world exploits.

We set out to build something better.

Why Token-Only Models Also Fail

The first generation of deep learning approaches to vulnerability detection treated source code the same way NLP models treat text: as a sequence of tokens. Models like CodeBERT learn statistical co-occurrences. They learn that SELECT, WHERE, and execute appear together. They learn that os.system and subprocess appear near command-like strings. These patterns are suspicious. But suspicion is not detection.

What "Token Co-Occurrence" Actually Means

To be concrete: a token model doesn't read code, it reads a flat sequence of sub-word units, the same way it would read a sentence. It has no built-in notion of "this token is a function argument," "this token is the return value of that call," or "this variable was assigned three lines up and is now being used here." Everything is positional and statistical. During pretraining, the model learns that certain tokens tend to appear near other tokens — execute tends to follow strings that look like SQL, eval tends to appear near user-controlled-looking variable names, os.system tends to sit close to subprocess or shell=True. These are real correlations in code, and they give the model some signal. But a correlation between tokens is not the same as understanding what the code actually does with those tokens.

The Causal Chain a Vulnerability Actually Is

A real SQL injection vulnerability is not a collection of SQL-adjacent tokens. It is a causal chain: an untrusted value enters through a function parameter, passes through one or more assignments and string operations, and reaches a database execution call without sanitization. Concretely, that chain might look like: user_id arrives as a request parameter → it gets assigned to a local variable → that variable is interpolated into an f-string → the f-string is passed as the queryargument to cursor.execute(). Each of these steps, on its own, is completely unremarkable Python. Assigning a variable is not dangerous. Building an f-string is not dangerous. Calling execute() is not dangerous. The vulnerability exists only in the connection between these steps — specifically, in the fact that an untrusted value reaches a sensitive sink without anything sanitizing it along the way.

Token models cannot see this chain. They see the tokens at each step — user_id, f"...", cursor.execute — and they may even have learned that this combination of tokens is statistically associated with vulnerable code. But "statistically associated with" is not the same as "I can trace that this specific value, from this specific source, reaches this specific sink." The model has no mechanism for following a variable across lines, across function boundaries, or through transformations. It is reasoning about which words appear, not what happens to the data.

Two Failure Modes, Same Root Cause

This single limitation — no data-flow reasoning — produces two distinct failure modes, and both are costly in a real security context.

The first is false positives on safe code. Plenty of legitimate, secure code uses tokens that a token model has learned to associate with danger. A function that builds a SQL query using parameterized queries (the correct, safe way to do it) still contains tokens like query, execute, and variable names that look like user input — because they often are user input, just handled safely via placeholders and bound parameters instead of string concatenation. A token model, lacking the ability to distinguish "this value is interpolated directly into the query string" from "this value is passed as a separate, escaped parameter," may flag both patterns identically. In practice, this is exactly the kind of false positive that erodes trust in a security tool — if a scanner flags safe, well-written code as vulnerable often enough, developers start ignoring its output entirely, which defeats the purpose of having it.

The second, more dangerous failure mode is false negatives on real injections that don't match the training distribution. The token-level patterns a model learns during pretraining are necessarily a reflection of the kinds of vulnerable code that were common in its training data — typical variable names like user_input, query, cmd, typical function calls like os.system, eval, cursor.execute. But real-world code, especially AI-generated code, doesn't always follow these conventions. A variable might be named x, payload, data_5, or something entirely project-specific. A dangerous sink might be wrapped in a thin custom helper function with an unfamiliar name that itself calls subprocess.run three layers down. If the surface tokens don't match what the model has seen before, but the underlying data-flow path — untrusted input to dangerous sink — is identical to a thousand vulnerabilities the model has seen, a token model has no way to recognize that. It missed the pattern not because the vulnerability is novel, but because it was only ever looking at the wrong thing: the words, not the wiring between them.

Why This Matters for CSI's Design

Both failure modes point to the same underlying gap: vulnerability detection is fundamentally a question about the paths data takes through a program, not about which words appear in the program's source. A model that wants to close this gap needs access to something a flat token sequence cannot provide — an explicit representation of how data flows from one point in the code to another, independent of what the variables along that path happen to be named. This is exactly the gap GraphCodeBERT's data-flow graph is designed to fill, and it's the reason we built CSI around it rather than around a purely token-based model like CodeBERT.

Our Foundation: GraphCodeBERT

Microsoft's GraphCodeBERT addresses the structural blindness of token models by parsing source code into three complementary graph representations and attending over all three simultaneously during pretraining.

Abstract Syntax Tree (AST) The AST captures the syntactic structure of code: how functions are defined, how expressions compose, how variables are declared relative to their scope. It gives the model a hierarchical view of code that token sequences cannot provide.

Data Flow Graph (DFG) The DFG is the critical representation for vulnerability detection. It traces how values propagate through a program: which variables receive which values, how those values are transformed, and where they ultimately flow. For an injection vulnerability, the DFG makes the taint path explicit: user_id → query → db.execute(). This is the path that token models cannot see.

Control Flow Graph (CFG) The CFG maps which code paths execute under which conditions. It captures branch logic, loops, and exception handlers — the execution context that determines whether a tainted value can actually reach a dangerous sink in practice.

Together, these three representations give GraphCodeBERT a structural understanding of code that enables meaningful vulnerability detection. For a SQL injection, it traces the full semantic chain from untrusted input through concatenation to database execution. Token models see three words at that point. GraphCodeBERT sees a taint flow.

Making It Trainable: LoRA Parameter Efficiency

GraphCodeBERT has 125 million parameters. Full fine-tuning on a domain-specific dataset at this scale requires significant GPU memory, long training times, and a dataset large enough to update 125M parameters meaningfully without overfitting. We had approximately 4,000 training samples and access to Google Colab.

We applied LoRA (Low-Rank Adaptation), a parameter-efficient fine-tuning technique that injects small trainable adapter matrices into the query, key, and value projection layers of each transformer attention block while keeping all backbone weights frozen. The adapter for a weight matrix W is parameterized as two low-rank matrices B and A, where the effective weight update is W + (α/r) × BA. With rank r=16 and scaling factor α=32, the number of trainable parameters drops from 124M to 2.07M — 0.24% of the full model.

This is not a compromise on performance. The LoRA constraint actively prevents overfitting on small datasets by limiting the effective model capacity. Our CSI-GCB model achieved F1 = 0.7012 after 30 training epochs, with validation F1 improving monotonically across all epochs — no overfitting, no degradation. The parameter-efficient constraint was a feature, not a limitation.

Metric	Value
Base model parameters	124M
Trainable parameters (LoRA)	2.07M (0.24%)
LoRA rank (r)	16
LoRA scaling factor (α)	32
Training epochs	30
Optimizer	AdamW, lr=2e-5
Best validation F1	0.7012

Building the Dataset: Three Real-World Sources

We could not use existing C/C++ vulnerability datasets. Cross-language transfer from C/C++ to Python is problematic: graph structures differ, tokenization differs, and the vulnerability patterns that dominate C/C++ (buffer overflows, memory corruption) are largely irrelevant in Python. We needed Python-native training data.

We unified approximately 4,000 deduplicated Python functions from three complementary real-world sources.

Source 1: AI-Generated Vulnerable Code (121 records) A curated dataset of AI-generated Python functions, each labeled with its CWE identifier. Every record pairs a natural-language prompt with the insecure Python function produced by the AI model, covering 68 unique CWE types. This source directly targets the threat model motivating CSI: AI-assisted code generation introducing security vulnerabilities that no one audits.

Source 2: GitHub Security Commits (2,173 records) Commit-level vulnerability pairs extracted from real GitHub security fix commits. Pre-patch function = vulnerable, post-patch function = safe. Labels verified using GPT-4 at approximately 94% accuracy — compared to 40–51% accuracy for automated commit-only labeling strategies. Our GPT-4 verification step was essential for training signal quality.

Source 3: Raw GitHub Diff Files (~300 records) Approximately 300 raw GitHub diff records across seven vulnerability types: XSS, SQL injection, command injection, open redirect, path disclosure, RCE, and XSRF. Incorporated as augmentation for underrepresented CWE categories.

One Critical Insight: Commit-Stratified Splitting Random train/test splits leak information when applied to commit-level data. Functions from the same Git commit share context: the same bug fix, the same coding style, the same changeset patterns. Published research shows this inflates F1 scores by up to 40 percentage points. Our solution: entire commits assigned to a single partition. No commit ever split across train and test.

The Preprocessing Pipeline

Six stages transform raw data into model-ready tensors.

Stage 1 — Parse and Unify: Each source normalized into a unified schema: source code, binary label, CWE identifier, provenance tag.
Stage 2 — Label Encoding: CWE-to-integer mapping constructed. Categories with fewer than 5 samples discarded.
Stage 3 — Negative Sampling: Safe samples drawn from post-patch functions and CodeSearchNet. Target ratio: 1:1 vulnerable-to-safe, correcting the natural 8:1 imbalance.
Stage 4 — Class Weighting: Per-class weights via scikit-learn compute_class_weight. Positive weight pos_weight = n_neg/n_pos for the binary head.
Stage 5 — Mid-Truncation Tokenization: Max 512 tokens. First 128 (function signature, entry logic) + last 384 (return statements, taint sinks) retained. Standard head truncation discards function tails — exactly where SQL and command injection sinks most commonly appear.
Stage 6 — Commit-Stratified Split: 70/15/15 train/validation/test. All functions from the same commit in the same partition.

Data Augmentation

We applied two augmentation techniques to the training set.

Variable and function name normalization replaces all identifier tokens with abstract symbolic tokens (VAR_1, FUNC_1, etc.), adopted from the DetectVul preprocessing strategy. A SQL injection through a variable named user_input and one through a variable named x are the same vulnerability. The model should treat them identically.

Dead-code insertion and minor refactoring variants of each vulnerable function were generated to increase intra-class diversity. This was motivated by a known failure mode in GNN-based detectors: models trained to distinguish vulnerable code from its fixed version perform near-randomly because security patches introduce minimal code differences. Increasing intra-class diversity forces the model to learn structural patterns rather than diff signatures.

The Architecture: Dual-Encoder Fusion

The architecture decision was one of the first major forks in the project, and it shaped almost everything that came after it. Early on, we had to decide: do we build one encoder that does everything, or do we combine two encoders that each bring something the other lacks? We went with the second option, but getting there — and then getting the two halves to actually work together — took several iterations.

Why Two Encoders At All

The case for a single encoder is simplicity: one model, one set of weights to fine-tune, fewer moving parts to debug. But GraphCodeBERT and VulBERTa are good at fundamentally different things. GraphCodeBERT understands structure — how data moves through a function, how control flow branches, how an AST is shaped. VulBERTa understands vulnerability semantics — it was pre-trained exclusively on NVD entries and CVE-linked code, so it has effectively memorized what dangerous code idioms look like: unsanitized input patterns, risky function calls, structures that resemble known CVEs.

A function can be structurally unremarkable — a simple, shallow control flow, nothing exotic in its data flow graph — and still be dangerous because of what it does with a specific input. Conversely, a function can have a complex data flow graph and still be perfectly safe. Structure alone doesn't tell you "this looks like a CVE I've seen before," and vulnerability-pattern memorization alone doesn't tell you "this input actually reaches this sink." We wanted both signals available to the classification heads simultaneously, which meant running both encoders on every input and combining their outputs — rather than picking one.

Encoder 1: GraphCodeBERT + LoRA

The first encoder is GraphCodeBERT, adapted with LoRA as described earlier. For every input function, GraphCodeBERT processes two things at once: the tokenized source code itself, and the data-flow graph edges extracted from the function's AST. Internally, its attention layers attend across both — a token can attend not just to nearby tokens in the sequence, but to other tokens it has a data-flow relationship with, even if they're far apart in the raw text. This is what lets the model "see" that a variable assigned on line 3 flows into a database call on line 40, even though those two lines are nowhere near each other as tokens.

The LoRA adapters sit on the query, key, and value projection matrices of every attention layer in this encoder. Everything else in GraphCodeBERT is frozen. After the full forward pass, we take the per-token output and apply mean pooling across the sequence dimension — averaging every token's final representation into a single vector. The result is a 768-dimensional embedding that we think of as the structural signal: it encodes how this specific function is built, how its data moves, and how its control flow is organized.

Encoder 2: VulBERTa (Frozen)

The second encoder is VulBERTa, used as a fixed feature extractor. Unlike GraphCodeBERT, VulBERTa receives no adapters and no gradient updates at all — its weights are exactly as they came from pretraining on NVD and CVE-linked code. We made this choice deliberately: VulBERTa's value to us is precisely the vulnerability-domain knowledge baked into its pretraining, and fine-tuning it on our comparatively small dataset risked overwriting that knowledge faster than it could learn anything useful from 4,000 samples — a classic catastrophic forgetting problem.

For every input function, VulBERTa runs its own tokenization (it uses RoBERTa's BPE tokenizer, separate from GraphCodeBERT's tokenizer — these are two different views of the same source code, tokenized differently) and produces a sequence of hidden states. Rather than mean pooling, we take the CLS token's final representation — the standard approach for classification-style embeddings in BERT-family models — giving us a second 768-dimensional embedding. We think of this as the vulnerability-domain signal: it encodes how similar this function "feels" to the vulnerable and CVE-linked code VulBERTa was trained on, independent of the function's own internal structure.

Fusion Layer: Combining Two 768-Dimensional Views

At this point we have two 768-dimensional vectors describing the same input function from two different angles. The fusion layer's job is to combine them into a single representation that downstream classification heads can use.

The simplest possible approach — and the one we settled on — is concatenation: stack the two 768-dimensional vectors end to end into a single 1,536-dimensional vector. We considered alternatives (element-wise addition, learned gating, cross-attention between the two embeddings) but concatenation has one major advantage: it loses no information. Addition or gating require the two vectors to already be in a compatible space, which they aren't — they come from different models with different pretraining objectives. Concatenation defers that reconciliation to a layer that's actually trained for it.

That reconciliation happens in the FusionProjectionLayer immediately after concatenation: a Linear layer projects the 1,536-dimensional concatenated vector back down to 768 dimensions, followed by LayerNorm, a GELU activation, and Dropout. This is the layer that actually learns how to weigh and combine the structural signal from GraphCodeBERT against the vulnerability-domain signal from VulBERTa — effectively learning, per-feature, how much to trust each encoder's contribution. The output is a single 768-dimensional fused representation that both downstream heads consume.

Classification Heads: Two Tasks, One Shared Representation

The fused representation feeds two separate heads, trained jointly.

The Binary Head is intentionally minimal: a single Linear(1536 → 1) layer followed by a sigmoid, producing a vulnerable/safe probability. We kept this head simple because the binary task is, relatively speaking, the easier of the two — most of the discriminative work needed for "is this vulnerable at all" is already present in the fused representation, and adding more layers here mainly risked overfitting on a task that didn't need it.

The CWE Head is deliberately deeper: Linear(1536 → 384) → GELU → Dropout → Linear(384 → 8). Classifying which of seven CWE categories a vulnerability belongs to (plus an eighth "unknown" class) is a harder, more fine-grained task than the binary one — it requires distinguishing between vulnerability types that can share a lot of surface-level structure (an XSS and a command injection can look superficially similar in terms of "untrusted input flows somewhere dangerous," but the kind of dangerous matters for classification). The extra hidden layer gives the head room to learn these finer distinctions from the same shared representation, without needing a separate encoder pass.

One detail that mattered in practice: for safe samples, the CWE label is set to −1 and masked out of the CWE loss entirely. A safe function has no CWE to predict, and including it in the CWE loss with some placeholder label would inject noise into a head that's already working with a smaller, more imbalanced label space than the binary head. Masking keeps the CWE head's gradient signal coming only from samples where a CWE label is actually meaningful.

Where the Two-Head Design Came From

This two-head structure wasn't the original plan — early versions of the architecture experimented with a single combined output space (CWE categories plus an explicit "safe" class, predicted by one head). We moved to separate binary and CWE heads after running into a familiar problem: a single combined classifier tends to behave like a generalist binary detector with poor sensitivity to specific weakness types, because the "safe" class dominates the label distribution and pulls the decision boundary toward itself. Splitting the binary detection objective from the CWE classification objective let each head specialize — one for the broad "is this dangerous" question, one for the fine-grained "what kind of dangerous" question — while still sharing the same upstream encoders and fusion layer, so neither head requires its own separate feature extraction.

Training Strategy: Composite Loss and Adversarial Training

Getting a model to F1 = 0.7012 on a 4,000-sample dataset with eight unevenly distributed classes is not a single-loss problem. A model trained with plain cross-entropy on this data converges quickly to predicting the dominant classes and essentially ignores rare CWE categories — the validation F1 looks acceptable on paper while the model is functionally blind to the vulnerability types that matter most. We addressed this by combining four loss components, each solving a different failure mode we hit during early experiments.

Focal Loss: Fixing the Class Imbalance Problem

Our CWE distribution is heavily skewed — some categories have hundreds of examples, others barely clear the five-sample minimum from Stage 2 of preprocessing. With standard cross-entropy, the gradient signal from the dominant classes drowns out the rare ones, and the model learns to be "confidently correct" on easy majority-class examples while never improving on the hard minority-class examples.

Focal Loss adds a modulating term, (1 − p_t)^γ, to the standard cross-entropy loss. When the model is already confident and correct on an example (p_t close to 1), this term shrinks toward zero and the loss contribution from that example is suppressed. When the model is wrong or uncertain (p_t low), the term stays close to 1 and the full loss applies. In practice, this redirects the gradient budget toward the examples the model is actually struggling with — which, in our case, were almost always the underrepresented CWE categories. We ran an ablation over γ ∈ {1, 2, 3} to find the value that best balanced this trade-off without destabilizing training on the majority classes.

SCL-CVD: Making the Embedding Space Class-Aware

Focal Loss fixes the gradient imbalance, but it doesn't directly address a separate problem: two functions with the same CWE label can look extremely different at the token level, while two functions with different CWE labels can look superficially similar (a few lines of diff apart). Without an explicit signal to organize the embedding space, the classification head has to do all the work of separating classes from a representation that wasn't built with that goal in mind.

Supervised Contrastive Learning for Code Vulnerability Detection (SCL-CVD) adds a second objective directly on the embeddings, before the classification heads. For every anchor sample in a batch, it pulls embeddings from the same CWE class closer together in representation space and pushes embeddings from different classes apart, using a temperature parameter (tau) to control how sharply similarity is weighted. The result is an embedding space where same-class functions cluster together even if their surface code differs substantially, and where the classification head's decision boundaries become easier to learn because the classes are already partially separated upstream.

This was one of the more iteration-heavy components of the project. We ran a direct SCL vs. no-SCL F1 comparison to confirm the contrastive objective was actually helping (it was), then separately tuned the temperature tau and the SCL loss weight alpha — two parameters that interact with each other and with the rest of the composite loss in non-obvious ways. Too high a weight on SCL and the model over-prioritizes embedding geometry at the expense of classification accuracy; too low and the contrastive signal gets lost in the noise of the other three losses.

R-Drop: Consistency Under Dropout

Dropout is applied during training for regularization, but it introduces a subtle problem: the same input, passed through the model twice, can produce noticeably different output distributions depending on which neurons happen to be dropped each time. For a model that needs to make confident, stable predictions about whether a specific line of code is vulnerable, this stochasticity is undesirable — it means the model's "opinion" about a given function can shift run to run without any change to the input.

R-Drop addresses this directly. Each training sample is passed through the model twice, with two independent dropout masks, producing two output distributions. The loss adds a KL divergence term between these two distributions, on top of the standard task loss. This forces the model to produce consistent predictions regardless of which dropout mask is active — effectively training the model to be robust to its own regularization noise. We tested this on small batches first to confirm the KL term behaved as expected (it shouldn't dominate the loss or collapse the distributions to a degenerate point) before integrating it into the full training loop.

EDAT: Adversarial Robustness with Syntactic Guarantees

The fourth component, Embedding-Disturbed Adversarial Training (EDAT), was the most involved to build, and the one that touched the most hands on the team.

The motivation is straightforward: vulnerability detection models are often brittle to small, semantically meaningless changes in code — renaming a variable, adding a comment, reordering independent statements. A model that flips its prediction because of a cosmetic change isn't actually reasoning about the vulnerability; it's keying on superficial patterns. EDAT trains against this by generating adversarial perturbations in the model's embedding space using Projected Gradient Descent (PGD) — small, gradient-directed nudges to the embedding designed to push the model toward a wrong prediction — and then training the model to be robust to those nudges via an adversarial KL loss between the clean and perturbed predictions.

The catch is that perturbations applied carelessly in embedding space can correspond to nothing — there's no guarantee that a perturbed embedding still maps back to anything resembling valid Python. That's where AST constraint checking comes in: before a perturbed sample is accepted into the adversarial training loop, it's checked against AST-level constraints to ensure the perturbation corresponds to a syntactically valid transformation, not an out-of-distribution artifact the model could "cheat" against.

Building this pipeline required several distinct pieces working together: tree-sitter-based identifier extraction to identify which tokens in a function are safe to perturb without breaking syntax, the AST constraint checker itself, the PGD perturbation loop, the adversarial KL loss term, and a tunable epsilon controlling the perturbation magnitude. Each of these had to be validated independently on small batches before being wired into the full training run, because a bug in any one component (a perturbation that's too large, an AST check that's too permissive, an epsilon that's miscalibrated) can silently degrade training without throwing an error — the model still trains, it just gets worse, and that's much harder to debug than a crash.

Who Built What

This was genuinely distributed work, and each piece depended on the one before it. Hend Elhout built the tree-sitter identifier extraction that underpins EDAT — the foundation everything else in the adversarial pipeline depends on. Jomana Mekheimar implemented the AST constraint checking on top of that, and later ran the full EDAT training and the EDAT vs. no-EDAT F1 comparison that validated the whole approach was worth the added complexity. Menna Reda built the PGD perturbation loop and the adversarial KL loss, and ran the small-batch tests that caught early calibration issues. Youstina Adel tuned the epsilon parameter and did the final integration of EDAT into the main training loop. Separately, Farida Hassan implemented and tested both SCL-CVD and R-Drop, and later ran the full GraphCodeBERT training run that produced our final F1 = 0.7012 result.

The composite loss that resulted from all of this — Focal Loss, SCL-CVD, R-Drop, and EDAT, combined and weighted together — was the single largest factor separating our early-epoch results (F1 ≈ 0.54) from the final 0.7012. No individual component alone got us there; it was the combination, tuned iteratively, that did.

Results

Model	Method	Language	F1	Notes
Bandit	Regex	Python	~0.62	Industry standard
DetectVul	Dual-BERT, full fine-tune	Python	0.7447	Prior SOTA
CSI-GCB	GraphCodeBERT + LoRA	Python	0.7012	0.24% of params trained
CSI-Dual	GCB + VulBERTa fusion	Python	0.6630	Faster early convergence

CSI-GCB outperforms Bandit by 8 percentage points — roughly 13 additional vulnerabilities caught per 100. The single-encoder LoRA model outperformed the dual-encoder fusion model by 3.82 points. CSI-Dual showed faster early convergence (F1 = 0.6099 at epoch 1 vs ~0.541 for CSI-GCB) but plateaued earlier because frozen VulBERTa could not adapt to our seven-class CWE taxonomy. At ~4,000 training samples, the fusion layer's complexity outweighs its gains. This is a data scale problem, not an architecture problem.

Why This Matters

For security teams: a tool that traces taint flows rather than flagging token patterns, catching injections that Bandit misses entirely.

For ML practitioners: validation that LoRA is viable for production security tasks. 0.24% of parameters updated, F1 = 0.7012. Meaningful security tooling without large GPU infrastructure.

For researchers: a reproducible Python-specific multi-task vulnerability detection baseline demonstrating parameter-efficient single-encoder fine-tuning can approach full fine-tuning performance at significantly lower compute cost.

Metric	Result
Training Time	~30 epochs on A100
Trainable Parameters	2.07M (0.24%)
F1 Score	0.7012

Building the Dataset: Three Sources, One Pipeline

We unified approximately 4,000 deduplicated Python functions from three real-world sources:

AI-Generated Code (121 records): GPT-assisted Python functions labeled with CWE identifiers, spanning 68 unique CWE types. Primary training signal for the CWE classification head.
GitHub Security Commits (2,173 records): Real commit-level vulnerability pairs where pre-patch = vulnerable and post-patch = safe, verified by GPT-4 at ~94% label accuracy — vs. 40–51% for automated commit-only labeling.
Raw Diff Files (~300 records): GitHub diffs across seven vulnerability types: XSS, SQL injection, command injection, open redirect, path disclosure, RCE, and XSRF. Used as augmentation for underrepresented CWE categories.

One Critical Insight: Commit-Stratified Evaluation

Random data splits leak information. Functions in the same Git commit share context. If the model sees part of a commit during training and the rest during testing, it learns commit-specific signatures — metrics inflate by up to 40 percentage points. Solution: entire commits are assigned to either train or test, never split across partitions.

The Architecture: Dual-Encoder Fusion

CSI runs two encoders in parallel on every input:

GraphCodeBERT + LoRA: captures AST structure, data flow between variables, and token relationships. Outputs a 768-dimensional embedding.
VulBERTa (frozen): a RoBERTa model pre-trained exclusively on NVD entries and CVE-linked code. Captures dangerous code idioms, unsanitized input patterns, and similarity to known vulnerable code. Outputs a 768-dimensional embedding.

Both embeddings are concatenated into a 1,536-dimensional vector, passed through a FusionProjectionLayer (Linear → LayerNorm → GELU → Dropout), then routed to two task heads: a binary head (Linear 1536→1) predicting vulnerable vs. safe, and a CWE head (Linear 1536→384 → GELU → Dropout → Linear 384→8) classifying across seven CWE categories plus unknown.

Training used a composite loss: Focal Loss for class imbalance, SCL-CVD for intra-class compactness, R-Drop for output consistency, and EDAT adversarial perturbation on embeddings with AST constraint checking to preserve program semantics.

Results: Outperforming Baselines

Model	Language	Method	F1
Bandit	Python	Regex	~0.62
DetectVul	Python	Dual-BERT	0.7447
CSI-GCB	Python	GraphCodeBERT+LoRA	0.7012
CSI-Dual	Python	GCB+VulBERTa	0.6630

CSI-GCB outperforms Bandit by 8 percentage points on Python. The single-encoder LoRA approach outperforms the dual-encoder fusion under current data scale — dual-encoder architectures require larger corpora to realize their complementary representation advantage.

Why This Matters

Security teams get a tool that understands code semantics, not surface patterns. ML practitioners see validation of parameter-efficient fine-tuning (LoRA) on a real security task. Microsoft's GraphCodeBERT + PEFT ecosystem proves viable for production Python security tooling.

Meet the Team

CSI was built by a 9-person team at the Egyptian Chinese University.

Member	Tasks Owned	Specific Contributions	LinkedIn
Anas Abuelhaag	Project lead, architecture, training infrastructure	GitHub repo setup, Drive infrastructure, CWE classification head design, checkpoint save/load, validation loop (F1/precision/recall), SCL integration into training loop, overall system architecture	LinkedIn Profile
Sohaila Tamer	Graph extraction, SCL tuning, VulBERTa go/no-go	AST/CFG/DFG graph extraction, SCL vs no-SCL F1 comparison, temperature tau tuning, SCL weight alpha tuning, results logging, VulBERTa go/no-go decision	LinkedIn Profile
Farida Hassan	Tokenization, loss functions, full training run	GraphCodeBERT tokenization cache, checkpoint co-development, validation loop co-development, supervised contrastive loss implementation and testing, R-Drop KL divergence loss, full GraphCodeBERT training run	LinkedIn Profile
Hend Elhout	EDAT identifier extraction, Streamlit UI	Tree-sitter identifier extraction for EDAT, Streamlit line highlighting display, 7 fix suggestion texts, example code snippets, loading spinner, edge case handling, VulBERTa fusion layer implementation	LinkedIn Profile
Jomana Mekheimar	AST constraints, hyperparameter search, VulBERTa tokenization	EDAT AST constraint checking, full EDAT training run, EDAT vs no-EDAT F1 comparison, LoRA rank ablation (4 vs 8 vs 16), focal loss gamma ablation (1 vs 2 vs 3), VulBERTa BPE tokenization	LinkedIn Profile
Menna Reda	PGD adversarial training, VulBERTa training	EDAT PGD perturbation loop, adversarial KL loss, small-batch EDAT testing, VulBERTa dual-encoder training run	LinkedIn Profile
Youstina Adel	Epsilon tuning, VulBERTa LoRA	EDAT epsilon tuning, EDAT integration into training loop, VulBERTa LoRA adapter implementation	LinkedIn Profile
MennatAllah Amr	Hyperparameter optimization, VulBERTa forward pass	Batch size ablation (4 vs 8), learning rate ablation (1e-5, 2e-5, 5e-5), hyperparameter results compilation and best config selection, VulBERTa dual-input forward pass update	LinkedIn Profile
Hesham Elshimy	Streamlit app, deployment, demo	Streamlit UI layout design, hardcoded data testing, app.py with code input, 4 metric cards, CWE description section, 7 CWE descriptions, model download integration, model connection, safe/vulnerable code testing, demo video production	LinkedIn Profile

REFERENCES

[1] PyVul Team, "PyVul: A Real-World Python Vulnerability Benchmark with LLM-Assisted Data Cleansing," arXiv:2404.15687, 2024.

[2] Y. Chen et al., "DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning-Based Vulnerability Detection," in Proc. ACM

RAID, 2023, pp. 1-16.

[3] H. Husain et al., "CodeSearchNet Challenge: Evaluating the State of Semantic Code Search," arXiv:1909.09436, 2019.

[4] Y. Feng et al., "CodeBERT: A Pre-Trained Model for Programming and Natural Languages," in Proc. EMNLP Findings, 2020.

[5] M. T. Tran et al., "DetectVul: Statement-Level Python Vulnerability Detection Using Dual-BERT," Future Generation Computer

Systems, 2025.

[6] R. Mussabayev, "Structure-Aware Code Vulnerability Analysis with Graph Neural Networks," arXiv:2307.11454, 2023.

[7] Anonymous, "From Generalist to Specialist: Exploring CWE-Specific Vulnerability Detection," arXiv:2408.02329, 2024.

[8] H. Hanif and S. Maffeis, "VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detection," in Proc. IJCNN, 2022.

[9] J. Liu et al., "Vul-LMGNNs: Vulnerability Detection by Fusing Language Models and Online-Distilled Graph Neural Networks,"

arXiv:2404.14719, 2025.

[10] X. Wen et al., "AMPLE: Vulnerability Detection with Graph Simplification and Enhanced Graph Representation Learning,"

arXiv:2302.04675, 2023.

[11] L. Peng et al., "ANGEL: Accurate Vulnerability Detection for Large Code Graphs," arXiv:2412.10164, 2024.

[12] J. de Kraker, H. Vranken, and A. Hommersom, "MultiGLICE: GNNs with Program Slicing for Multiclass Vulnerability Detection,"

Computers, vol. 14, no. 3, p. 98, 2025.

[13] Anonymous, "Vignat: Vulnerability Identification by Learning Code Semantics via Graph Attention Networks," arXiv:2310.20067,

2023.

[14] Anonymous, "Enhancing Vulnerability Detection Using Code Property Graphs and CNNs," in Proc. ACM CCS Workshop, 2023.

[15] Y. Hu et al., "Interpreters for GNN-Based Vulnerability Detection: Are We There Yet?," in Proc. ACM ISSTA, 2023.

[16] K. Wartschinski et al., "VUDENC: Vulnerability Detection with Deep Learning for a Large Codebase in Python," Computers, vol. 14,

3, 2025.

[17] C. Liang et al., "Source Code Vulnerability Analysis Based on Deep Learning: A Survey," Computers & Security, vol. 148, 2025.

[18] Y. Hu et al., "SoK: Automated Vulnerability Repair, Methods, Tools, and Assessments," arXiv:2506.11697, 2025.

[19] G. Bhandari, P. Gavric, and A. Shalaginov, "PatchLM: Generating Vulnerability Security Fixes with Code Language Models,"

Information and Software Technology, vol. 185, 2025.

[20] D. Guo et al., “GraphCodeBERT: Pre-Training Code Representations with Data Flow,” in Proc. International Conference on Learning

Representations (ICLR), 2021.

Building ShadowQuest: A Multi-Agent RPG

ShardaKaur — Mon, 15 Jun 2026 08:00:00 GMT

Artificial Intelligence is rapidly evolving beyond traditional chatbots. Today, developers are building intelligent systems where multiple AI agents collaborate, retrieve knowledge, and solve problems together. Microsoft's Agents League Hackathon provided the perfect opportunity to explore this new approach through the Reasoning Agents challenge.

For this challenge, I built ShadowQuest, a fantasy role-playing game (RPG) powered by Microsoft Foundry, Foundry IQ, Azure AI Search, GPT-4.1, and GitHub Copilot. The project demonstrates how specialized AI agents can work together while using Retrieval-Augmented Generation (RAG) to deliver accurate and context-aware responses.

About the Challenge

Microsoft Agents League is a global developer challenge designed to encourage developers to build intelligent AI applications using Microsoft's latest AI technologies. Participants could choose from three tracks: Creative Apps, Reasoning Agents, and Enterprise Agents.

I selected the Reasoning Agents track because I wanted to explore how multiple AI agents could collaborate instead of relying on a single large language model. Another important requirement for this year's challenge was integrating at least one Microsoft Intelligence Layer. For ShadowQuest, I chose Foundry IQ as the project's intelligence layer.

The Idea Behind ShadowQuest

Fantasy RPGs are built around storytelling, exploration, and collaboration between different characters. Every character usually has a unique role, whether it's a warrior protecting the team, a mage interpreting magical knowledge, or a rogue discovering hidden paths.

I wanted to recreate this experience using AI.

Instead of building one AI assistant responsible for everything, I designed a system where multiple specialized agents collaborate to create a richer and more immersive adventure.

ShadowQuest is set in a fantasy world filled with magical artifacts, forgotten kingdoms, mysterious locations, and story-driven quests. Players can ask questions about the world, explore different locations, and learn about the game's lore through conversations with AI agents.

Building the Multi-Agent Architecture

The architecture follows a simple but scalable design.

At the center of the system is the Game Master Agent, which acts as the orchestrator. Every player interaction starts with the Game Master. It receives the player's request, determines what information is needed, retrieves additional knowledge when required, and generates the final response.

Supporting the Game Master are three specialized agents:

Warrior Agent – Focuses on combat strategy and tactical decisions.
Mage Agent – Provides magical knowledge, world lore, and information about ancient artifacts.
Rogue Agent – Specializes in exploration, investigation, and discovering hidden information.

Each agent has a clearly defined responsibility, making the system easier to understand, maintain, and extend in the future.

Using Foundry IQ as the Knowledge Layer

One of the most important parts of the project was integrating Foundry IQ.

Instead of storing every piece of game information inside prompts, I created a dedicated knowledge base containing information about characters, magical artifacts, locations, quests, and the history of the ShadowQuest world.

This approach separates knowledge from reasoning.

Whenever a player asks a question, the Game Master Agent first retrieves relevant information from the knowledge base before generating a response. This ensures that answers remain consistent with the game's world while reducing hallucinations.

Foundry IQ became the central source of truth for the entire project, making it easy to manage and expand the game world without constantly modifying prompts.

Azure AI Search and Retrieval-Augmented Generation

To enable intelligent retrieval, I connected Foundry IQ with Azure AI Search.

The RPG documents were indexed, and vector embeddings were generated using Microsoft's embedding models. This enables semantic search, allowing the system to understand the meaning behind a player's question instead of relying only on keyword matching.

For example, if a player asks about a magical relic without mentioning its exact name, Azure AI Search can still retrieve the correct information based on semantic similarity.

The complete workflow looks like this:

The player submits a question.
The Game Master Agent receives the request.
Foundry IQ queries Azure AI Search.
Relevant documents are retrieved.
GPT-4.1 generates a grounded response using the retrieved context.

This Retrieval-Augmented Generation (RAG) approach significantly improves the quality and reliability of responses.

Accelerating Development with GitHub Copilot

GitHub Copilot played an important role throughout the development process.

It helped generate Python classes, improve documentation, create helper functions, and speed up repetitive coding tasks.

During the live demonstration, I also showed how Copilot could quickly generate a new Healer Agent, demonstrating how AI-assisted development makes it easier to extend a multi-agent application while maintaining a consistent architecture.

Rather than replacing the developer, Copilot acted as an intelligent coding assistant, allowing me to focus more on architecture and design decisions.

Demonstrating ShadowQuest

During the Microsoft Agents League Reasoning Agents Battle, I demonstrated the Game Master Agent by asking questions about the ShadowQuest world, magical artifacts, and game lore.

One of the most interesting parts of the demonstration was observing the retrieval process.

Before generating a response, the Game Master Agent called the knowledge retrieval function through Foundry IQ. This confirmed that the system was retrieving relevant information from the indexed knowledge base rather than relying only on GPT-4.1's internal knowledge.

This demonstrated how RAG can create more grounded, reliable, and context-aware AI experiences.

Lessons Learned

Building ShadowQuest taught me that designing multi-agent systems is as much about architecture as it is about AI models.

Clearly defining responsibilities for each agent made the application easier to maintain and opened the door for future expansion.

I also learned how valuable Retrieval-Augmented Generation can be for applications that depend on structured knowledge. Separating reasoning from knowledge allows AI systems to remain accurate while making it easier to update information over time.

Finally, participating in the Microsoft Agents League was an incredible opportunity to experiment with Microsoft's latest AI technologies, learn from other developers, and share ideas with a global community passionate about agentic AI.

Looking Ahead

ShadowQuest is only the beginning.

In future iterations, I plan to expand the project by introducing additional agents such as a Merchant Agent and Healer Agent, implementing persistent player memory, adding dynamic quest generation, improving combat mechanics, and enabling deeper collaboration between agents.

These improvements will make the game world more immersive while continuing to explore the possibilities of agent-based AI systems.

Conclusion

ShadowQuest demonstrates how Microsoft Foundry, Foundry IQ, Azure AI Search, GPT-4.1, and GitHub Copilot can be combined to build intelligent multi-agent applications.

More importantly, the project reinforced an important idea: the future of AI is not a single assistant performing every task, but a team of specialized agents collaborating with shared knowledge to solve increasingly complex problems.

Participating in the Microsoft Agents League was an inspiring experience that allowed me to explore the next generation of AI development while building a project that combines storytelling, reasoning, and knowledge retrieval. I look forward to continuing this journey and discovering new ways to build intelligent applications using Microsoft's growing AI ecosystem.

Gamifying World Improvement: A Reasoning-Agent RPG on Microsoft Foundry

Princeps — Mon, 15 Jun 2026 06:40:36 GMT

Building a multi-agent demo is the straightforward part. Building one where you can prove, live, with judges watching, that the agents reasoned, retrieved grounded context, called tools, and deferred to a human before awarding anything: that is where most teams run into friction.

We found out on stage. This project was demoed live at the Agents League Reasoning Agents battle on Microsoft Reactor: real-time narration, real Foundry calls, real latency, and a host watching the terminal.

For Microsoft Agents League Battle #2 (Reasoning Agents) the brief was the classic Game Master pattern: an orchestrator decomposes a goal, specialist agents execute, shared state tracks the world. We reskinned it with the biggest stakes we could defend. The game opens in a world that terraforms the Sahara and automates basic needs: food, water, energy, shelter. A vision that size is never commanded into existence; it has to be aligned. The player enters the story by founding a company on one front of the mission and taking the CEO's chair.

From there the loop is concrete. The player pitches the idea (or pastes a real company URL); a Master Narrator turns it into a quest line; a digital workforce, designed per company rather than hardcoded, produces real launch artefacts: positioning, landing-page structure, launch copy. Nothing earns XP until the human CEO approves it at a verification gate. That is the game's one law, and its title mechanic: your company is the dungeon.

A mission too big to command needs a workforce you can verify: reason on Foundry, ground with Foundry IQ, validate with deterministic tools, ask the human before anything counts.

Why a reasoning-agent demo is different

A normal chatbot demo gates on "did it answer." A reasoning-agent demo has to gate on something harder: can you show the work, and can you stop the work? That reframes the whole build. The deployment unit is not "a model." It is an orchestrated run with a visible decomposition, logged tool calls, cited retrieval, and a human decision point. Borrowing Lee Stott's framing from CI/CD for AI Agents on Microsoft Foundry: release gates should be driven by evaluation outcomes, not just test results. We applied that idea one level down, to each artefact, at runtime.

The architecture in one diagram

Two properties to read off the diagram. First, the loop is closed: the CEO's gate decision is written to agent memory and recalled into the next chapter's brief. Second, every cloud arrow has a keyless fallback. Clone the repo with no credentials and the whole game still plays in simulation mode.

The pipeline: four agents before any work happens

Most multi-agent demos hardcode their cast. We do not. The business defines the workforce; the workforce defines the quests. Four agents run before any artifact work starts:

Company Analyst. Scrapes a real URL, reasons about the business, seeds the brief.
Org Designer. Designs a digital-workforce blueprint for this specific company.
World Designer. Decomposes the pitch into a chapter quest line.
Worker Factory. Binds each chapter to a worker and builds it as an Agent Framework agent with tools.

Pitch a bakery and you get a different org chart, and different quests, than a dev-tools startup. That is the difference between a scripted demo and a system.

Workers are real Agent Framework agents on Foundry

Each designed worker is built at runtime with the Microsoft Agent Framework, with inference through the Foundry project Responses endpoint under AAD auth. Keyless, no secrets in .env:

# agents/maf_runtime.py
from agent_framework.foundry import FoundryChatClient

client = FoundryChatClient(
    project_endpoint=foundry_project_endpoint(),
    model=deployment,          # gpt-5 family deployment
    credential=_aad_credential(),   # DefaultAzureCredential - no API key
)

Our deterministic validators are exposed to the model as real FunctionTools, capped so a stuck model cannot loop, and every mid-run call writes a receipt (args, result, latency) into the replay log:

@tool(name=tool_name,
      description=f"Run the deterministic '{tool_name}' check on a draft artifact...",
      max_invocations=2)
def _t(artifact_json: str) -> str:
    meta["maf_tools_called"].append(tool_name)
    receipt = {"tool": tool_name, "source": "maf-midrun", "args": {}, "result": "", "ms": 0.0}
    ...

So the model can check its own draft mid-run, but the gate score never comes from the model alone.

Best practice: deterministic gates first, model judgement second

This is the single most important reliability decision, and it is the same principle Lee Stott states in the hybrid agents post: code the rules, and let the LLM judge only what is left. Our gate score is a weighted rubric with a deterministic floor. Structural validators (does the landing page have a headline, a CTA, a hero section; is the email parseable; do URLs resolve) set the minimum, and the narrator's rubric judgement can only move the score above that floor, never below the facts.

Three scoring layers, and only one of them can award XP:

Mid-run tool calls. Scored by deterministic validators. Advisory to the model only; cannot award XP.
rubric_evaluate. A Foundry model judges weighted dimensions, floored by the validators. Cannot award XP.
The CEO gate. The human. The only path to XP.

Four proof points are logged on every single invocation, including simulation mode: iq_hits, memory_injected, tools_called, inference_usage. Every claim in the demo is a logged number in the replay log. It is the same discipline as carrying a correlation ID through every path.

Operational lessons learned

Stream the reasoning, not just the answer. The UI is a reasoning theater fed by SSE. Every decomposition beat, tool receipt, and retrieval hit arrives as an event. If a user cannot see the handoff, the system feels like one big chatbot.
Give every cloud arrow a dashed twin. Foundry IQ falls back to local markdown retrieval; the Agent Service memory store falls back to a local JSON file; the model path falls back to scripted simulation. A live demo that depends on perfect connectivity is a demo waiting to fail.
Cap tool invocations. max_invocations=2 on every FunctionTool. Without it, a model in a tight spot calls the validator in a loop.
Reasoning models and strict JSON do not mix. Anything that must emit JSON gets a tolerant extractor (_extract_json) that survives think-blocks and markdown fences. This is the same think-block gotcha Lee Stott flags for local router models.
Scrub secrets at the sink. Captured reasoning traces run through a secret scrubber before they touch the replay log.

Responsible AI

The verification gate is the responsible-AI story, and it is also the lore's law: in the game's fiction, a human holds the seal, because a vision too big to command must keep a human at the root of every result. No artefact becomes progress without explicit human approval; every approval is logged with the full reasoning chain; rejected work goes back for rework with the rejection written into agent memory as binding direction. Deterministic validators bound what the model can claim about its own output, and the replay log preserves the whole chain for audit. Auth is keyless via DefaultAzureCredential. Nothing to leak, rotate, or commit by accident.

Try it

Five minutes, no Azure account needed for the full game loop:

git clone https://github.com/princepspolycap/agentsleague-afterbuild
cd agentsleague-afterbuild
python3 -m venv .venv && source .venv/bin/activate
pip install -r submission/requirements.txt

python3 submission/tools/run_quest_simulation.py --pitch "Your idea here"

For live Foundry runs, copy submission/.env.example to submission/.env and point it at your Foundry project endpoint and a gpt-5 family deployment, then az login.

Where this goes next

Asked on stream where the project goes after the battle, the answer was already on the roadmap: "I'll deploy what I have as a web app that people can just use... I want to do some local models, make it even more accessible. Something people can play with even after the fact." That is the rest of this series. Part 2 covers how the agents remember the CEO's decisions, and Part 3 covers the fallback architecture and routing toward Foundry Local. The dungeon stays open after the competition ends.

Useful links:

Watch the live battle: Agents League - Reasoning Agents on Microsoft Reactor
Clone the repo: github.com/princepspolycap/agentsleague-afterbuild
The official challenge (submissions close 14 June 2026): Agents League registration
Microsoft Foundry docs

Key takeaways

Map your domain onto a proven orchestration pattern (Game Master) instead of inventing one.
Design the workforce per input. The business defines the org, the org defines the quests.
Deterministic gates first, model judgement second. A validator floor stops a model talking its way past a broken artefact.
LLMs create, tools validate, humans approve, replay logs preserve.
Ship a simulation fallback for every cloud dependency. Forkability is reliability.
Log proof points on every invocation so every claim is an auditable number.

The dungeon is not a metaphor for difficulty. It is a metaphor for structure: a mission too big to command gets aligned one company at a time. Rooms you cannot skip, gates a human must pass, and a logged map of every step you took.

Copilot Notebooks and Study guide now available to Copilot Chat users

AnnoPadte — Wed, 17 Jun 2026 02:31:33 GMT

Every student knows the feeling: the test is coming, the materials are everywhere— and the hardest part is not finding information, it is knowing where to start. There is a PDF from the teacher. Slides from last week. A Word document with notes. Each piece has value. But studying means turning all of it into something usable: what to review first, what connects together, what still feels fuzzy, and what to practice learning.

That is why we are excited to share two updates for education.

First, Copilot Notebooks is now rolling out to Copilot Chat users, available with Microsoft 365 Education licenses. Copilot Notebooks are AI-powered workspaces for a subject or group project built on reference materials—bringing together all context behind a topic in one place for you or your study group and Copilot to collaborate on. This addresses one of the top asks we have heard from Microsoft 365 Education customers: bring the power of Copilot Notebooks to the education licenses schools already use.

This education expansion builds on the broader Copilot Notebooks announcement that brings Notebooks to all education and enterprise Copilot Chat users, including new ways to work with your own materials in Notebooks with mind maps, Study Guides, and coming soon - the ability to create Word documents, Excel spreadsheets, and PowerPoint presentations. You can read that announcement in the broader commercial Copilot Notebooks blog.

Copilot Notebooks are available in the Microsoft 365 Copilot web and desktop versions for Education users. Expect them to be available in Education tenants in the next two weeks. They will be available in OneNote in the weeks to follow.

Get started: Find Copilot Notebooks located in the Microsoft 365 Copilot app waffle.

Second, Copilot Notebook Study Guide is now generally available both for education and enterprise users. Study Guide is an AI-powered feature in Notebooks that turns the learning materials you provide into a complete, interactive study companion. Organized, editable, grounded in your references. Ready when you are.

For Education IT admins - what you need to know

Study guide lives inside Copilot Notebooks. Copilot Notebooks are available in the Microsoft 365 Copilot app. Copilot Notebooks are located in the Microsoft 365 Copilot app waffle.

With this update, Copilot Notebooks are available to Microsoft 365 Education A1, A3, and A5 users.

Study guide is available for education users ages 13+. Student accounts need the right Age Group in Microsoft Entra ID, and K-12 students ages 13-17 need Microsoft 365 Copilot Chat enabled by an IT admin before they can use Copilot Notebooks and Study Guide.

No additional deployment is needed for Study Guide. Study Guide is rolling out to enterprise and education customers starting June 11. It may take a few days to show up in your account.

What Study guide does

Study Guide takes the materials learners already have and helps turn them into a collection of organized study topics and activities of your choice.

Drop in PDFs, Word documents, PowerPoint presentations, or Excel files. Study Guide reads across those references, identifies the key ideas, and creates a multi-page study guide inside the notebook.

The important part: it is grounded in the sources you provide. It is not pulling random facts from the internet. Summary pages and Topic pages include citations back to the original materials, so learners can check where information came from and return to the source when something needs a closer look.

That matters for learning. It helps students stay connected to the actual course materials. It helps educators trust what students are practicing from turns citation-checking into a habit, not an afterthought.

What is available in Study guide

Study Guide creates materials that span all phases of learning: understand, practice, and test.

Understand: deepen your knowledge of the material

Summary page: Start with a high-level overview of the materials you added. The Summary includes an overview, why the topic matters, key topics, a glossary, common misconceptions, and citations back to the source materials.
Topic pages: Study Guide creates deeper pages for the major topics it finds in your content. These pages work like mini-chapters that cover content across all your references. They include explanations, sub-topic deep dives, worked examples, questions that make you think critically and analyze concepts, short exercises, and citations throughout.

Practice: strengthen recall, and make connections

Flashcards: Study Guide generates interactive cards from the learner's materials. Learners can flip cards, use hints, and edit the set so the wording matches how they think about the concept.
Fill in the blanks: Key terms are removed from important sentences, and learners choose from a set of distraction answers to complete sentences. It is especially useful for processes, and sequences of events where the order and relationships matter.
Matching: Study Guide creates matching tiles that ask learners to connect related ideas: terms to definitions, causes to effects, structures to functions, or concepts to examples.

Test: check what is sticking

Quiz: Study Guide creates a Microsoft Forms-powered quiz with questions generated from the materials. Learners can answer directly from the page, review results, and see explanations for multiple-choice answers. Results are private to the learner unless they choose to share them.

Every one of these formats is designed to move studying from passive review to active practice. Not just rereading or highlighting. Actually trying to remember, connect, explain, and check.

Study Guide supports 21 languages at launch: Arabic, Chinese (Simplified), Chinese (Traditional), Danish, Dutch, English (US), Estonian, French (Canada), French (France), German, Hebrew, Italian, Japanese, Korean, Norwegian Bokmal, Portuguese (Brazil), Spanish (Mexico), Spanish (Spain), Swedish, Ukrainian, and Vietnamese.

That means students can study from the materials they already use, in the language they already learn in, without having to move everything into a separate tool.

For educators

A few ways you can bring Study guide to your students or use it yourself:

Get started by taking the Microsoft Learn course. Available now at aka.ms/notebooksandstudyguidemodule
Point learners to a specific study moment. "Before Friday's quiz, add this week's slides and generate flashcards" is more useful than "use AI to study."
Encourage active practice. Flashcards, fill-in-the-blanks, matching, and quizzes help your students retrieve information from memory instead of only rereading it.
Use citations as an AI literacy moment. Study Guide shows where information came from. That opens a natural classroom conversation about checking sources, verifying AI-generated content, and staying grounded in the material.
Keep assessment separate from practice. Study Guide quizzes are for self-checking. They are not a gradebook, and quiz results are private unless a student chooses to share them.
Keep building your own AI fluency.

Study Guide is built with privacy, safety, and learner control in mind. Study Guide pages are private by default, stored in the learner's Microsoft 365 notebook, and can be edited or deleted by the learner. Prompts and outputs are not used to train AI models, and quiz results are private unless a learner chooses to share them.

Get started

Take the professional development course at aka.ms/notebooksandstudyguidemodule
Learn more about Study guide

Anoo Padte is Principal Product Manager for AI in Education at Microsoft.

Compliance Academy: A Multi-Agent Cyber Mystery Built on Microsoft Foundry Agent Service

lwhieldon — Wed, 10 Jun 2026 07:39:39 GMT

The Challenge

Microsoft Reactor invited three individuals to compete in a live coding battle: build something that shows what reasoning agents can do, in front of a streaming audience, in roughly twelve minutes of airtime each.

Watch live at 9am PT 10th June https://developer.microsoft.com/en-us/reactor/events/26942/

The temptation is always to default to a clean Q&A demo or a document summarizer. I wanted to try something harder.

The solution I will be building is Compliance Academy: a multi-agent cyber-mystery role-playing game where the player is the lead investigator at Helix Dynamics, a fictional biotech that just lost 14 GB of clinical trial data. Five suspects. One perpetrator. Multiple frameworks (SOC 2, HIPAA, ISO 27001) and the company's own policies are all in play. By the end of the case, the player has learned a real compliance lesson, grounded in real policy excerpts retrieved from a real search index.

The whole stack runs on Microsoft Foundry Agent Service, with Azure OpenAI for the model layer and Azure AI Search for retrieval. The source code is open: github.com/lwhieldon/msft-enterprise-learning-agent.

Why a Game?

Compliance training is something most professionals click through to satisfy a deadline. The content is dense, the stakes feel abstract, and the answers are usually "ask Legal." People learn just enough to pass the quiz, which is the wrong outcome for material that should change how they make decisions.

Reasoning agents let us flip the contract. Instead of pushing content at the learner, a multi-agent system can stage a scenario where the learner has agency. They ask questions because they need answers. They interrogate suspects who are evasive on purpose. They piece evidence together. When the Compliance Officer agent steps in at the end and cites SOC 2 CC6.1 alongside Helix Dynamics' own HD-SEC-AC-001 §4.1, the citation lands because the player just spent twenty minutes earning it.

That was the hypothesis worth testing on stream: reasoning agents are good enough now that we can make compliance training feel like a case file instead of a quiz.

The Architecture

Compliance Academy uses a Connected Agents pattern on Microsoft Foundry Agent Service. Four party agents are always present, providing broad expertise. A roster of suspect agents activates dynamically as the player interrogates them. The player is the orchestrator. They decide who to talk to and when.

The four party agents

Agent	Role	Backing
Game Master	Scene-setting, action menu, scene close	gpt-4.1-mini
Forensic Analyst	Evidence reasoning, log analysis, framework lookups	gpt-4.1-mini + Azure AI Search retrieval
Compliance Officer	Post-scene verdict and framework grounding	gpt-4.1-mini + Azure AI Search retrieval
Scenario Generator	Hot-loads brand-new cases from a one-sentence breach prompt	gpt-4.1-mini

The five suspect agents

Each suspect is a separate agent instance with a templated system prompt that loads from scenario JSON. Personas include an HR Director, an IT Administrator, a vendor account manager, an executive assistant, and a summer intern. Each carries a distinct backstory, alibi, conversational style, and a list of "leak conditions" that determine when they reveal guarded information.

The two surfaces

The audience-facing surface is a Chainlit UI, branded for SC&H Group, with click-through action buttons (Briefing, Suspects, Evidence, Generate, Accuse, Wrap) and a side panel that shows the retrieved policy snippets the agents are reasoning over.

The proof-of-orchestration surface is a live activity log streaming to a second terminal. Every Azure AI Search retrieval, every Azure OpenAI POST, every first-token latency, every source name and relevance score scrolls by in real time. This terminal is the show-don't-tell evidence that the agents are genuinely doing work, not just dressing up a single prompt.

Grounding Compliance Answers in Real Policy

The non-negotiable for compliance content is hallucination prevention. If the Forensic Analyst tells a learner that SOC 2 CC6.1 says something it doesn't, the entire training is worse than useless. So both the Forensic Analyst and the Compliance Officer ground their responses against an Azure AI Search index containing 52 chunked policy documents covering SOC 2, HIPAA, ISO 27001, NIST 800-53, and the fictional Helix Dynamics internal policy library.

The retrieval is straightforward:

def retrieve_context(query: str, top_k: int = 5) -> list[dict]:
    """Search the compliance knowledge index and return top-k snippets."""
    client = build_search_client()
    results = client.search(
        search_text=query.strip(),
        top=top_k,
        select=["uid", "snippet", "blob_url", "snippet_parent_id"],
    )
    return [
        {
            "source_url": r["blob_url"],
            "snippet": r["snippet"],
            "score": r["@search.score"],
        }
        for r in results
    ]

What makes it land in the demo is the per-source event logging. Each retrieval emits its filename and relevance score to the activity log:

[Foundry IQ]    Retrieved 5 sources in 1233ms
[Foundry IQ]      vendor_breach_response.md  (score=11.20)
[Foundry IQ]      helix_dynamics_overview.md  (score=7.65)
[Foundry IQ]      credential_compromise_response.md  (score=7.41)
[Azure OpenAI]  POST gpt-4.1-mini  (max_tokens=1500, temp=0.4)
[Azure OpenAI]  First token in 8328ms
[Azure OpenAI]  Stream complete: ~816 tokens in 14.9s

When the Forensic Analyst then cites HD-SEC-AC-001 §4.1 on MFA exceptions, the audience can see the specific document the citation came from. The trust loop closes on screen.

Live World-Building with the Scenario Generator

The piece I am most proud of technically is the Scenario Generator. Mid-demo, the host or an audience member can pitch a breach in one or two sentences. Roughly forty seconds later, a brand new scenario hot-loads into the game.

The agent emits structured JSON: a premise narration, five suspects with assigned roles and hidden truths, six to ten pieces of evidence, four to six violated controls, a clue graph, and a compliance lesson. The output then runs through a validation layer:

try:
    merged_scenario = load_scenario_from_dict(scenario_override)
except ScenarioValidationError as exc:
    if validation_attempt < max_validation_retries:
        # Feed the validation error back to the model as a corrective message
        user_message = _build_validation_retry_message(
            breach_description, exc
        )
        continue
    raise

If validation fails (wrong perpetrator count, missing canonical suspect, malformed evidence reference), the generator retries with the specific error fed back as a corrective message. Two validation cycles is usually enough. The session state then resets cleanly, the new briefing renders, and the player can start investigating immediately. Same agents. Brand new world.

Lessons Learned

A few takeaways I will carry to the next reasoning-agent project:

Ground anything compliance-related in retrieval, always. Even "general knowledge" policy citations are a hallucination risk. The Connected Agents pattern made it natural to wire retrieval into the two agents that needed it (Forensic Analyst, Compliance Officer) without complicating the others.
Plan the observability surface as a first-class deliverable. The activity log was not an afterthought; it was the audience's proof that the agents were real. For a live demo, observability is part of the user experience.
Validation with corrective retry beats perfect prompting. Let the model fix its own structured output errors when the schema permits.
Specialize and route. The Connected Agents pattern works well for clear role separation. Don't try to make one agent do everything; give each agent a tight remit and a clear handoff.
Keep a terminal-driven backup surface. When the live demo gods are unhappy, falling back to a CLI is a graceful recovery path. The CLI orchestrator and the Chainlit UI in this project share the same agent functions, so either surface tells the same story.

Why Microsoft Foundry Agent Service Was the Right Fit

Foundry Agent Service was the right home for this build because:

Connected Agents maps cleanly onto a party-of-agents game design.
Model Router gives one endpoint that picks the right backing model per agent without rebuilding clients.
Azure AI Search with agentic retrieval is first-class — no separate retrieval service to provision.
Azure Entra ID integration means the entire stack runs under enterprise SSO with no token juggling.
Streaming responses from Azure OpenAI deployments work cleanly with Python's async generators, which made the activity log timing events trivial to wire up.

The full stack — Foundry Agent Service, Azure OpenAI, Azure AI Search, and Azure Storage — sits inside a single resource group, which made teardown between dry runs trivial.

Join and watch live to learn more how we tackle the Agents League Challenges https://developer.microsoft.com/en-us/reactor/events/26942/

Try It Yourself

The repository is open source. Clone it and follow the README:

git clone https://github.com/lwhieldon/msft-enterprise-learning-agent.git
cd msft-enterprise-learning-agent
python -m venv .venv
.\.venv\Scripts\activate
pip install -r requirements.txt
chainlit run app.py -w

Three pre-built scenarios ship with the repo (Default, Supply Chain, Vishing). The Generate button creates new ones live.

If you build on top of this, want to compare notes on multi-agent design, or want to swap reasoning-agent war stories, I am @Lwhieldon on GitHub and Lee Whieldon on LinkedIn.

Thanks to Lee Stott and Carlotta Castelluccio at Microsoft for hosting the Reactor battle and giving these projects a stage.

About the author. Lee Whieldon is a Principal at SC&H Group, leading the Data Analytics & AI advisory practice. She works at the intersection of structured data, reasoning agents, and enterprise delivery.

Make Your Copilot Credits Count: A Student's Guide to Smarter AI Usage

Lee_Stott — Tue, 09 Jun 2026 07:40:42 GMT

If you're a student enrolled in GitHub Education, you already have something most developers pay for: free access to GitHub Copilot and its premium features. That's incredible. But here's the thing, free access doesn't mean unlimited usage, and not all AI interactions cost the same. Every chat message, every agent task, every model call consumes something called AI Credits, and knowing how they work will help you use Copilot smarter, produce better code, and build the kind of disciplined AI habits that professional developers are only just starting to learn.

This post is inspired by a fantastic deep-dive from my collegaue developer advocate Bruno: "GitHub Copilot and Tokens: How to Keep Using AI Without Burning Your Budget" . We've taken those professional lessons and tailored them specifically for students because your learning environment, your assignments, and your goals are different from a seasoned engineer at a tech company.

TL;DR: Use autocomplete before chat. Choose the right model. Keep context small. Start fresh chats often. Plan before you build. These habits will make you a better developer and stretch your credits further.

What Are AI Credits and Why Do They Matter?

When you interact with GitHub Copilot through chat, agent mode, or inline edits the model processes tokens. Tokens are small chunks of text (roughly 3–4 characters each). Every interaction consumes:

Input tokens — everything sent to the model (your message, attached files, chat history, instructions)
Output tokens — everything the model generates back to you
Cached tokens — context the model reuses from previous turns (cheaper)

These tokens are converted to AI Credits, where 1 AI Credit = $0.01 USD. Different models have very different token costs a lightweight model like GPT-5 mini charges $0.25 per million input tokens, while a powerful model like GPT-5.5 charges $5.00 per million input tokens (20x more expensive). Using the wrong model for a simple task is like taking a taxi to a destination that's a 5-minute walk.

See the official pricing table: GitHub Copilot Models and Pricing .

Figure 1: The four cost tiers of Copilot interactions. Autocomplete and Next Edit Suggestions are free — they do not consume AI Credits on paid plans

Strategy 1: Tab Before Chat The Free Tier is Powerful

Here is the single most impactful habit you can build: always try autocomplete before opening chat.

According to GitHub's official billing documentation, code completions and Next Edit Suggestions are not billed as AI Credits on paid plans. That means every time you press Tab to accept an inline suggestion, you are getting AI assistance for free.

Use autocomplete (Tab) for:

Completing a line or a simple function
Generating repetitive boilerplate (constructors, properties, getters/setters)
Completing a repeated pattern you've started
Writing obvious next lines like console.log, imports, or variable declarations
Adjusting variable names inline

Only move to Inline Edit (Ctrl+I / Cmd+I) when autocomplete isn't enough for a local change. Only open a Chat window when you need genuine reasoning an explanation, a plan, or a multi-step solution.

As Bruno puts it: "The most expensive model in the world should not be helping you write public string Name { get; set; }. That's what Tab is for. And coffee."

Strategy 2: Choose the Right Model for the Job

GitHub Copilot gives you access to models from OpenAI, Anthropic, and Google each at different price points and capability levels. The key insight from VS Code's official Copilot usage guide is: reserve powerful reasoning models for tasks that genuinely need them.

Your Task	Recommended Model Tier	Example Models
Simple question or boilerplate	Lightweight	GPT-5 mini, Gemini 3 Flash
Code explanation or basic docs	Lightweight	GPT-5 mini, GPT-5.4 nano
Writing tests or debugging a single function	Medium / Versatile	Claude Haiku 4.5, GPT-5.4
Multi-file refactor or code review	Medium / Versatile	Claude Sonnet 4.6, GPT-5.4
Complex system design or architecture	Powerful	Claude Opus 4.7, GPT-5.5
Long agentic workflows	Powerful (scoped!)	Claude Opus 4.8, GPT-5.5
Not sure what you need	Auto (recommended default)	Copilot selects for you

GitHub Copilot's Auto Model Selection feature automatically chooses a model based on task complexity, availability, and policies. For most students, Auto should be your default only switch manually when you have a specific reason. And when the complex task is done, switch back to Auto or a lighter model.

Strategy 3: Context is Currency Smaller is Smarter

Here's the counterintuitive truth that surprises most developers: the expensive part of a prompt is usually not the question you type it's everything surrounding it.

Every token consumed by Copilot includes:

All your previous chat messages in the session
Every file you have open or attached
Workspace search results Copilot pulled in
Build output, terminal logs, or diff content
Responses from any MCP (Model Context Protocol) servers you have enabled
Your custom instructions file (.github/copilot-instructions.md)

A single question inside a conversation with 80 messages, 12 open files, and 3 tool call results can cost significantly more than the same question asked fresh in a new chat with one relevant file attached.

Figure 2: The same task asked two ways. Scope your prompts to save credits and often get better answers.

Practical rules for context management:

Attach only 2–3 relevant files — not your entire project
Don't ask Copilot to analyse the whole repo when you only need changes in one module
Paste only the first relevant error from a log, not 2,000 lines of output
Remove timestamps and duplicate stack traces from pasted logs
State the expected output format explicitly so the model stops early
Use /compact in VS Code Chat to summarise a long conversation without losing key context
Use /fork to explore an alternative direction without polluting the main conversation

Strategy 4: Start Fresh Chats When You Change Tasks

This is one of the simplest optimisations and one of the most ignored. The VS Code Copilot usage guide is explicit about it: when a conversation grows, it carries context from all previous messages. If you switch to an unrelated task in the same session, the model still processes that irrelevant history and you pay for it in credits.

Bad pattern:

Chat session:
  - "Help me fix the JWT bug in auth.ts"   [10 messages]
  - "Now write unit tests for my sorting algorithm"  [still in same chat!]
  - "Can you generate the README for my project?"    [still in same chat!]
  - "Now debug this CSS layout issue..."             [still in same chat!]

Smart pattern:

Chat 1: "Fix JWT bug in auth.ts" - DONE, close chat.
Chat 2: "Write unit tests for sorting algorithm" - DONE, close chat.
Chat 3: "Generate README for project" - fresh context, fresh cost.

New task = new chat. Your human brain benefits too — focused sessions produce better outcomes than sprawling multi-topic conversations.

Strategy 5: Plan Before You Build Use Agent Mode Wisely

Agent mode is one of the most powerful Copilot features for students working on larger assignments — it can create files, run terminal commands, edit across multiple files, and execute tests. But agent mode also carries the highest token cost, because it loops: it plans, acts, observes tool output, then plans again.

The VS Code documentation recommends separating planning from implementation to reduce rework and back-and-forth. Here's a phased approach that saves credits and produces better results:

Figure 3: The credit-smart workflow. Always try the cheaper option first, escalate only when needed.

Phase 1: Plan (lightweight model, low cost)

I need to add user authentication to my Express app.
Before writing any code, give me a step-by-step plan
covering which files to create, which packages to install,
and what tests to write. Do not write code yet.

Phase 2: Scoped Implementation (one feature at a time)

Using the plan we agreed, implement only Step 1:
create src/middleware/auth.ts with JWT validation.
Do not modify any other files yet.

Phase 3: Validate

Run the existing tests in tests/auth.test.ts
and report the results. Fix only test failures
related to the new auth middleware.

Phase 4: Cleanup

The implementation is complete.
Update README.md with setup instructions for the auth module.
Keep it under 200 words.

Each phase is small, scoped, and verifiable. You can stop at any phase, check the result, and only continue when you're satisfied. This dramatically reduces expensive re-runs where the agent reverses its own changes.

Strategy 6: Review Your MCP Servers and Custom Instructions

MCP Servers

MCP (Model Context Protocol) servers let Copilot connect to external tools databases, GitHub issues, Jira, Slack, browser automation, and more. Each enabled server expands what the agent can do, but also adds to the context the model must consider, which increases token usage.

For students, a practical rule: only enable MCP servers relevant to your current project. If you're working on a simple Python web app, you probably don't need browser automation, a Kubernetes connector, and a Slack integration all active at the same time.

See the VS Code MCP servers documentation for how to enable, disable, and configure them.

Custom Instructions

A .github/copilot-instructions.md file in your repository lets you give Copilot standing instructions — coding standards, testing commands, architecture conventions. This is a fantastic feature. But that file is included in every prompt's context, so a bloated instructions file costs credits on every single interaction.

A good custom instructions file is:

Short — under 200 words for a student project
Specific to this repository's real conventions
Clear about test commands (e.g., npm test, pytest)
Free of generic advice that applies to every codebase on earth

Example of a good student instructions file:

# Copilot Instructions for MyWebApp

Language: TypeScript (strict mode)
Framework: Express.js with Prisma ORM
Tests: Run with `npm test` (Jest)
Lint: Run with `npm run lint` (ESLint + Prettier)

Conventions:
- Use async/await, not callbacks
- Validate all request inputs with Zod
- Keep controllers thin; put logic in service files
- Write a test for every new public function

That's it. Short, actionable, and genuinely useful — not a 500-line manifesto.

Strategy 7: Use Traditional Tools First

AI is excellent for reasoning, explaining, planning, and connecting ideas. It is not the right tool for every job. Before reaching for Copilot chat, ask yourself whether a traditional tool can answer your question faster, cheaper, and more reliably:

Compiler / type-checker — to find type errors (TypeScript, mypy)
Linter — to find style and logic issues (ESLint, Pylint, Checkstyle)
Formatter — to fix formatting (Prettier, Black, gofmt)
Test runner — to confirm whether your code works (Jest, pytest, JUnit)
Debugger — to step through execution and inspect state
Docs / Stack Overflow — for well-documented APIs and common patterns

If your linter tells you there's a missing import, fix it directly — don't ask Copilot to analyse your code to find it. Let deterministic tools do deterministic work, and let AI do the reasoning where it genuinely adds value.

Your GitHub Education Benefits: What You Get

If you haven't already, apply for GitHub Education with your school email address. Once verified, you receive:

Free GitHub Copilot including premium features — see how to enable Copilot as a student
Free GitHub Codespaces — 180 core hours per month, equivalent to GitHub Pro (great for browser-based coding with Copilot built in)
GitHub Student Developer Pack — free access to dozens of professional tools from GitHub's partners, including cloud credits, domains, and IDEs
GitHub Classroom — your instructors can manage assignments and provide feedback
GitHub Community Exchange — discover and contribute to student-built projects
Campus Experts program — become a student leader in your tech community

These benefits are designed to give you real-world tools in an educational setting. Copilot is the standout feature — it's the same tool professional developers use every day. Using it wisely during your studies means you'll arrive in the workforce already ahead of the curve.

Pre-Prompt Checklist for Students

Before you fire off your next Copilot prompt, run through this checklist. It takes 10 seconds and can save significant credits — and more importantly, it builds the mental habits of a professional AI user.

Figure 4: Two-column checklist covering what to check before opening chat and when writing your prompt.

Before you open chat:

☐ Can Tab / autocomplete solve this?
☐ Is inline edit (Ctrl+I) enough for this local change?
☐ Can a linter, compiler, or test runner answer this?
☐ Is this a different task from my last message? If so, start a new chat.
☐ Am I on Auto model selection (or the right tier for this task)?
☐ Should I ask for a plan before asking for code?
☐ Do I have MCP servers enabled that I don't need right now?
☐ Is my copilot-instructions.md file concise and current?

When writing your prompt:

☐ Attach only 2–3 relevant files, not the whole project
☐ Paste only the first relevant error from any logs
☐ Define the files to change, the goal, and any files not to touch
☐ Ask for a plan before implementation on complex tasks
☐ Remove timestamps and duplicate stack traces from pasted logs
☐ State the expected output format and length
☐ Use /compact if the session is getting long
☐ Use /fork to explore alternatives without polluting the main thread

A Note on Responsible AI Use in Education

Using Copilot smartly is not just about saving credits it's about developing genuine skills. When you ask Copilot to write all your code without understanding it, you lose the learning opportunity the assignment was designed to create. When you review and understand every suggestion Copilot makes, you learn faster, build better instincts, and can confidently explain your own work.

Best practices for academic integrity with AI tools:

Understand before you accept — never paste code you can't explain
Use Copilot to learn, not to skip learning — ask it to explain the code it generates
Follow your institution's AI policy — many universities have specific guidance on AI use in assessments
Treat Copilot as a senior pair-programmer, not an answer machine — question its suggestions, push back, iterate
Verify facts and documentation links — AI can hallucinate; always check official sources

GitHub Education exists to give you real professional tools while you learn. The goal is for you to graduate with genuine skills, a real portfolio, and the confidence that comes from building things yourself — with AI as your collaborator, not your ghostwriter.

Key Takeaways

Tab first — autocomplete and Next Edit Suggestions are free; use them for everything small
Auto model by default — only switch to a powerful model when you have a clear reason
Context is cost — fewer files, fewer messages, fewer tools = fewer tokens
New task = new chat — don't carry stale context into unrelated work
Plan before you build — a 10-message plan session is cheaper than 50 messages of rework
Keep instructions short — your copilot-instructions.md runs on every prompt
Use traditional tools first — linters and compilers are free, fast, and deterministic
Understand your code — Copilot is a collaborator, not a replacement for learning

Resources and Next Steps

GitHub Education — apply for your free student benefits
GitHub Student Developer Pack — explore free tools for students
Enable GitHub Copilot as a student
GitHub Copilot: Models and Pricing — understand exactly what each model costs
Auto Model Selection in GitHub Copilot
VS Code: Optimising GitHub Copilot Usage — the official guide that inspired many of these tips
Managing MCP Servers in VS Code
El Bruno: GitHub Copilot and Tokens (the original professional perspective)
GitHub Education Community Discussions — connect with students and educators worldwide

This post draws on insights from El Bruno's developer blog and best practices from GitHub Education. All pricing figures are sourced from the official GitHub Copilot billing documentation and are correct as of June 2026.

Communication with parents in Teams / Teams Communitu

FmGonzalez — Wed, 03 Jun 2026 18:14:29 GMT

We have teams' chats set up with families in Teams Education but for some reason parents (I.e external ) are being removed by ??????

This is causing a lot of issues with messages between sch and home

Can you help.

Announcing the 2026 Imagine Cup World Champion

StudentDeveloperTeam — Tue, 02 Jun 2026 16:31:19 GMT

On June 2 at Microsoft Build, student founders from around the world watched as this year’s winner was revealed live on stage. Hosted by Dona Sarkar of Microsoft Enterprise AI Advocacy, the Imagine Cup World Championship brought together a global audience of developers, founders, investors, and technology leaders as Hans Yang, Vice President of Microsoft for Startups and Microsoft Learn Programs, announced the 2026 Imagine Cup World Champion. The final three startups showcased bold solutions to meaningful problems through AI.

Built by Patrick Brown from the University of Oxford, CopyFlag enables creators to protect their original work in the age of generative AI. Using Azure-powered AI models, the platform detects copied and AI-modified content across the internet, helping creators identify infringement faster and automate takedowns at scale.

The winning startup received $150,000 USD, a mentorship session with Microsoft Chairman and CEO Satya Nadella, and the opportunity to continue growing through Microsoft for Startups. In addition, the World Champion received an additional 6 months and another $5,000 USD in credits from Replit to continue building what comes next.

But the World Championship was never just about one moment on stage. It represented months of iteration, collaboration, learning, and resilience from student founders building solutions with real-world impact.

This year’s finalists showed how students are applying AI across industries including creator protection, healthcare, and supply chain intelligence; not just imagining what technology could become, but actively building it.

Meet the 2026 Imagine Cup World Champion: CopyFlag

CopyFlag is redefining how creators protect their work online. As AI-generated and modified content continues to scale, creators often lack the tools to identify where original work is being copied, altered, or redistributed.

CopyFlag uses Azure-based AI models to detect both direct copies and AI-modified derivatives, helping creators trace misuse across platforms and automate enforcement workflows in minutes.

What began through Patrick’s personal experience of having his own work copied online evolved into a mission to make advanced protection tools accessible beyond the world’s largest brands and enterprises.

By combining AI, pattern recognition, and scalable detection systems, CopyFlag is building toward a future where creators can focus more on creating and less on defending their work.

Celebrating This Year’s World Finalists

Revora Health

Carnegie Mellon University - United States 🇺🇸

Founded by Surya Kukkapalli, Revora Health is reimagining physical therapy and recovery through AI-guided care. The platform connects patients to licensed providers in minutes while delivering real-time multimodal support powered through Azure Health Data Services and AI-driven computer vision.

Built from Surya’s firsthand experience working with clients navigating injury recovery, Revora Health focuses on making specialized care more accessible, connected, and continuous beyond the clinic setting.

SpoilSafe

Carnegie Mellon University - United States 🇺🇸

Built by Advika Vuppala, Rohan Ganesh, and Troy McBride; SpoilSafe is bringing predictive intelligence into the food supply chain.

Using sensor data processed through Azure IoT Hub, Azure Data Explorer, Azure Synapse Analytics, and Azure AI Foundry, SpoilSafe helps teams estimate freshness in real time, enabling smarter operational decisions and reducing food waste before spoilage occurs.

Their work represents a shift from reactive decision-making toward more resilient, data-driven systems across the global food ecosystem.

More Than a Competition

Imagine Cup is a global stage for student founders building commercially viable startups on Microsoft’s cloud and AI platforms.

From mentorship and technical skilling to investor visibility and startup support, the program helps students move from ideas to scalable businesses while learning how modern teams build, iterate, and grow. Through Microsoft for Startups, founders continue receiving the tools, support, and opportunities needed to keep building beyond the competition.

The future is being built by students bold enough to solve problems that are good for business and good for the world. And this year’s finalists proved what’s possible when great ideas meet the right mentorship, technology, and momentum.

Think you could be next? The 2027 Imagine Cup is officially underway. Use this summer to build your MVP, refine your idea, and explore what it takes to turn your startup into something bigger. Learn more and get started today.

Evaluating the Evaluator: How to Test an LLM Judge with Microsoft Agent Framework

Abdulhamid_Onawole — Tue, 02 Jun 2026 13:09:41 GMT

The four verdicts, up front

```
Consistency: mean CV across posts 5.30%
```

Pipeline format checks: pipeline pass rate 100%

Rubric adherence (strict judge): 5.00 / 5, mean math drift 0.05 pts

Calibration vs. labels: Pearson r = 0.51, MAE = 22.9 pts

Three of those say the model is healthy. The last one is the only one that compared the model against anything real, and it tells a different story.

Where we left off

In Post 1 I built Viral or Fail, three Microsoft Agent Framework agents that pressure-test a gaming social post before you publish it. A Content Creator drafts the post, an Algorithm Simulator scores it the way a recommendation system might, and an Audience Persona reacts the way an actual viewer would. The whole thing runs on the GitHub Models free tier, with no paid API keys.

That post ended on a cliffhanger I left deliberately open. The Algorithm Simulator scored the post 75/100, but how do I know the Simulator itself is any good? How consistent are its scores? Do they track real engagement? Would a human social strategist agree with its rubric weights?

This post answers that empirically. I built four tests: consistency, pipeline format checks, rubric adherence, and calibration. Three came back healthy. The fourth caught a problem structural enough that it changed how I think about evaluating LLM judges in general.

The surprising part isn't that the model failed somewhere but that it passed the three tests you naturally reach for first, and only failed the one most will skip.

Why I built my own harness

The Microsoft Agent Framework ships a real evaluation surface. You get evaluate_agent, LocalEvaluator, an @evaluator decorator, and the EvalItem / EvalResults data types. It's well designed, and for production agents it's the right choice.

It also pairs most naturally with Azure AI Foundry. The path of least resistance assumes you already have an Azure project, a model deployment, and the budget for cloud-tier LLM-as-judge calls. Post 1 went the other way on purpose: zero paid keys, GitHub Models free tier only. To keep that footing, I wrote a small in-house harness that mirrors the call shape of evaluate_agent.

The framework's evaluation surface is provider-agnostic in principle, but it leans toward Azure in practice. What the SDK hands you for free on Azure, you can rebuild for yourself on GitHub Models in as you would see shortly, and the patterns transfer directly when you upgrade.

The harness is one file, roughly 150 lines. The trick that makes it more than a wrapper is that it tries to import the SDK's primitives first and only defines its own if they aren't there yet:

try: from agent_framework import EvalItem, EvalResults, evaluator _USING_SDK_PRIMITIVES = True except ImportError: # agent-framework-core==1.0.0rc1 doesn't ship these yet, # so we define local equivalents with the same shape. @dataclass class EvalItem: query: str response: str expected_output: str | None = None scores: dict[str, float] = field(default_factory=dict) repetition: int = 0 # ... EvalResults, evaluator defined the same way

The day Microsoft ships these types, the suite picks them up with no code change. An evaluator looks like this:

@evaluator def correlates_with_truth(response: str, expected_output: str) -> float: sim = parse_weighted_total(response) if sim is None or expected_output is None: return 0.0 truth = float(expected_output) return 1.0 - (abs(sim - truth) / 100.0)

If you've used the SDK's @evaluator, you've used this one. Same parameter-name dispatch (query, response, expected_output), same return convention (a float in [0, 1]). The runner wraps a retry-aware async loop around a list of these. GitHub Models caps this model at about 15 requests per minute, so the loop sleeps 4.5 seconds between calls (12 a minute, comfortably under the cap). When it does hit a 429 it waits 30 seconds and up, rather than the short exponential backoff it uses for ordinary transient failures. Boring glue code, and important glue code.

When you eventually move to Azure, you swap runner.run(...) for evaluate_agent(...) and nothing else in your codebase has to change.

What 'good' even means for a judge

Before running anything, it's worth being precise about what "good" even means for a judge agent. There are four versions of it, and they split into two camps.

The first three are process checks. They probe the model against itself. No external reference data, just the model and its own outputs.

Consistency means same input, same output. Run the Simulator twice on the same post and the scores should land in roughly the same place. If they don't, the score is noise.

Pipeline format checks ask whether each agent followed its required output shape. Did the Creator produce platform-native text? Did the Simulator emit a parseable weighted total? Did the Persona stay in character? These are the cheapest tests of all, just regex and keyword matching, no LLM judge needed.

Rubric adherence is harder. The Simulator's prompt asks it to score five weighted criteria and report a weighted total. Did it actually do that, or did it list the criteria and then invent a number? Checking this needs an LLM. The cloud-tier equivalent is FoundryEvals.TaskAdherence, and I'll build the free-tier version below.

The fourth check is a different animal. Validation against ground truth. Calibration asks whether the Simulator's scores correlate with real engagement. It's the same operation you'd run on any predictive model: predict, compare against a labeled set, report correlation and error. It's the only check that tells you the model is correct rather than merely consistent and well-formatted, and it's the only one that needs data the model didn't produce.

That's the thesis of this post, and the reason the order matters. The three process checks can all come back green and still tell you nothing about the validation result. And because validation needs ground truth, the design of the ground-truth dataset becomes part of the result. I'll be explicit about that when we get there.

The posts under test

Every test runs against the same thing: a 10-post golden dataset of gaming social posts I wrote and hand-labeled. Each entry carries the post content, its real-world engagement numbers, a normalized engagement_score from 0 to 100, and a label (viral, decent, flop, or outlier). Here's the viral Valorant post that Test 1 keeps referencing, in full:

json

{ "id": "post_001", "platform": "Twitter/X", "topic": "Valorant Champions 2025", "content": "EG winning Champions 2025 was the most underrated moment in Valorant esports history and people still don't talk about it enough.\n\nDemon1 carried that grand final on a level we won't see again until at least Champions '26. The map veto into Bind alone deserves a documentary.", "real_engagement": { "impressions": 2100000, "likes": 45000, "shares_or_retweets": 8000, "replies_or_comments": 1200, "engagement_rate_pct": 2.58 }, "engagement_score": 82, "label": "viral", "notes": "Hot take + esports nostalgia + named callout (Demon1) drove QRTs from competing fanbases." }

The full set is in the repo at evals/golden_dataset.json: two viral hits, four decent posts, three flops, and one outlier, across Twitter/X, TikTok, YouTube, and Instagram.

Test 1: Consistency

The easiest test to write. Run the Simulator ten times on the same post with identical input. Compute the mean, standard deviation, and coefficient of variation. Repeat across five posts spanning viral, decent, flop, and outlier labels.

The harness call is one line:

runner = EvalRunner(rate_limit_sleep=4.5) # 12 RPM, under the cap results = await runner.run( agent=agent, queries=[_build_simulator_prompt(p) for p in selected], evaluators=[weighted_total_score], # parses the score out of each run num_repetitions=NUM_REPETITIONS, # 10 )

Fifty Simulator runs in total. Group by query, compute std/mean per post, then average the resulting CVs.

Mean coefficient of variation across the five posts: 5.30%. With the rubric pinned, the Simulator is meaningfully non-deterministic, but it isn't chaotic. Most scores cluster within about four points of the per-post mean.

That's the headline, and it's fine.

Now look at the chart again. post_001 (the viral Valorant Champions post, mean 70.3) and post_003 (the decent Steam Deck OLED post, mean 72.4) sit at almost the same place on the x-axis. The decent post averages slightly higher than the viral one. Across ten reps each. Twenty data points, and the Simulator can't reliably tell which one is supposed to be the success. If you trace the mean diamonds left to right, the decent post outranks both viral posts.

A consistency test won't flag this as a problem, because the Simulator is being consistent. It consistently rates these two posts in the same band. The problem is what that band means. If consistency were your only check, you'd close the laptop and ship.

Hold onto that. It comes back.

Test 4: Pipeline format checks

Now zoom out from a single agent and run the full Viral or Fail pipeline (Creator, then Simulator, then Persona) on five live trending gaming topics, applying format-level checks to each agent's output.

The checks are deliberately cheap. For the Creator: does the output contain Twitter/X-native vocabulary (the keyword list looks for things like thread, ratio, QRT, take, based)? For the Simulator: is there a parseable weighted total between 0 and 100? For the Persona (TryHard_Tyler, the competitive esports fan, in this run): does the output use any of the persona's keywords, like diff, cope, goated, ratio, cap?

Five topics, fetched live from Google Trends: xbox game pass, the hobbit mtg collector booster, crimson desert patch notes, xbox, olden era steam.

Per-agent pass rate: Creator 100%, Simulator 100%, Persona 100%. Pipeline pass rate 100%.

The format checks are doing their job. Every agent produced output in the shape it was supposed to, on every topic. No regex misses, no missing weighted totals, no out-of-character personas.

This is the point where, if you'd only run consistency and pipeline checks, you'd write the triumphant report. "Our agents are reliable. CV under 6%. Pipeline pass rate 100%. Ship it." That report would be true. It would also be wrong about whether the model is correct, because format adherence is not output validity. Keep going.

Test 3: Rubric adherence, and a free-tier LLM-as-judge

Format checks tell you what the output looks like. Rubric adherence asks whether the Simulator actually did the work it was prompted to do: score five weighted criteria, sum them correctly, and explain each score with platform-mechanic reasoning rather than vibes.

There's no regex for that. You need an LLM to read the Simulator's full evaluation and judge whether it followed its own rubric. That's an LLM-as-judge, and the cloud-tier equivalent is FoundryEvals.TaskAdherence on Azure. Since we're staying free, I built it.

The judge is just another Agent with a stricter system prompt:

JUDGE_SYSTEM_PROMPT = """You are a Rubric Adherence Judge — strict and skeptical. You evaluate whether another AI agent ACTUALLY followed its scoring rubric, not just whether it produced output that looks like it did. You will check three things, in order of severity (the strictest failing check sets the score): A. MATHEMATICAL FIDELITY (most important). Compute sum(criterion_score × weight) yourself from the agent's per-criterion scores. Compare it to the agent's stated WEIGHTED TOTAL. If they differ by more than 2 points, the agent is doing the rubric wrong even if it looks correct on the surface. Report the difference as `math_diff`. B. REASONING SPECIFICITY. Each criterion's justification must reference platform-specific algorithm mechanics — "FYP retention threshold", "QRT velocity", "average view duration". Generic praise ("strong hook", "good engagement") is GENERIC and lowers the score. Classify reasoning as "specific", "mixed", or "generic". C. COVERAGE. Every criterion in the rubric must be explicitly scored. Missing criteria fail this check. ... Be strict. Format-following ≠ rubric-following."""

The full prompt is in the repo. The key decision is point A: the judge recomputes the math itself. That catches the failure where an agent lists every criterion with a score, but the weighted total it reports doesn't actually equal the weighted sum. That kind of quiet drift is exactly what format checks miss.

The judge returns strict JSON: adherence_score (1 to 5), math_diff, reasoning_quality, criteria_present, missing_criteria, weight_drift, plus a few sentences of reasoning. Test 3 doesn't go through runner.run; it orchestrates the two agents by hand, one post at a time, so the judge sees the Simulator's full evaluation:

for post in posts: sim_text = await call_agent_with_retry(simulator, build_simulator_prompt(post)) verdict = await judge.judge( rubric=PLATFORM_RULES[post["platform"]], post_content=post["content"], evaluation_output=sim_text, )

Run across all 10 posts in the golden dataset, here's what comes back.

Mean adherence score: 5.00 / 5. Mean absolute math drift: 0.05 points (max 0.25). Reasoning quality classified "specific" on 100% of evaluations. Zero missing criteria, zero weight drift.

This was not the result I expected. I built the judge to be strict on purpose, after my first version turned out too lenient (more on that in the bugs section). The strict version recomputes the weighted sum, classifies generic praise as a failure, and demands platform-mechanic citations. The Simulator passed every dimension anyway.

The per-post reasoning is genuinely fun to read. On the Activision Blizzard flop, the judge noted that the Simulator's reasoning leaned on engagement velocity, quote-retweet incentives, topicality timing, and hashtag discoverability rather than generic praise. On the GTA 6 viral TikTok, it cited pattern interrupts, trending-cluster signals, and share-velocity drivers. That's the language I asked for, and the Simulator is producing it.

So the Simulator does the rubric correctly. The math is right, the reasoning is specific, every criterion is covered. By every internal measure, it works.

You can probably see where this is going. There's exactly one thing left to check, and it's the most expensive and most important one.

Test 2: Calibration, the reckoning

This one isn't a test in the same sense as the first three. They asked whether the model was malfunctioning. This asks whether it's correct, which is a different question entirely, because it's the only one that needs data the model didn't produce.

And because it's a validation, what I validate against matters as much as the model. So before running anything, here's exactly what the ground truth is: a 10-post golden dataset that I built, not measured. I wrote the post content myself in platform-native style, then assigned each post an engagement_score from a back-of-envelope formula (impressions x engagement rate x shareability), calibrated against publicly observable performance for similar posts. The set spans two viral hits, four decent posts, three flops, and one deliberate outlier (a post that got ratio'd into orbit, with high reach and terrible reception).

So when I show you a Pearson r in a moment, hold it loosely. The exact number is partly a function of how I designed the labels. The shape of the failure (whether the Simulator's predictions cluster, spread, invert, or track the labels) is what's actually informative, because the shape doesn't depend on the labels being precise. It only depends on them being roughly ordered: viral out-ranks decent, decent out-ranks flop. Whether viral is 91 against flop 18, or viral 85 against flop 25, doesn't change which way the comparison runs.

With that on the table: run the Simulator once per post, compute Pearson r and Spearman rho, compute MAE.

Pearson r = 0.51. Spearman rho = 0.52. MAE = 22.9 points.

That r-value isn't a small problem. Here's what it means in practice, post by post:

Post	Topic	Label	Truth	Simulator	Error
001	Valorant Champions 2025	viral	82	69.75	12.25
002	GTA 6 reveal reaction	viral	91	65.75	25.25
003	Steam Deck OLED price	decent	55	71.00	16.00
004	Genshin Impact 5.0 pulls	decent	48	65.00	17.00
005	Hollow Knight: Silksong	decent	60	76.50	16.50
006	Xbox Showcase 2025	decent	42	74.75	32.75
007	Activision Blizzard acquisition	flop	18	59.50	41.50
008	5 games to play this weekend	flop	22	37.00	15.00
009	Pentiment retrospective	flop	15	60.75	45.75
010	Concord shutdown post-mortem	outlier	50	57.00	7.00

The pattern is structural. The Simulator's natural output band is roughly 60 to 76. Posts that should clear 80 get pulled down to 65 to 70. Posts that should land below 25 get pulled up toward 60, with one flop (post_008, the "5 games this weekend" listicle) the only exception at 37. The model has an attractor zone in the middle of the scale and refuses to leave it.

Look at the most accurate prediction in the table. It's post_010, the outlier (truth 50, Simulator 57, error 7). Why is it the most accurate? Because 50 happens to sit inside the attractor zone. The Simulator's bias accidentally cancels out for posts that are supposed to be average. It isn't accurate, it's wrong in a way that lands near the truth for one specific case.

This was the test I almost didn't run. It needs labeled data, which is annoying to gather, and three out of four tests had already declared the model healthy. By every internal measure, the Simulator was working as designed.

It just couldn't tell viral from decent. It rated the GTA 6 reaction TikTok (truth 91) at 66, and the Steam Deck OLED post (truth 55) at 71. The model is consistent, rubric-faithful, and format-stable, and on real cases it literally inverts virality and decentness.

The shape of that failure (flops pulled up hard, by 15, 42, and 46 points; virals pulled down by 12 and 25; the whole range collapsed into a narrow band) is what survives the synthetic-label uncertainty. If the labels were simply inaccurate, you'd see scatter. A symmetric squeeze toward the middle requires the Simulator itself to be conservative. The Pearson r of 0.51 (p around 0.13, not significant on n = 10) is the number to hold loosely. The squeeze is the result. Running this against measured engagement metrics is the natural Post 3, and I'd expect the qualitative finding to hold.

Bugs the suite caught along the way

This is something I want to keep doing in my write-ups. I usually publish the clean, glamorous version (here's what I built, here's what I learned, the end), which quietly erases the bugs that taught me the most about how the system actually behaves. So here are three real ones the eval suite caught while I was running it.

The production parser regex was silently failing. Post 1's viral_or_fail.py extracts the Simulator's weighted total with a regex like Weighted\s*Total[^\n]+. That works for same-line layouts (Weighted Total: 73/100). It does not work for the multi-line layout the model produces about half the time:

**WEIGHTED TOTAL:** 

= 22.5 + 15 + 14 + 12.75 + 9 =

 **73.25/100**

When the regex misses, the production code silently falls back to a default of 50. Which means the public Viral or Fail demo had been quietly showing readers 50/100 on many of its runs since Post 1 went live. The eval suite caught it on the very first call: parse_weighted_total returned None, the harness logged it loudly, and the bug had nowhere to hide. The fix strips the bold markers, finds the header, then scans a few non-blank lines past it, preferring N/100, then a trailing = N, then the first number it sees:

clean = response.replace("**", "") header = _WT_HEADER_RE.search(clean) # r"Weighted\s*Total\s*:?" if not header: return None after = clean[header.end():] window = [] for raw in after.splitlines()[: _WT_LOOKAHEAD_LINES + 1]: line = raw.strip() if not line and window: break if line: window.append(line) blob = " ".join(window) # prefer "N/100", then a trailing "= N", then the first number found

That regex hunt alone justified the whole exercise.

The Google Trends "Games" topic is contaminated. Test 4 originally fetched live trending topics and got back "kentucky derby 2026", "kentucky oaks", and "fanduel" alongside the actual gaming. The cause: Google's taxonomy bundles horse racing, gambling, and sportsbooks under the same Games topic ID it uses for video games, and the trends_tool.py filter from Post 1 was matching on that topic ID alone. The fix was a two-layer filter: require the games topic and not topic 17 (Sports), plus a small denylist for gambling keywords. Now the results come back as xbox game pass, crimson desert, and the hobbit mtg collector booster, with no horse racing.

The first version of the judge was too lenient. My initial RubricAdherenceJudge rewarded "every criterion explicitly scored." But the Simulator's system prompt forces exactly that, so the judge handed out 5/5 trivially across all 10 posts and told me nothing. I tightened it to recompute the weighted sum and report math_diff, and to classify reasoning as specific, mixed, or generic based on whether justifications cite platform mechanics. Even under the strict judge the Simulator still scored 5/5, but now I'd earned that result instead of getting it for free.

Why this matters in production

I built four tests to evaluate the Algorithm Simulator from Post 1. Three of them (consistency, rubric adherence, pipeline format checks) declared it healthy. The fourth, calibration, compared its scores against labeled engagement and found systematic bias: the predictions are squashed into a narrow band regardless of how the post actually performed. A flop with engagement of 18 gets a 60. A viral hit with engagement of 91 gets a 66. The model isn't broken in any visible way. It's just consistently, faithfully, formally wrong. That's exactly why validation against ground truth isn't optional. It's the only check that catches a model doing everything right except being correct.

Format, consistency, and rubric-coverage tests tell you the model isn't malfunctioning. They cannot tell you it's correct. They test the process, and only validation tests the output. A model can have a flawless process and still produce numbers that don't track reality.

Now zoom out. Viral or Fail's Simulator is low stakes. Worst case, a creator publishes a post the Simulator liked and it flops. Embarrassing, not dangerous. The same failure at higher stakes is dangerous, and the same shape shows up everywhere in production AI. Ask a language model to be "objective" and it hedges toward the middle. Content moderation agents under-flag clearly harmful content and over-flag clearly benign content, because both extremes feel risky to the model. Resume screeners compress every candidate into a 60-to-80 band and call the lack of spread "fairness." Code-review bots return a comfortable 7/10 on a PR with real problems and on a PR with none. Support routing labels almost everything "medium priority" and quietly breaks the downstream automation that relied on the signal meaning something.

Each of those has shipped in real deployments and then underperformed for months before anyone noticed. The teams weren't careless. They had observability, CI, process checks. What they lacked was a labeled validation set. And without one, a confidently miscalibrated model looks identical to a working one. A model that's wrong randomly gets caught, because outliers get flagged and reviewed. A model that's wrong consistently gets trusted, because it never trips an alarm. Once a downstream product depends on the miscalibrated output, the bias gets amplified at scale.

Most production AI systems are not validated this way. Most LLM-as-judge components in agentic systems have never had their predictions compared against any external ground truth at all. And when something does feel off, teams reach for fine-tuning. But you can't fine-tune what you haven't characterized, and characterization is exactly what calibration testing produces. Without it, fine-tuning is guesswork in an engineering costume. "It works in eval" usually means it passed process checks, which is not the same thing as working.

So evaluation is a discipline, not a phase. It belongs in the same loop as deployment, not as a one-off before launch. Internal-process checks belong in CI. Validation against labels belongs on a schedule. Both should alert when they regress, and both should be visible to the people accountable for the model's decisions.

If there's one thing to take from this post: build a validation step into your eval suite from day one, even with synthetic labels, and especially if you can't get measured ones yet. Process tests keep you safe from regressions. Only the validation step keeps you honest about whether the model is right.

What's next: the cloud-tier upgrade path

Everything here runs on the GitHub Models free tier. That's deliberate, and it also means I've built the free-tier version of three things Microsoft already does better at production scale.

The first is FoundryEvals in agent_framework_azure_ai. My RubricAdherenceJudge is a homemade FoundryEvals.TaskAdherence. Foundry's version uses Azure-hosted judges on a managed pipeline, with calibration handled internally and a portal for tracking runs over time. Same structural test, but operationally serious. The same idea applies to Relevance, Coherence, Groundedness, IntentResolution, and the rest of the catalogue. If you've built the harness from this post, swapping it for evaluate_agent plus FoundryEvals is mostly an import change.

The second is the AI Red Teaming Agent. I didn't run any safety evaluation in this suite. The Audience Persona is the agent most likely to drift into unsafe territory, and the natural counterpart to quality evaluation is adversarial probing with PyRIT. The AI Red Teaming Agent wires that straight into Foundry. That's a Post 4.

The third is observability. DevUI gives you real-time visualization of agent sessions, and OpenTelemetry traces flow into Azure Monitor. Both earn their keep when an eval flags a regression and you need to walk back through the failing run to find the cause.

And then there's Post 3: the calibration test against real engagement data. If you have a Twitter, YouTube, or TikTok dataset with both post content and post-hoc engagement metrics, and you'd be open to collaborating, I'd love to hear from you.

The full eval suite is on GitHub: github.com/HamidOna/viral-or-fail. Run pip install -r requirements.txt, set GITHUB_TOKEN, and run python -m evals.run_all. Six to eight minutes start to finish on the free tier. The suite runs, the JSONs write, the plots render, and you'll see the same thing I did: the easy tests will tell you everything is fine.

The last test will tell you what's actually happening.

OneNote Future?

RobertoFoster1971 — Mon, 01 Jun 2026 17:26:26 GMT

MS has been putting out a lot of new apps, but OneNote seems to be left out to pasture. Are there any development roadmaps for OneNote? There are some other notetaking apps in the market: Notion, Obsidian, etc. If there are any MS OneNote devs on this thread, I'd be interesting if I should cut bait or keep my notes in OneNote.

Making Academic Standards More Accessible

MikeMast — Mon, 01 Jun 2026 14:01:25 GMT

Why standards matter

Academic standards are the shared language that connects curriculum, instruction, and assessment. When educators can easily access and apply them:

Lesson planning becomes more intentional. You design instruction around clear learning goals rather than guessing what to cover.
Assessment aligns with instruction. Quizzes, rubrics, and assignments reflect what students are actually expected to demonstrate.
AI-powered tools become more relevant. AI-generated content is grounded in real curriculum expectations, not generic suggestions.
Collaboration improves. Teachers across grade levels and departments can speak the same language about what students should know and do.

How Microsoft Education uses standards

Standards are woven into the experiences educators use every day.

In the Teach module and Microsoft 365 LTI, educators can align lesson plans to specific standards by location, subject, and grade band, use the "Align to Standards" tool to refine lesson instructions, and generate quizzes and rubrics grounded in standards.

In Assignments in Teams for Education and Microsoft 365 LTI, educators can tag assignments with curriculum expectations, build standards-aligned rubrics, and create a clear thread from instruction to assessment.

Across AI-powered workflows, standards can serve as grounding data that helps make generated lesson plans, quizzes, and rubrics more relevant to real curriculum expectations. This reflects Microsoft’s broader approach to AI in education: using AI to support educators with useful, contextual assistance while helping institutions maintain alignment with their instructional goals, policies, and professional judgment. Educators can select standards by location, subject, and language.

Expanding coverage through partnership with EdGate

Making standards useful in digital tools globally requires more than a large catalog. It requires structured, machine-readable data, ongoing maintenance, and a partner with deep expertise in education standards. EdGate has spent years building and maintaining one of the largest catalogs of digitized standards in education technology. Microsoft partnered with EdGate to help make that infrastructure more accessible inside the workflows educators and institutions already use.

Through this partnership, Microsoft has significantly expanded the set of standards EdGate offers, especially internationally. Together, we have grown coverage to include:

All 50 U.S. states, including Common Core, NGSS, and state-specific frameworks
70+ countries, with international standards covering core subjects, vocational education, and qualification frameworks
Hundreds of supplemental frameworks, from career and technical education to world languages and the arts

We continue to expand coverage with new international standards rolling out regularly.

EdGate offers access to over 5 million standard statements, aggregating and normalizing global standards for consistent delivery across platforms. Their capabilities include a comprehensive standards catalog, standards authoring tools used by ministries of education, API-based access for platform integration, and certified CASE 1.1 compliance.

Microsoft and EdGate are partnering to make a select set of standards freely available to education institutions, lowering barriers for educators and developers who want to explore standards-aligned workflows without a commercial commitment. To expand the impact even further, EdGate is piloting a project in 1EdTech's CASE Global Ecosystem initiative, to demonstrate how interoperable, machine-readable frameworks can improve the discoverability, alignment, and portability of learning and credentialing data across platforms, institutions and borders.

The CASE format: Why it matters

CASE stands for Competencies and Academic Standards Exchange, an open standard from 1EdTech that defines how learning outcomes and standards are represented in a machine-readable, interoperable format.

Why does CASE matter?

Machine-readability: Platforms, AI tools, and learning management systems can read, search, and apply standards programmatically.
Interoperability: Standards move between systems. An assignment tagged with a standard in Microsoft Teams can be understood by an LMS, a reporting tool, or a curriculum mapping platform without manual re-entry.
Cross-region equivalence: CASE enables comparing and mapping standards across countries and frameworks.

EdGate is a certified CASE 1.1 provider, meaning the standards they deliver to Microsoft (and to the broader ecosystem) follow this open, interoperable format. The expanded catalog we have built together benefits not just Microsoft's products, but the entire ecosystem of education technology that relies on structured standards data.

A shared commitment to open standards

Microsoft is proud to be a Contributing Member of 1EdTech, the organization that stewards CASE and other critical interoperability standards for education technology, including LTI, OneRoster, and Open Badges. By collaborating with fellow 1EdTech members like EdGate, we ensure that investments in standards infrastructure benefit educators everywhere, regardless of which platforms or tools they use.

When standards are open, structured, and interoperable, everyone wins: educators spend less time on manual alignment, developers can build smarter tools, and students benefit from instruction that is intentionally connected to what they are expected to learn.

What this means for educators

Within Microsoft Education, you do not need to think about CASE or data formats to benefit from this work. What you will see is:

More standards available in the Teach module and Assignments, covering more countries, subjects, and grade bands
AI-powered experiences that are better grounded in your actual curriculum
Less manual work translating curriculum documents into classroom materials

We are committed to continuing this investment: expanding coverage, improving the experience, and working with partners like EdGate and the 1EdTech community to make standards-aligned teaching easier for educators everywhere.

Helpful links

Have questions or feedback about standards in Microsoft Education? Drop a comment below or submit a request through our Standards Feedback form.

Building a hands-free voice concierge with Microsoft Foundry Voice Live and a Hosted Agent

Lee_Stott — Fri, 29 May 2026 10:16:09 GMT

This post walks through a small, working sample that wires the browser microphone to Azure AI Speech Voice Live, binds the realtime session to a Foundry hosted agent, and lets the agent answer travel questions using tool calls. The full source, infrastructure, and labs live in the repository linked at the end.

Why this combination matters

Voice user interfaces have historically been hard to build well. Streaming audio, partial transcripts, barge-in, voice activity detection, tool dispatch, and audio playback have traditionally meant stitching together five or six services. The combination of Voice Live and a Foundry hosted agent collapses that into one realtime WebSocket session with a single binding field.

Voice Live owns the audio loop: speech to text, neural text to speech, semantic turn detection, noise suppression, and echo cancellation.
The Foundry hosted agent owns the brain: instructions, memory, model selection, evaluators, and tool calling.
The link between them is one query parameter on the WebSocket URL.

What this means in practice: the browser never sees a model API key, never instantiates a tool, and never owns the agent prompt. The browser does microphone capture and audio playback. Everything else lives server-side.

The scenario

The sample is called Contoso Travel Concierge. The user is mid-journey, hands and eyes busy, and wants to ask things like:

What is the weather in Tokyo this weekend?
Is BA005 from Heathrow on time?
What time is check-in at the Marriott Marquis?

Each question triggers a tool call on the hosted agent. The reply is short, speakable, and synthesised back to the user in under a second on a warm connection.

Architecture

There are four moving parts. Three of them are managed Azure services. Only the broker is your code.

Browser client – captures PCM16 audio at 24 kHz and streams it over a WebSocket to the broker. Plays back audio chunks the broker forwards from Voice Live.
Session broker (FastAPI) – authenticates to Azure with DefaultAzureCredential, builds the Voice Live WebSocket URL with a short-lived bearer token, and relays frames in both directions.
Voice Live – the Azure AI Speech realtime endpoint. Transcribes the user, hands the text to the bound agent, and synthesises the agent’s reply.
Foundry hosted agent – a prompt-kind agent in Azure AI Foundry with instructions, tool definitions, and the microsoft.voice-live.enabled metadata flag set to true.

Two design choices are worth calling out.

The broker is small on purpose. It does authentication, URL construction, and WebSocket relay. It does not transcode audio, run business logic, or hold conversation state. Voice Live and the agent already do those things well.

The agent binding is a URL query parameter, not an SDK call. There is no per-turn HTTP request to the agent runtime. Voice Live opens a session against the agent once and streams turns through it for the lifetime of the WebSocket. That is what keeps latency low.

The Voice Live URL contract

This is the single most important thing to get right. The public Microsoft sample that ships under liupeirong/ai-foundry-voice-agent targets a different URL shape (services.ai.azure.com host, agent-id + agent-access-token parameters, an Authorization header). That shape is rejected by Foundry resources that expose voice-live-enabled agents. The shape below is the one the portal itself uses, and the one this sample dials.

Three details cause most failures:

The host must be <resource>.cognitiveservices.azure.com, not services.ai.azure.com. The broker rewrites this automatically from VOICE_LIVE_ENDPOINT.
The bearer token travels in the authorization query parameter, URL-encoded, with a literal Bearer prefix and a + (or %20) before the token. No Authorization header is sent.
agent-name and model are both the agent’s display name. agent-version is empty when you want the latest published version.

Walkthrough: from clone to spoken reply

Prerequisites

Python 3.11 or later (the sample is developed on 3.13).
The Azure CLI, signed in with az login --tenant <your-tenant-id>.
An Azure AI Foundry project in a Voice Live region (eastus2, swedencentral, or westus2).
A deployed prompt-kind agent in that project with Enable Voice Live turned on.
The Cognitive Services User role on the Foundry resource for the identity the broker will use.

Configure the broker

Copy .env.sample to .env and fill in four values:

AZURE_AI_PROJECT_ENDPOINT=https://<your-resource>.services.ai.azure.com
AZURE_AI_PROJECT_NAME=<your-foundry-project-name>
VOICE_LIVE_ENDPOINT=wss://<your-resource>.services.ai.azure.com/voice-live/realtime
VOICE_LIVE_API_VERSION=2025-10-01
FOUNDRY_AGENT_ID=<your-agent-name>

The agent name is what the Foundry portal shows on the agent card. The broker uses it for both the agent-name and model query parameters.

Install and run

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
.\scripts\start-local.ps1

The broker exposes three endpoints:

GET /healthz – liveness probe.
GET /config – returns the session.update the browser sends as its first frame.
WS /ws – the bi-directional relay to Voice Live.

Smoke test

.\scripts\test-session.ps1

A successful run prints:

[OK] /ws upgraded
   -> sent session.update
   <- {"type":"session.created",…}
   <- {"type":"session.updated",…}
[OK] session.updated received -- E2E works

This confirms the entire chain: local broker, DefaultAzureCredential token, Foundry Portal URL shape, Voice Live handshake, and the bound agent acknowledging the session.

Open the browser UI

Browse to http://localhost:8000/, click Start talking, and ask one of the sample questions. Transcripts appear in real time and the spoken reply plays back through the audio context.

Inside the broker

The relay logic is tiny – the heavy lifting is the URL construction. The function below is the canonical reference; copy it if you are porting the pattern to another language.

def build_voice_live_ws_url(agent_access_token: str) -> str:
    """
    Build the Foundry Portal style Voice Live WebSocket URL.

    Auth lives in the query string only. No Authorization header is sent.
    """
    host = _ws_host_from_endpoint(VOICE_LIVE_ENDPOINT)
    qs = urlencode(
        {
            "trafficType": "FoundryPortal",
            "agent-name": FOUNDRY_AGENT_ID,
            "agent-version": "",
            "agent-project-name": AZURE_AI_PROJECT_NAME,
            "api-version": VOICE_LIVE_API_VERSION,
            "model": FOUNDRY_AGENT_ID,
            "client-request-id": str(uuid.uuid4()),
            "authorization": f"Bearer {agent_access_token}",
        },
        quote_via=quote,
    )
    return f"wss://{host}/voice-live/realtime?{qs}"

The relay itself is a pair of asyncio tasks: one forwarding browser frames upstream, one forwarding Voice Live frames back. Audio bytes are passed straight through – the broker never decodes them.

Deploying the hosted agent

The most reliable way to create a voice-live-enabled agent is the Foundry portal. Agents created via the Assistants v2 SDK do not carry the required metadata by default and will be rejected by the Voice Live URL shape above.

The portal steps are:

Open the Foundry project, go to Agents, and click New agent.
Choose Prompt agent as the kind, name it (for example travel-concierge), and pick a model deployment.
Paste the contents of agent/src/prompts/system.txt into the instructions box.
On the Voice tab, switch Enable Voice Live on. This is what sets the microsoft.voice-live.enabled = true metadata.
Add the three tools (get_weather, get_flight_status, get_hotel_info) from agent/agent.yaml on the Tools tab.
Publish the version and write the agent name back to .env as FOUNDRY_AGENT_ID.

The full deployment guide, including how to host the broker on Azure Container Apps with a managed identity, is in docs/deployment.md in the repository.

Three lessons from getting this to production

1. Voice output must be written for speech, not for screens

Foundry agents tend to format answers in markdown with citations like ([data.jma.go.jp](https://…)). When Voice Live synthesises that text, the user hears the URL read aloud, character by character. The fix is to write the agent instructions so the spoken text never contains URLs, markdown, or symbols. A short block at the end of the agent instructions does the job:

Voice output rules
- This output is read aloud by TTS. Never include URLs, domain names, or
  citation markers like "(source.com)" in your reply. Cite by speakable
  source name only.
- Never use markdown for formatting. No asterisks, brackets, backticks,
  bullets, or hashes. Write in plain spoken sentences.
- Keep numbers speakable: say "thirty degrees Celsius", not "30C / 86F".
- Keep replies under about 40 words unless the user asks for detail.

The browser transcript can still render markdown for the eyes. The sample does so with a small, escaping markdown renderer that whitelists bold, italic, code, and http(s) links only, so the same agent reply looks polished on screen even though the spoken version contains none of it.

2. Identity is simpler than it looks

The broker uses DefaultAzureCredential and requests the https://ai.azure.com/.default scope. Locally that resolves to your az login credentials. In Azure Container Apps it resolves to the user-assigned managed identity. In both cases the only role assignment you need on the Foundry account is Cognitive Services User. There is no API key path on the working URL shape – it is bearer tokens all the way down.

3. The wrong sample wastes a day

If you start from the public liupeirong/ai-foundry-voice-agent repository against a portal-provisioned voice-live agent, the WebSocket either returns HTTP 400 or closes silently with code 1006. The cause is the URL shape, not your code. The reference probe in scripts/probe_portal_shape.py is the single source of truth for the working contract – keep it as a regression test.

Responsible AI and security notes

Credentials never reach the browser. Tokens are minted server-side and travel only on the upstream Voice Live URL.
No secrets in source. The .env file is gitignored. The .env.sample contains only placeholders.
Markdown rendering is escape-first. The browser HTML-escapes the agent reply before applying its small markdown whitelist, and links are restricted to http(s) URLs so the rule cannot emit javascript: hrefs.
Tool calls are auditable. Every turn shows up as a run in the Foundry portal under the agent, with the prompt, model output, and tool inputs and outputs visible for review.
Voice biometric considerations. If you plan to handle account verification by voice, plug in dedicated speaker recognition rather than relying on the conversational model.

Key takeaways

Voice Live plus a Foundry hosted agent is a session-level integration, not an API integration. One URL, one binding field, one WebSocket.
The browser is a thin client. Authentication, URL construction, and relay all live in a small FastAPI broker.
Get the URL shape right (cognitiveservices.azure.com, token in the query string, agent-name equals model equals the agent display name) and the rest is plumbing.
Use the Foundry portal to create the agent so the voice-live metadata is set correctly.
Write agent instructions for the ear, not the eye, then layer screen formatting on top in the browser.

Get the code and try it

Repository: github.com/microsoft/foundry-agent-voice-mode-sample
Deployment guide: docs/deployment.md in the repository.
Labs: three progressive workshops under labs/ – basic voice, adding tools, and binding a hosted agent.
Reference docs: Voice Live in Azure AI Speech and Agents in Microsoft Foundry.

If you build something on top of this pattern, open an issue or pull request on the repository. The sample is intentionally small so it stays easy to fork.

Building Reliable AI Coding Workflows Using Modular AI Agent Optimization

ShardaKaur — Thu, 28 May 2026 19:53:45 GMT

Artificial Intelligence is rapidly transforming the modern software development industry. AI-powered coding assistants such as GitHub Copilot, Claude Code, and other Large Language Model (LLM)-based systems are helping developers automate repetitive coding tasks, improve productivity, and accelerate software development processes. These tools can generate code, assist with debugging, provide recommendations, and support developers during implementation. However, despite their growing capabilities, many AI coding assistants still face challenges related to reliability, maintainability, project-specific conventions, and structured software engineering workflows.

Most coding assistants perform well for generic programming tasks but often struggle when working with domain-specific development requirements, API integrations, project architectures, validation workflows, and coding standards. In real-world software engineering environments, developers require systems that not only generate code but also follow project conventions, maintain readability, support modular development, and improve long-term maintainability.

The project “AI Agents Optimization” focuses on improving the reliability and effectiveness of AI coding agents by designing structured workflows, modular configurations, validation mechanisms, and optimized task execution strategies. The objective of the project is to investigate how AI agents can become dependable collaborators in practical software engineering tasks instead of functioning only as autocomplete systems.

The project explores different approaches for organizing AI agent workflows using structured instruction handling, modular task division, context management, validation systems, and integration of external tools and documentation sources. Different agent configurations are analyzed and evaluated to understand how workflow optimization affects software development quality and performance.

Why Existing AI Coding Workflows Often Fail

Most AI coding assistants perform well for isolated coding tasks but struggle in real-world engineering environments where projects involve multiple files, coding standards, APIs, validation requirements, and contextual dependencies.

For example, a generic prompt such as:

“Build authentication middleware”

may generate functional code, but the output often lacks:

Project-specific architecture
Error handling consistency
Validation logic
Security best practices
Dependency awareness

This project approaches the problem differently by introducing a structured workflow pipeline where AI agents operate in defined stages rather than generating outputs in a single step.

The workflow separates planning, generation, validation, and refinement into independent modules. This improves maintainability, reduces inconsistent outputs, and supports iterative refinement similar to real software engineering workflows.

Project Objectives

The primary objective of this project is to optimize AI coding agents for real-world software engineering workflows. The project aims to improve how AI systems handle development tasks such as code generation, debugging, testing, validation, feature implementation, and workflow management.

Another major objective is to design modular AI workflows where different stages of software development are managed systematically. The workflow focuses on task planning, instruction processing, validation, refinement, and output evaluation. This structured approach improves transparency, maintainability, and consistency in AI-generated outputs.

The project also aims to evaluate how AI coding agents perform under different configurations and development scenarios. By testing multiple workflows and structured instruction methods, the project analyzes how optimization techniques improve development reliability and coding quality.

Technologies and Tools Used

The project utilizes multiple modern technologies and development tools for experimentation and workflow optimization.

Technology / Tool	Purpose
Python	Automation and scripting
GitHub Copilot	AI-assisted coding
Claude / LLM APIs	AI workflow experimentation
Visual Studio Code	Development environment
Git & GitHub	Version control and repository management
Structured Prompting	Workflow optimization
MCP Concepts	Tool and context integration

These tools collectively support the implementation and testing of optimized AI coding workflows.

Implementation Workflow

The system was implemented using a modular AI workflow pipeline where each stage performs a dedicated engineering task.

Step 1 — Task Parsing

The user submits a development task or coding requirement. The Instruction Processing Module extracts:

Objective
Constraints
Project context
Expected output format

Example structured prompt:

Task: Create JWT authentication middleware

Language: Node.js

Constraints:

- Use Express.js

- Add token validation

- Follow modular architecture

- Include error handling

Step 2 — Planning & Reasoning

The Planning Module divides the task into subtasks such as:

Route handling
Token verification
Error management
Security validation

This improves reasoning consistency before generation begins.

Step 3 — Code Generation

The Code Generation Module produces outputs using structured prompts and contextual references instead of generic instructions.

Step 4 — Validation

Generated outputs are validated using:

Syntax checks
Logical consistency checks
Formatting standards
Dependency validation

Step 5 — Refinement

If validation fails, the workflow loops back into refinement where issues are corrected before final delivery.

System Workflow

The workflow of the AI Agents Optimization system is based on modular task execution and structured development processes. The workflow begins with task planning and requirement analysis. The AI agent receives structured instructions along with coding constraints, project context, and validation requirements.

The system processes the provided instructions and generates outputs according to defined workflows and development standards. Different configurations are tested to evaluate how instruction structures and modular task handling influence the quality of generated code

The workflow also includes validation and refinement stages where generated outputs are analyzed for correctness, maintainability, and consistency. The project focuses not only on code generation but also on improving readability, workflow transparency, debugging support, and adherence to project conventions.

Key Features of the Project

Structured AI workflow design
Modular task execution
AI-assisted software development
Workflow optimization strategies
Validation and refinement mechanisms
Integration of development tools and documentation
Improved maintainability and readability
Support for practical software engineering workflows

Challenges Faced During Development

One of the major challenges encountered during the project was maintaining consistency and reliability in AI-generated outputs. Different AI models often produce different responses depending on prompts, context, and task structure. Designing workflows that improve output stability and maintain coding standards required careful experimentation and optimization.

Another challenge involved integrating structured workflows while ensuring flexibility in task execution. AI systems often require clear instructions and contextual information to produce accurate outputs. Balancing automation with maintainability and project-specific requirements was an important aspect of the project.

Managing validation and refinement processes was also challenging because generated outputs needed to be evaluated not only for correctness but also for readability, maintainability, and software engineering best practices.

Observations and Outcomes

During experimentation, structured workflows produced more reliable and maintainable outputs compared to single-prompt generation approaches.

Some important observations included:

Reduced repetitive corrections during code refinement
Improved consistency in generated outputs
Better adherence to coding structure and formatting
More stable workflow behavior for multi-step tasks
Improved readability and maintainability of generated code

The validation and refinement stages were particularly effective in reducing incomplete outputs and improving response quality.

Although the project focuses primarily on workflow architecture and qualitative analysis rather than benchmark testing, the results demonstrate that modular AI pipelines can significantly improve practical software engineering workflows.

Future Enhancements

The project can be further enhanced by implementing advanced multi-agent collaboration systems where multiple AI agents work together on complex software development tasks. Future versions may also include real-time documentation integration, automated testing frameworks, cloud-based workflow management, and improved reasoning models.

Additional enhancements may include IDE extensions, intelligent debugging systems, automated code review mechanisms, and adaptive workflow optimization based on project requirements.

Conclusion

The AI Agents Optimization project demonstrates how structured workflows and modular configurations can improve the effectiveness of AI-powered coding assistants in modern software engineering environments.

By focusing on workflow optimization, validation mechanisms, modular task execution, and structured instruction handling, the project highlights the future potential of AI agents as reliable development collaborators capable of supporting real-world software engineering processes.

The project represents an important step toward building dependable AI-assisted development systems that improve productivity, maintainability, and software quality while supporting modern engineering practices.

How to Try This Workflow

Define a structured development task
Provide project constraints and context
Break the task into subtasks
Generate output using structured prompts
Validate output quality
Refine based on validation feedback

Hybrid AI Agents in Python: Routing Between Foundry Local and Microsoft Foundry

Lee_Stott — Wed, 27 May 2026 07:00:00 GMT

Why hybrid, and why now

If you build AI features today, you are caught between three forces. Users want low latency and strong privacy. Product teams want frontier reasoning capability. Finance teams want predictable cost. No single model satisfies all three. Run everything on a small on-device model and you bottleneck on complex questions. Send everything to a frontier cloud model and you pay for trivial requests, leak sensitive data across a network boundary, and add hundreds of milliseconds of latency to greetings.

The pragmatic answer is hybrid inference: a lightweight local model classifies every request first, simple or sensitive ones stay on the device, and only the genuinely hard or frontier-capability requests escalate to the cloud. Microsoft now ships both halves of that pattern as supported Python SDKs — foundry-local-sdk for on-device inference and azure-ai-projects for Microsoft Foundry cloud models. This post walks through a working reference implementation that combines them behind a single ask() call.

The full source is at github.com/leestott/fl-mixedmodel. It is Python-only, secretless by design, and ships with a Gradio diagnostics UI, a CLI demo mode, and a full pytest suite.

The contract: one schema, two paths

The most important architectural decision is that callers never know which path served a request. Every response, local or cloud, returns the same dataclass:

class InferencePath(str, Enum):
    LOCAL = "local"
    CLOUD = "cloud"
    LOCAL_FALLBACK = "local_fallback"   # cloud attempted, fell back to local
    CLOUD_FALLBACK = "cloud_fallback"   # local attempted, fell back to cloud

@dataclass
class AgentResponse:
    answer: str
    path: InferencePath
    model: str
    reason: str
    confidence: float
    latency_ms: float
    correlation_id: str
    prompt_tokens: Optional[int] = None
    completion_tokens: Optional[int] = None
    fallback: bool = False
    fallback_reason: Optional[str] = None
    metadata: dict = field(default_factory=dict)

This is what makes the design honest. The router can change, the cloud model can be swapped from gpt-4o to gpt-5.4, fallback policies can flip — and the calling code never breaks. The four InferencePath values give you full observability without leaking implementation details into the API surface.

Architecture in one diagram

┌─────────────┐   prompt    ┌──────────────────────────┐
│   caller    │ ──────────► │   HybridAgentService     │
└─────────────┘             │      .ask(prompt)        │
                            └────────────┬─────────────┘
                                         │
                            ┌────────────▼─────────────┐
                            │     RoutingPolicy        │
                            │  1. Heuristic gate       │
                            │  2. Local router LLM     │
                            │  3. Hard policy gates    │
                            └─────┬─────────────┬──────┘
                                  │             │
                          LOCAL  ◄┘             └► CLOUD
                                  │             │
                       ┌──────────▼──┐   ┌──────▼───────┐
                       │ Foundry     │   │ Microsoft    │
                       │ Local SDK   │   │ Foundry      │
                       │ (phi-4-mini)│   │ (gpt-5.4)    │
                       └─────────────┘   └──────────────┘

Best practice: the two-stage router pattern

Before walking through the implementation, it is worth stating the design pattern explicitly, because it is the part that generalises beyond this specific repo. The cleanest design for hybrid inference is a two-stage router.

Stage 1 — local router. A small local model performs intent and complexity classification first. It does not answer the question; it decides where the question should go.
Stage 2 — route the answer.
- If the prompt is simple, private, latency-sensitive, or clearly within local capability, route to a local task model on the device.
- If the prompt is complex, needs deeper reasoning, a larger context window, or a capability unavailable locally, escalate to a cloud frontier model in Microsoft Foundry.

Microsoft's current guidance for the cloud side is to use the Responses API and choose one of two control modes:

Pass a specific deployment name (for example gpt-5.4) when you want deterministic control over which model serves the request, which is the right choice for regulated workloads, repeatable evaluations, or cost ceilings.
Pass model-router as the deployment when you want Microsoft Foundry to automatically select the best available cloud model for each request. This is a sensible default for general-purpose agents where you would rather let the platform optimise the model choice as new ones are released.

The reference repo exposes both as environment variables so you can switch without code changes:

# .env.example
FOUNDRY_CLOUD_MODEL_DEPLOYMENT=gpt-5.4        # deterministic
FOUNDRY_CLOUD_ROUTER_DEPLOYMENT=model-router  # auto-select

Best practice: pin the right SDK versions

Two SDKs do the heavy lifting and both have had recent breaking changes, so version discipline matters.

Local development — foundry-local-sdk. The current public guidance is to use the Foundry Local SDK package foundry-local-sdk, which provides model discovery, download, cache, load, unload, chat completions, embeddings, audio transcription, and an optional built-in web service. Use version 1.1.0, released on 5 May 2026. Earlier versions used an OpenAI-compatible client surface that has since been replaced by the FoundryLocalManager → load_model → get_chat_client → complete_chat chain shown above. Pin it explicitly:
```
# requirements.txt
foundry-local-sdk>=1.1.0
```
Cloud orchestration and agents — azure-ai-projects. For cloud-side orchestration, Microsoft's current Python guidance is to use azure-ai-projects, which the docs describe as part of the Microsoft Foundry SDK and as the entry point for agents, deployments, connections, datasets, evaluations, and an OpenAI-compatible client returned by get_openai_client(). The current PyPI listing shows azure-ai-projects 2.1.0. Pin it explicitly:
```
# requirements.txt
azure-ai-projects>=2.1.0
azure-identity>=1.17.0
```

If you find yourself reading old samples that import azure.ai.inference as the cloud entry point, or that initialise Foundry Local through a raw openai.OpenAI(base_url=...) client, you are looking at pre-2026 patterns. The current shape is what the reference repo uses: FoundryLocalManager.initialize(Configuration(...)) for the device and AIProjectClient(...).get_openai_client() for the cloud.

Stage 1: a deterministic privacy gate

Before any model touches a prompt, a deterministic heuristic classifier scans for sensitive patterns — passwords, API keys, SSN/NHS numbers, PII signals, explicit "do not share" flags. If the heuristic returns PrivacyClass.RESTRICTED, the prompt is forced local. The router LLM is not called. The cloud provider is not called. The decision is auditable from a single regex pass.

# app/routing/policy.py
def decide(self, prompt: str, correlation_id: str = "") -> RoutingDecision:
    hint, privacy, complexity, h_reason = self._heuristic.classify(prompt)

    # Hard gate: restricted content never leaves the device
    if privacy == PrivacyClass.RESTRICTED:
        return self._make_decision(
            target=RouteTarget.LOCAL,
            confidence=1.0,
            reason=f"Policy hard-gate: {h_reason}",
            privacy=privacy,
            complexity=complexity,
            deterministic=True,
            correlation_id=correlation_id,
        )

    # Hard gate: very high complexity always goes to cloud
    if complexity == ComplexityBand.VERY_HIGH:
        return self._make_decision(
            target=RouteTarget.CLOUD,
            confidence=1.0,
            reason="Policy hard-gate: very_high complexity requires frontier model",
            ...
        )

This is the most important responsible-AI control in the whole system. If your privacy review depends on an LLM correctly classifying every prompt, you do not have a privacy control — you have a probability distribution. Deterministic gates first, model judgement second.

Stage 2: a local LLM as the router

For everything that passes the privacy gate, a small local model classifies whether the prompt needs frontier capability. This is the bit that surprises most engineers: you can do useful routing with a 4B parameter model running on a laptop CPU. The router does not need to answer the question. It only needs to classify it.

The reference implementation uses phi-4-mini via Foundry Local. Initialising it is two lines:

# app/providers/local_provider.py (excerpt)
from foundry_local import FoundryLocalManager
from foundry_local.models import Configuration

self._manager = FoundryLocalManager.initialize(
    Configuration(app_name="hybrid-agent")
)
self._router_model = self._manager.load_model(self._config.local_router_alias)
self._chat_client  = self._router_model.get_chat_client()

response = self._chat_client.complete_chat(
    messages=[
        {"role": "system", "content": ROUTER_SYSTEM_PROMPT},
        {"role": "user",   "content": prompt},
    ],
)

The router prompt asks for a strict JSON response: { "target": "local|cloud", "confidence": 0.0-1.0, "complexity": "low|medium|high|very_high", "reason": "..." }. The application parses it, applies the confidence threshold from config (default 0.6), and falls back to the heuristic decision if the router LLM is unsure or its JSON is malformed. The router never blocks the answer path — that is a deliberate reliability choice.

Cloud inference via Microsoft Foundry

When the policy returns RouteTarget.CLOUD, the request goes through AIProjectClient, which gives you an openai.OpenAI-compatible client wired to your Foundry project with DefaultAzureCredential. No API keys. No secrets in .env.

# app/providers/cloud_provider.py (excerpt)
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

self._project = AIProjectClient(
    endpoint=self._config.foundry_project_endpoint,
    credential=DefaultAzureCredential(),
)
self._openai_client = self._project.get_openai_client()

response = self._openai_client.chat.completions.create(
    model=self._config.foundry_cloud_model_deployment,  # e.g. "gpt-5.4"
    messages=messages,
    max_completion_tokens=max_tokens,
)

A subtle gotcha worth flagging: gpt-5 and o-series deployments reject the legacy max_tokens parameter and require max_completion_tokens. They also reject custom temperature values. The reference repo handles this by trying the new parameter first and falling back to the legacy one only when the API returns the specific unsupported parameter error. That keeps the same code working against older deployments without forking the provider.

Graceful degradation: the fallback paths

Hybrid systems fail in interesting ways. The cloud can be down. The local model can throw because the GPU ran out of memory. A reasoning model can return an empty completion. The service handles all of these by attempting the alternative path and labelling the response so observability stays honest:

Cloud route fails → local fallback. The response carries path=LOCAL_FALLBACK, fallback=true, and a populated fallback_reason. The user gets an answer instead of an error.
Local route fails → cloud fallback, but only if privacy class is not RESTRICTED. A sensitive prompt that the local model could not handle never leaks to the cloud as a fallback. It returns a clear error instead. This is the second hard gate in the system.
Both fail. A structured error response with a correlation ID, never a stack trace.

That last rule — fallback respects privacy class — is the kind of decision that is easy to skip and impossible to bolt on later. Encode it once in the service layer and your privacy reviewers will thank you.

What it looks like in practice

The diagnostics panel in the Gradio UI shows the routing decision live: path, model, confidence, latency, privacy class, complexity band, and the full JSON response. Five canonical scenarios shake out the entire decision tree:

"hello" → path=local, confidence=1.0, complexity=low. Heuristic only. No router LLM call. ~3 seconds end-to-end with phi-4-mini cached.
"explain transformer self-attention in depth with maths" → path=cloud, model=gpt-5.4, complexity=high. Router LLM classifies, hard gate confirms.
"my password is hunter2, suggest a stronger one" → path=local, privacy=restricted, deterministic=true. Privacy gate fires before any model sees it.
"summarise this 8 KB document" with cloud unavailable → path=cloud_fallback (local handles it, response is labelled).
Complex prompt with local model error → path=local_fallback, fallback_reason populated.

You can reproduce all five without any models installed by running python -m app.main --demo. The demo mode swaps the providers for deterministic stubs so you can validate the routing logic and the response schema in under a second on any machine.

Operational lessons learned

Some things the reference implementation only gets right because it got them wrong first:

Pick a non-reasoning model for the router. Reasoning-tuned local models (Phi-4-reasoning, o-style) wrap their output in <think> blocks and blow your JSON parser. phi-4-mini is faster and more reliable for classification.
Cache the local model. First load can take 30–60 seconds while Foundry Local downloads weights. Initialise the service once at process startup, not per request.
Use correlation IDs everywhere. The service attaches one per request and the structured JSON logger emits it on every event. When you are debugging a fallback path across two model providers, this is the difference between five minutes and five hours.
Run the privacy heuristic on every fallback path too. A naive implementation might route locally, fail, and then send the same sensitive prompt to the cloud as a "graceful" fallback. That is not graceful, it is a data leak.
Keep configuration in .env and out of code. Privacy mode, fallback toggles, confidence threshold, model aliases — all environment-driven. The config.py module is the only place that reads them.

Responsible AI in a hybrid topology

Hybrid does not make responsible AI harder, but it does make it different. Three controls earn their keep:

Data residency by default. The local path keeps prompts and answers on the device. For RESTRICTED content this is mandatory; for everything else it is a free latency and cost win.
Auditability. Every routing decision is logged with the deterministic reason, the heuristic class, the router LLM output, the confidence, and the correlation ID. You can answer "why did this prompt go to the cloud?" months later.
Keyless auth. DefaultAzureCredential means there is no API key to leak, rotate, or commit by accident. The repo's .gitignore, SECURITY.md, and pre-push checklist enforce this end-to-end.

Try it

Five minutes, no Azure account needed for the demo:

git clone https://github.com/leestott/fl-mixedmodel.git
cd fl-mixedmodel

python -m venv .venv
.venv\Scripts\activate          # Windows
# source .venv/bin/activate     # macOS / Linux

pip install -r requirements.txt
python -m app.main --demo       # all five scenarios, no models required

To run with real models, install Foundry Local, copy .env.example to .env, set your FOUNDRY_PROJECT_ENDPOINT, then:

az login
python -m app.main --ui --port 7860

Where to go next

Repository: github.com/leestott/fl-mixedmodel — full source, tests, specification, screenshots.
Foundry Local SDK: pypi.org/project/foundry-local-sdk and the Foundry Local docs.
Azure AI Projects SDK: pypi.org/project/azure-ai-projects and the Microsoft Foundry docs.
Azure Identity: DefaultAzureCredential reference.
Phi-4-mini: Phi-4-mini on Hugging Face.

Key takeaways

The best-practice pattern is a two-stage router: local model classifies first, then either a local task model or a Microsoft Foundry cloud model answers.
For cloud control, use the Responses API with either a named deployment (deterministic) or model-router (auto-select).
Pin foundry-local-sdk >= 1.1.0 (5 May 2026) and azure-ai-projects >= 2.1.0. The 2026 SDK surfaces are not backwards-compatible with pre-2026 samples.
Hybrid inference is a routing problem, not a model problem. A small local model is enough to classify the request.
Deterministic privacy gates beat probabilistic ones. Code the rules; let the LLM judge only what is left.
Return the same response schema from every path. Label fallbacks honestly. Carry a correlation ID everywhere.
Keep auth keyless with DefaultAzureCredential and your .env out of git.
Test the routing decisions, not just the model outputs. Demo mode and a strong pytest suite pay back every time you swap a model.

Hybrid AI is not a compromise between local and cloud. It is the supervisor pattern applied to inference — fast and private where you can be, frontier where you have to be, observable everywhere. The hard part is the contract, not the models.

Set clear AI expectations for every assignment with Student AI Guidelines

Leif Brenne — Tue, 26 May 2026 14:30:34 GMT

The challenge: students don't know where they stand with AI

Every educator has a different approach to AI in their classroom. Some want students using it freely. Others want AI limited to brainstorming or editing. Some assignments shouldn't involve AI at all.

The problem? Students are left guessing. Educators have been piecing together workarounds — writing AI policies into assignment instructions, referencing school handbooks, or adding disclaimers to rubrics. None of these are built into the assignment itself, and students often miss them entirely.

Student AI Guidelines in Assignments

Student AI Guidelines give educators a structured way to set AI expectations directly inside an assignment in Microsoft Teams. When creating an assignment, educators now see a new option to set a guideline level with suggested text:

Full AI use allowed. Students can use Copilot for any part of the assignment.
AI for editing only. Students write their own work first, then use Copilot to polish, revise, or check grammar.
AI for brainstorming only. Students can use Copilot to generate ideas or explore topics, but the final work should be their own.
No AI. The assignment should be completed without AI assistance.

Student AI Guidelines are available for all grade levels, on desktop and mobile. All students in the assignment see the same guideline.

A note on what these guidelines are — and aren't. Student AI Guidelines are a communication tool, not a lockdown. They set clear expectations that students see in the assignment, but they don't technically block access to AI tools. They work the same way a teacher's verbal instruction does: "Here's what I expect for this assignment." The value is in making that expectation visible, consistent, and built into the assignment itself.

These are starting points, not fixed rules. Each level comes with suggested text that educators can edit freely to match their school's policies, terminology, or classroom norms. If your school uses different language around AI use — or has its own framework — update the text to reflect that. The feature adapts to your school, not the other way around.

Even if your school hasn't enabled Copilot, Student AI Guidelines give you a structured way to communicate AI expectations to students — whether that's encouraging responsible AI use or formalizing a no-AI policy.

What students see

When an educator sets a guideline, students see it in their assignment view — no hunting through instructions or attachments. The guideline card shows the educator's expectations and, for levels that allow AI use, a direct button to launch Copilot Chat.

The Copilot launch button appears for students aged 13 and older at schools where an IT admin has enabled Copilot. If your school hasn't set up Copilot yet, check out the Copilot setup guide for IT admins to get started. If Copilot isn't enabled, students still see the guideline — just without the launch button.

If no guideline is set, nothing changes — the student experience stays exactly as it is today.

Save time: set a default and reuse across classes

Two features help you avoid repeating setup work:

Set as default. Any guideline level — including "No AI" — can be set as the default for all new assignments you create. If your school's policy is that most assignments should restrict AI use, set that as your default and you're covered. You can always override it on individual assignments when you want to allow more (or less) AI use.

Import Settings. Once you've configured your Student AI Guidelines in one class, you can apply those same settings to other classes using Import Settings. This copies your guideline levels and custom text across classes so you don't have to re-create them each time. Learn more: Import Settings in Assignments and Grades.

Why this matters

This feature sits at the intersection of two things educators have been asking for: clarity around AI use, and an easy on-ramp to Copilot. Instead of competing with third-party AI tools through restriction, Student AI Guidelines give educators a way to channel AI use purposefully — on their terms, per assignment, with clear communication to students.

Resources

Set Student AI Guidelines on and assignment in Microsoft Teams

Manage Student AI Guidelines in Assignments

CI/CD for AI Agents on Microsoft Foundry

Lee_Stott — Fri, 22 May 2026 08:50:02 GMT

Introduction

Building an AI agent is the straightforward part. Shipping it reliably to production with version control, evaluation-driven quality gates, multi-environment promotion, and enterprise governance is where most teams run into friction.

Microsoft Foundry changes this. It is Microsoft's AI app and agent factory: a fully managed platform for building, deploying, and governing AI agents at scale. It provides a first-class agent runtime with built-in lifecycle management, making it possible to apply the same CI/CD rigour you already use for application software to AI agents — regardless of whether you are building containerised hosted agents or declarative prompt-based agents.

This post walks through a complete, production-ready reference architecture for doing exactly that. You will find the GitHub Actions workflow, the Azure DevOps pipeline YAML, and the architecture diagram linked throughout.

Reference implementation repository: foundry-agents-lifecycle
and CI/CD for AI Agents on Microsoft Foundry

Why Agent CI/CD Is Different

Traditional software pipelines gate releases on test pass/fail. Agent pipelines require an additional, critical layer: evaluation-driven quality gates. Before any agent version can be promoted to the next environment, it must pass three categories of evaluation:

Quality — answer correctness, task completion rate, hallucination rate
Safety — grounded responses, policy compliance, tool usage validation
Performance — token usage per query, p95 response latency

A second key difference is the deployment unit. You are not deploying a binary or a container tag in isolation. You are deploying an agent version — an immutable artefact that bundles the model selection, system instructions, tool definitions, and configuration together. This is what enables deterministic promotion and full auditability across environments.

"Agents follow a standard CI/CD pattern, but with a critical shift: promotion happens at the agent version level, and release gates are driven by evaluation outcomes, not just test results."

Reference Architecture

Figure 1: End-to-end CI/CD reference architecture for hosted and prompt-based agents on Microsoft Foundry.

The architecture has five logical layers, flowing from developer commit to production monitoring:

Layer 1 — Developer Layer

The developer layer is a standard source-controlled repository in GitHub or Azure DevOps. It contains:

Agent code written in Python or .NET
agent.yaml or prompt definition files for prompt-based agents
Tool configurations: MCP servers, REST API connectors, or other integrations
Infrastructure as Code: Bicep or ARM templates for provisioning the Foundry project and dependencies

Layer 2 — CI Pipeline (Build · Validate · Evaluate)

Every push or pull request triggers the CI pipeline. It performs five steps:

Docker build — for hosted agents, build and tag the container image
Static checks — lint with ruff, security scan with bandit, agent YAML schema validation
Unit and tool tests — pytest suites covering agent logic and tool integrations
Evaluation gate — run evaluation datasets; fail the pipeline if thresholds are breached
Image push — push the validated container to Azure Container Registry (ACR)

Prompt-based agents skip the Docker build step. Instead, the YAML definition and prompt bundle are validated against schema and evaluated against golden datasets.

Layer 3 — CD Pipeline (Multi-stage Promotion)

A single agent version is promoted through three Foundry project environments:

Stage	Environment	Activities	Gate
Stage 1	Dev Foundry Project	Deploy vNext version, smoke tests, developer evals	Eval quality thresholds
Stage 2	Test / QA Foundry Project	Scenario tests, HITL validation, safety evaluation	Eval gates + human approval
Stage 3	Production Foundry Project	Promote version, enable endpoint, post-deploy smoke test	Required reviewer approval

Rollback is straightforward: switch the active version pointer back to the previous agent version. No re-deployment is needed.

Layer 4 — Microsoft Foundry Agent Service

The Foundry Agent Service runtime provides:

Hosted agent runtime — managed container execution supporting Agent Framework, LangGraph, Semantic Kernel, or custom code
Prompt-based agent runtime — declarative agent definitions, no container required
Built-in lifecycle operations — version, start, stop, rollback
Entra Agent Identity — each deployed version receives a dedicated Microsoft Entra managed identity
RBAC and policy enforcement — Azure role-based access controls per project
Observability — distributed traces, structured logs, and evaluation signals

Layer 5 — Monitoring, Governance, and Control Plane

Foundry control plane: agent registry, environment configuration, version history
OpenTelemetry forwarded to Azure Monitor and Application Insights
Continuous evaluation pipelines for ongoing quality, grounding, and safety monitoring
Azure Policy and RBAC enforcement at the platform level

Environment Topology

There are two topology options. We recommend Option A for all production workloads:

Option	Structure	Best for	Trade-off
A — Recommended	Dev Project → Test Project → Prod Project (separate Foundry projects)	Enterprise workloads	Full isolation, clean RBAC boundaries, easier governance
B — Lightweight	Single Foundry project with agent version tags (dev/test/prod)	Small teams, prototyping	Simpler setup, but weaker environment separation

Separate projects mean separate RBAC policies, separate connection strings, and separate evaluation signals. A developer service principal has access only to the Dev project; the CI/CD identity has restricted access to promote to Test and Production.

Evaluation Gates — The Core Difference

Evaluation gates transform a standard software pipeline into an AI-safe deployment pipeline. They run at two points: pre-merge (CI) and pre-promotion (CD).

Defining the Gates

Category	Metric	CI threshold	Prod threshold
Quality	Hallucination rate	< 5%	< 3%
Quality	Task completion rate	> 90%	> 95%
Safety	Grounded response rate	> 95%	> 98%
Safety	Policy violations	0	0
Performance	p95 latency	< 4 000 ms	< 3 000 ms
Cost	Token usage per query	Track only	Alert on > 20% regression

Gate Enforcement (Python)

import json
import sys

def check_gates(results_path: str) -> None:
    with open(results_path) as f:
        results = json.load(f)

    failures = []

    if results["hallucination_rate"] > 0.05:
        failures.append(f"Hallucination rate {results['hallucination_rate']:.1%} exceeds 5% threshold")

    if results["task_completion_rate"] < 0.90:
        failures.append(f"Task completion {results['task_completion_rate']:.1%} below 90% threshold")

    if results["latency_p95_ms"] > 4000:
        failures.append(f"p95 latency {results['latency_p95_ms']}ms exceeds 4000ms threshold")

    if results.get("policy_violations", 0) > 0:
        failures.append(f"Policy violations detected: {results['policy_violations']}")

    if failures:
        for f in failures:
            print(f"GATE FAILED: {f}", file=sys.stderr)
        sys.exit(1)

    print("All evaluation gates passed — proceeding to deployment")

if __name__ == "__main__":
    check_gates(sys.argv[1])

Hosted vs Prompt-Based Agents — Pipeline Differences

Capability	Hosted Agents	Prompt-Based Agents
Deployment unit	Container image + agent definition	YAML / prompt configuration bundle
Build step required	Yes — Docker build + ACR push	No — YAML validation only
Supported frameworks	Agent Framework, LangGraph, Semantic Kernel, custom	Foundry declarative runtime
Promotion artefact	Versioned agent with container image reference	Versioned prompt/config bundle
CI focus	Code quality, tool tests, evaluation	Prompt schema validation, evaluation
Rollback mechanism	Switch active agent version	Switch active agent version
Runtime management	Foundry manages container lifecycle	Foundry manages declarative runtime

CI Pipeline Walkthrough

The following steps are representative of the full GitHub Actions workflow available in github-actions-pipeline.yml alongside this post.

Hosted Agent CI

# 1. Static checks
ruff check .
bandit -r src/ -ll
python scripts/validate_agent_config.py --config agent.yaml

# 2. Tests
pytest tests/unit/ -v --tb=short
pytest tests/tools/ -v --tb=short

# 3. Evaluation gate
python scripts/run_evaluations.py \
    --dataset eval/datasets/golden_set.jsonl \
    --output  eval/results/results.json

python scripts/check_eval_gates.py \
    --results eval/results/results.json \
    --max-hallucination   0.05 \
    --min-task-completion 0.90 \
    --max-latency-p95     4000

# 4. Push container image
az acr build \
    --registry myregistry.azurecr.io \
    --image    "myagent:$SHA" \
    --file     Dockerfile .

Prompt-Based Agent CI

# Validate YAML / prompt definitions
python scripts/validate_agent_config.py --config agent.yaml

# Evaluation against golden dataset
python scripts/run_evaluations.py \
    --dataset eval/datasets/golden_set.jsonl \
    --output  eval/results/results.json

python scripts/check_eval_gates.py \
    --results eval/results/results.json

CD Pipeline Walkthrough

Stage 1 — Dev Deployment

python scripts/deploy_agent.py \
    --env              dev \
    --image            "myregistry.azurecr.io/myagent:$SHA" \
    --foundry-endpoint $FOUNDRY_ENDPOINT_DEV \
    --agent-config     agent.yaml

# Returns the new agent version ID, stored for promotion
AGENT_VERSION=$(python scripts/get_active_version.py --env dev)

Stage 2 — Promote to Test (after approval gate)

python scripts/promote_agent.py \
    --from-env         dev \
    --to-env           test \
    --agent-version    $AGENT_VERSION \
    --foundry-endpoint $FOUNDRY_ENDPOINT_TEST

# Run scenario tests and safety evaluation
python scripts/run_evaluations.py \
    --dataset  eval/datasets/scenario_set.jsonl \
    --output   eval/results/test-results.json

python scripts/check_eval_gates.py \
    --results              eval/results/test-results.json \
    --max-hallucination    0.03 \
    --min-task-completion  0.95

Stage 3 — Promote to Production (after required reviewer approval)

python scripts/promote_agent.py \
    --from-env         test \
    --to-env           prod \
    --agent-version    $AGENT_VERSION \
    --foundry-endpoint $FOUNDRY_ENDPOINT_PROD

# Enable the production endpoint
python scripts/enable_agent_endpoint.py \
    --agent-version    $AGENT_VERSION \
    --foundry-endpoint $FOUNDRY_ENDPOINT_PROD

Rollback

# Switch the active version to the previous known-good version
python scripts/promote_agent.py \
    --from-env         prod \
    --to-env           prod \
    --agent-version    $PREVIOUS_AGENT_VERSION \
    --foundry-endpoint $FOUNDRY_ENDPOINT_PROD

# OR delete the failing version
python scripts/delete_agent_version.py \
    --agent-version    $AGENT_VERSION \
    --foundry-endpoint $FOUNDRY_ENDPOINT_PROD

Deployment Using the Azure AI Projects SDK

The azure-ai-projects SDK provides programmatic control over the full agent lifecycle. This is the recommended approach for CI/CD scripts where you need deterministic, scriptable deployment.

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

# Connect to the Foundry project
client = AIProjectClient(
    endpoint=FOUNDRY_PROJECT_ENDPOINT,
    credential=DefaultAzureCredential()
)

# List existing agents (useful for idempotent deploy scripts)
for agent in client.agents.list():
    print(f"Agent: {agent.name}  version: {agent.id}")

# Create a new agent version (hosted agent)
agent = client.agents.create_agent(
    model="gpt-4o",
    name="my-enterprise-agent",
    instructions="You are a helpful assistant ...",
    tools=[...],  # tool definitions
    metadata={"version": GIT_SHA, "environment": "dev"}
)
print(f"Created agent version: {agent.id}")

For hosted agents, the SDK call also references the container image pushed to ACR. Refer to the Deploy a hosted agent — Microsoft Foundry documentation for the full SDK flow including container image registration and version polling.

Reference Implementation Stack

Concern	Technology
Source control and pipelines	GitHub Actions or Azure DevOps Pipelines
Infrastructure and agent deployment	Azure Developer CLI (`azd up`)
Programmatic agent lifecycle	`azure-ai-projects` Python SDK
Agent evaluation	`azure-ai-evaluation` Python SDK
Agent runtime	Microsoft Foundry Agent Service
Container registry	Azure Container Registry (hosted agents only)
Observability	OpenTelemetry, Azure Monitor, Application Insights
Identity and access	Microsoft Entra (Agent ID, OIDC workload identity federation)
Governance	Azure Policy, RBAC, Foundry control plane

Governance and Responsible AI

Shipping AI agents at enterprise scale requires governance beyond what a traditional CI/CD pipeline provides. Microsoft Foundry addresses this at the platform level:

RBAC per environment — each Foundry project has independent access controls. Developers deploy to Dev; only CI/CD service principals (with audited OIDC tokens) can promote to Test and Production.
Agent registry and audit trail — the Foundry control plane records which agent version is active in each environment, who deployed it, and when. This satisfies enterprise audit requirements without additional tooling.
Content safety and policy enforcement — Azure Policy governs model access, data handling, and content safety rules at the infrastructure level, not just at the application code level. Policy violations block deployment automatically.
Entra Agent Identity — each deployed agent version receives a dedicated, short-lived managed identity. Agents authenticate to downstream services using least-privilege credentials scoped to that specific deployment.
Continuous evaluation in production — evaluation pipelines run on sampled production traffic, alerting when quality, safety, or cost metrics drift from their baseline.

A key trade-off to be transparent about: evaluation datasets must be maintained and updated as the agent's tasks evolve. Stale datasets produce misleading pass/fail signals. Treat your golden evaluation set as a first-class engineering artefact alongside the agent code itself.

Pipeline Files

Two pipeline files accompany this reference architecture. Both implement the same four-stage pipeline (CI Build, CI Evaluate, CD Dev, CD Test, CD Production) with environment-appropriate approval gates.

github-actions-pipeline.yml — GitHub Actions workflow. Uses GitHub Environments for approval gates and OIDC Workload Identity Federation for passwordless Azure authentication. No stored Azure credentials required.
azure-devops-pipeline.yml — Azure DevOps multi-stage YAML pipeline. Uses ADO Environments with required approvers and variable groups per environment.

Both pipelines share these security practices:

OIDC / Workload Identity Federation — no long-lived Azure credentials stored in pipeline secrets
Per-environment variable groups, each with scoped connection strings and endpoints
Evaluation quality gates enforced before every promotion step
Mandatory human approval before production deployment

Summary

The full pipeline in one view:

Developer commit
        |
   CI Pipeline
   ├── Docker build (hosted agents) / YAML validation (prompt agents)
   ├── Static checks + unit tests + tool tests
   └── Evaluation gate  ←  quality · safety · performance
        |
   Agent Version created  ← immutable, versioned artefact
        |
   CD Pipeline
   ├── Deploy to Dev       → smoke tests + eval gate
   ├── Promote to Test     → scenario tests + HITL + approval gate
   └── Promote to Prod     → enable endpoint + monitoring
        |
   Microsoft Foundry Agent Service
   └── Versioned runtime · Entra identity · RBAC · Observability
        |
   Control Plane
   └── Agent registry · Governance · Continuous evaluation

Microsoft Foundry provides the platform primitives — versioned agent deployments, multi-environment Foundry projects, built-in lifecycle management, and an enterprise observability stack — needed to operate AI agents with the same confidence as any production software system.

The key takeaway: treat the agent version as your deployment artefact, and evaluation outcomes as your release gate. The rest follows familiar CI/CD patterns you already know and trust.

Next Steps

Clone the CI/CD Repo at leestott/foundry-cicd
Clone the reference demo: foundry-agents-lifecycle on GitHub
Set up your environment: Set up your environment for Foundry Agent Service
Deploy your first hosted agent: Quickstart: Deploy your first hosted agent
Understand hosted agent concepts: Foundry Hosted Agents concepts
Automate deployments in CI/CD: Automate deployment of Microsoft Foundry agents
Manage agent versions: Manage hosted agents — Microsoft Foundry
Deploy via SDK: Deploy a hosted agent — Microsoft Foundry
SDK and endpoint reference: Microsoft Foundry SDK and Endpoints reference
Azure AI Projects SDK: azure-ai-projects Python SDK
Azure Developer CLI: Azure Developer CLI (azd) overview
Microsoft Foundry documentation hub: Microsoft Foundry on Microsoft Learn

Meet Dileepa Bandara

StudentDeveloperTeam — Fri, 22 May 2026 05:22:47 GMT

This is the next segment of our blog series highlighting Microsoft Student Ambassadors who achieved the Gold milestone, the highest level attainable, and have recently graduated from university. Each blog in the series features a different student and highlights their accomplishments, their experience with the Student Ambassador community, and what they are up to now. 

How has being part of the Microsoft Student Ambassadors community helped you connect with other tech enthusiasts and expand your global network? 

Before beginning my journey as a Microsoft Student Ambassador in 2022, I had only a few connections on campus. Joining the program transformed my network and opened doors I had never imagined. It gave me the opportunity to connect with student organizations, collaborate with peers, and even establish new student-led communities inspired by the program.

Through this experience, I founded and led initiatives such as the NIBM Computing Society, NIBM FOSS Community, and Microsoft Student Club – NIBM. These initiatives not only strengthened the tech culture on campus but also expanded my network across the country through the recognition and credibility of the Microsoft Student Ambassadors program.

Beyond my university, I connected with like-minded individuals and communities such as Microsoft Student Ambassadors Sri Lanka, Sri Lanka Developer Forum, and AI Community Sri Lanka. One of the most memorable milestones was delivering sessions at the Microsoft Sri Lanka office, a lifetime experience where I met inspiring professionals.

Through these connections, I also had opportunities to volunteer and conduct sessions for industry-level audiences. From university students to professionals, I built meaningful relationships locally and globally, including connections formed through international sessions and LinkedIn networking.

Being part of this community has truly expanded my global network and strengthened my passion for technology.

What impact did access to Microsoft tools and resources - like Azure credits, Visual Studio Enterprise, and Copilot - have on your journey as an AI-focused tech leader?

Before becoming a Microsoft Student Ambassador, I had never used Microsoft Azure because I couldn’t afford paid cloud services as a student. Receiving Azure credits through the program removed that barrier and became a turning point in my journey as an AI-focused tech leader.

With Azure, I was able to explore AI services such as Foundry, Azure Machine Learning, and cloud-based deployments. I built proof-of-concept solutions, intelligent apps, and demo projects that addressed real-world problems. This hands-on experimentation deepened my understanding of model deployment, scalability, and responsible AI practices skills essential for leading AI initiatives.

As I grew technically, I also grew as a leader. I used these tools to conduct workshops, mentor peers, and guide student communities in building AI-powered solutions. Instead of just learning AI concepts, I was enabling others to adopt them.

Visual Studio Enterprise enhanced my development workflow, while Microsoft 365 improved collaboration and project management. Copilot accelerated my coding, supported debugging, and helped me explore new AI frameworks efficiently.

These resources did more than support my learning; they empowered me to innovate, lead AI-driven communities, and confidently position myself as an emerging AI-focused tech leader.

In what ways has the Student Ambassadors program strengthened your resume, boosted your confidence, or helped you develop AI-ready skills for your future career?

We are living in the AI era, where competition is intense and building a strong career path requires more than just academic knowledge. The Microsoft Student Ambassadors program became a turning point during this critical phase of my journey.

Through continuous learning, hands-on experimentation, and knowledge sharing, I was able to build strong foundations in cloud computing and AI. The program encouraged me not only to consume knowledge but to apply it through real projects, workshops, and community initiatives. This practical exposure significantly strengthened my resume and helped me showcase measurable impact rather than just technical interest.

One of the biggest advantages was access to free Microsoft certification exams. Earning these certifications validated my skills, increased my professional credibility, and made my profile stand out in a competitive job market. As a result, I secured a role as an Associate AI Engineer, marking a major milestone in my career.

Beyond technical growth, the program boosted my confidence. Speaking at events, leading communities, mentoring peers, and collaborating with industry professionals shaped my communication and leadership abilities.

Overall, the Student Ambassadors program equipped me with AI-ready technical skills, industry-recognized credentials, leadership experience, and the confidence to pursue my future career in AI with clarity and purpose.

Meet Azmat Ullah

StudentDeveloperTeam — Fri, 22 May 2026 05:18:31 GMT

This is the next segment of our blog series highlighting Microsoft Student Ambassadors who achieved the Gold milestone, the highest level attainable, and have recently graduated from university. Each blog in the series features a different student and highlights their accomplishments, their experience with the Student Ambassador community, and what they are up to now.

How has being part of the Microsoft Student Ambassadors community helped you connect with other tech enthusiasts and expand your global network? 

Being part of the Microsoft Student Ambassadors community has transformed my journey from a local learner into a truly global collaborator. I began my ambassador experience in the APAC region in 2021, where I connected with passionate technologists across South Asia and Southeast Asia. In 2023, transitioning to the Americas region expanded my perspective even further, allowing me to bridge communities across continents.

Through co-hosting events with ambassadors from India, Pakistan, Nepal, Sri Lanka, Malaysia, the Middle East, and Latin America, I experienced firsthand how technology dissolves borders. These collaborations didn’t just amplify impact; they built lasting relationships. I was also invited to speak at events hosted by Middle Eastern and Latin American ambassadors, giving me a platform to share knowledge while learning from diverse global voices.

Microsoft Learn became a common language for this network, helping me continuously skill up and mentor learners worldwide. Over the past five years, these experiences have helped me build over thousands of meaningful connections, including fellow ambassadors, Microsoft staff, and MVPs. More than a community, the Microsoft Student Ambassadors program became my gateway to a global ecosystem driven by learning, collaboration, and shared growth.

What impact did access to Microsoft tools and resources - like Azure credits, Visual Studio Enterprise, and Copilot - have on your journey as an AI-focused tech leader?

Access to Microsoft tools shaped how I grew into an AI focused tech leader. Azure credits gave me the confidence to move from learning to building real systems. I was able to deploy AI models, host live demos, and test ideas on a scale. As an AI Project Lead, this hands-on experience taught me how real-world AI solutions behave beyond theory.

The monthly credits also helped me run practical workshops on Azure Databases and cloud fundamentals. During my time as a Microsoft Student Trainer, I created sandbox environments where learners could experiment without fear. Watching others gain confidence through these environments reminded me that leadership in AI is about enabling people, not just building technology.

Visual Studio Enterprise and Copilot supported me every day. They helped me prototype faster, debug more clearly, and mentor more effectively. By removing barriers to experimentation, Microsoft tools allowed me to focus on impact. They helped me grow with empathy, technical depth, and a strong sense of responsibility toward the AI community I serve.

In what ways has the Student Ambassadors program strengthened your resume, boosted your confidence, or helped you develop AI-ready skills for your future career?

The Microsoft Student Ambassadors program played a major role in shaping my confidence, skills, and future career. Microsoft Learn helped me start from scratch and build a strong foundation, eventually growing into a subject matter expert on topics like Databases, Cloud Computing & AI. The structured learning paths and hands-on practice made complex technologies approachable and practical.

Support from Microsoft Cloud Advocates and the program staff helped me narrow my focus and refine my strengths. Skill-based training sessions challenged me to think deeper and build with intent. AI Cloud Skills Challenges and certifications strengthened my technical knowledge and added strong credibility to my resume.

Hosting global workshops significantly improved my communication and leadership skills. Engaging with diverse audiences taught me how to explain technical concepts clearly while keeping equity, diversity, and inclusion at the center. It helped me grow as a thoughtful and inclusive leader.

Throughout this journey, Copilot became a trusted learning companion. It helped me explore ideas faster, sharpen my thinking, and stay AI ready. Together, these experiences transformed me into a confident, future-reading AI professional.

Student Devs: Build AI Agents, Compete for $55K in Prizes

Lee_Stott — Thu, 21 May 2026 07:23:46 GMT

Student Devs: Build AI Agents, Compete for $55K in Prizes

🎮 AI Skills Fest • June 4–14, 2026 • Free to Enter

$55K
Prize Pool

3
Challenge Tracks

10
Days of Hacking

Free
To Enter

Whether you're a first-year CS student or a final-year senior with a portfolio full of projects, Agents League is the best way to gain hands-on experience with agentic AI this summer and walk away with real skills employers are hiring for right now.

What You'll Actually Learn

Forget passive tutorials. Agents League is project-based learning at full speed. By the end of the hackathon, you'll have built a working AI agent and gained practical experience with the tools shaping the future of software development.

🤖 AI-Assisted Development Use GitHub Copilot to accelerate your coding workflow — from scaffolding to debugging — the way professional developers do today.	🧩 Multi-Step Reasoning Build agents with Microsoft Foundry that can plan, reason, and execute complex tasks — the core of agentic AI.
🏢 Enterprise AI Patterns Learn to build production-ready agents that integrate with Microsoft 365 and Copilot Studio — skills that translate directly to industry jobs.	🔧 Prompt Engineering Design effective prompts and orchestration flows that make AI agents reliable and useful in the real world.
📦 GitHub Workflows Submit your project through GitHub — practising version control, README writing, and open-source collaboration.	🎯 Competitive Problem-Solving Work under real constraints with deadlines, judging criteria, and peer competition — just like industry hackathons and sprints.

Pick Your Track (or Try All Three)

Agents League has three challenge tracks, each using different Microsoft AI tools. Choose based on your interests or stretch yourself by competing in multiple tracks.

Track 01. Creative Apps

Build an innovative application with AI-assisted development. This track rewards creativity, dream big and let GitHub Copilot help you bring ideas to life faster than ever.
Tool: GitHub Copilot

Track 02. Reasoning Agents

Create intelligent agents that solve complex problems through multi-step reasoning. Think: agents that can research, plan, and act. This is the cutting edge of AI.
Tool: Microsoft Foundry

Track 03. Enterprise Agents

Build knowledge agents that integrate with Microsoft 365 Copilot. Learn how businesses are deploying AI today and add enterprise AI to your skillset.
Tool: Copilot Studio • M365

Opportunities You Won't Want to Miss

Agents League isn't just a competition, it's a launchpad. Here's what's in it for you beyond the code:

💰 Win from a $55,000 USD Prize Pool
Prizes are awarded across all three tracks smaller teams and solo hackers have a real shot.
📺 Watch Live Coding Battles at Microsoft Reactor
See industry experts go head-to-head building AI agents live. Learn advanced techniques you can apply immediately to your own project.
🎓 Free Learning Resources on Microsoft Learn
Access curated learning paths and the AI Skills Navigator, structured content designed to get you from zero to submission-ready.
🌍 Join a Global Developer Community
Connect with thousands of developers on the Agents League Discord. Find teammates, ask questions, and build your professional network.
📂 Build Your Portfolio with a Real Project
Every submission lives on GitHub. Walk away with a polished, public project that demonstrates your AI skills to future employers and grad schools.
🏆 Gain Recognition from Microsoft and the Community
Top projects get visibility across the Microsoft developer ecosystem. Stand out from the crowd in internship and job applications.

Key Dates to Remember

Event	Date
Hacking Period Opens	June 4, 2026
Registration Deadline	June 12, 2026 — 12:00 PM PT
Submission Deadline	June 14, 2026 — 11:59 PM PT

How to Get Started (Right Now)

You don't have to wait until June 4th to start preparing. Here's your pre-hackathon game plan:

Register for the hackathon it's free and open to everyone.
Pick a track that matches your interests or curiosity.
Explore the learning resources on Microsoft Learn and the AI Skills Navigator.
Join the Discord community to find teammates and get early tips.
Watch the Reactor event series for live coding battles and expert walkthroughs.
Set up your GitHub repo and start experimenting before the hacking window opens.

Helpful Links

Register for Agents League Free entry, sign up now
Microsoft Reactor Events Live coding battles & workshops
AI Skills Fest The broader event
Microsoft Learn Free learning paths

The Arena Awaits 🏆

Ten days. Three tracks. $55K in prizes. Whether you go solo or squad up, this is your chance to build something real with AI and have a blast doing it.

Register Now It's Free | Watch Reactor Events

Agents League is part of AI Skills Fest and is open to the public at no cost.
Review the Hackathon Rules and Regulations and the Microsoft Event Code of Conduct before participating.