Vibe Coding for the NPU

The industry calls it AI-augmented development. The internet calls it vibe coding. Same thing: you describe what you want, an AI coding assistant writes the code, you run it, you tell it what broke, it fixes it. Repeat. You’re the architect and the QA engineer. The AI handles the implementation.

Examples include GitHub Copilot CLI, Cursor, and similar AI coding assistants. The specific tool matters less than the workflow: describe a feature, the assistant writes code, you run it, you report what worked and what broke, the assistant fixes it. The AI handles the boilerplate... Flask routes, CSS layout, regex, JavaScript event handlers. You handle the architecture decisions, the hardware testing, and the product judgment.

Why does this matter for NPU development specifically? Because AI coding assistants already know the OpenAI SDK patterns inside and out, and Foundry Local speaks a compatible API. “Compatible” means the standard SDK works for chat completions and most common inference patterns, though you may encounter minor differences in model IDs or streaming behavior. Microsoft’s documentation frames this as compatibility with “OpenAI-compatible SDKs and HTTP clients.” In practice, for the workloads covered in this post, the SDK works as-is. The barrier is significantly lower than it used to be... if you can describe a workflow, you can start building for the NPU.

Before writing any code, start where I started: the AI Toolkit for VS Code. Two things matter here: the Model Catalog and the Playground.

Open the Model Catalog and filter for models optimized for local NPU execution:

For the full local model landscape, see Ready-to-use local LLMs in Microsoft Foundry on Windows and the Windows AI overview decision tree.

The Playground is your validation sandbox. Select a model, load it, chat with it directly in VS Code. Test your prompts. Feel the latency. Hit the context limits. All before writing a line of application code. It’s also the most reliable method we’ve found to trigger the initial model download and readying for Phi Silica.

Hardware

Any Copilot+ PC with an NPU. Surface consumer Copilot+ PCs are currently Snapdragon X. Surface commercial (for Business) Copilot+ PCs are available with both Snapdragon X and Intel Core Ultra.

Software Stack

Scope: Everything in this post is Windows-only. Foundry Local, AI Toolkit, and the NPU runtimes covered here require Windows 11 on Copilot+ PC hardware (Intel Core Ultra or Qualcomm Snapdragon X).

A note on Foundry Local: Foundry Local is currently in public preview. There is no SLA and no backward compatibility guarantee. Expect changes between releases and validate in staged rings before broad deployment. For the latest, see the Foundry Local documentation hub.

⚠️ Package name warning: Install foundry-local-sdk (the official Microsoft SDK). Don’t confuse it with similarly named packages on PyPI. At time of writing, foundry-local (v0.0.1) is not the official SDK and may fail (or behave unexpectedly). Verify against Microsoft’s SDK documentation and PyPI.

Foundry Local handles model selection across silicon families automatically through its alias system. For most developers and single-device use, you write one codebase and it works. If you’re deploying to a mixed enterprise fleet, here are the platform-specific optimizations we’ve found.

Architecture Detection Under Emulation

On Windows-on-ARM, Python x64 reports AMD64 via platform.machine(). Both it and PROCESSOR_ARCHITECTURE are wrong under emulation. Use WMI:

result = subprocess.run(
    ["powershell", "-NoProfile", "-Command",
     "(Get-CimInstance Win32_Processor).Name"],
    capture_output=True, text=True, timeout=5,
)
cpu = result.stdout.strip().lower()
if "qualcomm" in cpu or "snapdragon" in cpu:
    return "qualcomm"

Run native ARM64 Python where possible on Snapdragon devices for better performance and fewer emulation quirks.

Rule: On Snapdragon, prefer ARM64 Python. Avoid x64 emulation when you can.

Model Compatibility Varies by Silicon

Observed behavior during testing on preview runtimes (February 2026). Results may change with future Foundry Local or driver updates.

Foundry Local handles variant selection by alias. You request phi-4-mini, it pulls the best available build for your hardware. Model availability across NPU execution providers is expanding with each Foundry Local release. For tool-calling workloads on Qualcomm today, Qwen 2.5 7B is fully NPU-accelerated and production-ready. Your AI coding assistant handles the detection and model routing automatically.

Runtime Lifecycle Differs

if SILICON == "qualcomm":
    # Skip warmup - QNN is unstable with rapid reconnection attempts
    print("Skipping warmup on Qualcomm (first request will load model)...")
else:
    # Intel: warmup + keepalive for consistent latency
    warmup_model()
    start_keepalive_thread(interval=180)

Intel benefits from warmup and keepalive pings every 3 minutes. Qualcomm is the opposite: warmup destabilizes the QNN runtime. Let the first real request trigger model load. Use aggressive auto-reconnection on both platforms. These are observations from preview runtimes and may evolve.

NPU-to-GPU Fallback Chain

try:
    manager = FoundryLocalManager("phi-4-mini")  # NPU first
except Exception:
    try:
        from foundry_local.api import DeviceType
        manager = FoundryLocalManager("phi-4-mini", device=DeviceType.GPU)
    except Exception:
        # ⚠️ NOT RECOMMENDED FOR PRODUCTION - last resort only.
        # The port is dynamic; this may break if the service restarts.
        client = OpenAI(
            base_url="http://localhost:5272/v1",
            api_key="not-needed"
        )

For additional troubleshooting, see Foundry Local best practices.

This is the constraint that shapes everything. Context limits vary by model. Phi Silica has a ~4K window; Phi-4 Mini supports much larger contexts, but practical token budgets still matter for latency and cost. Observed ranges: structured field extraction ~1,884 tokens, document classification ~500 tokens, morning briefing ~2,200 tokens.

Design principle: build single-shot endpoints that do one thing well. Instead of “one chatbot that does everything,” build endpoints like /extract-fields, /classify-doc, /summarize. One call, one job, one clean response.

And honestly? This constraint produces better architecture. Not every AI task needs a frontier model. Most of them are casual... summarization, extraction, classification, one-shot and done. It maps directly to the 80/20 planning model: the NPU handles the 80% that doesn’t need GPT-4.

If you don’t need vision capabilities for your first app, skip this section. Start with Foundry Local + Phi-4 Mini. Come back when you need on-device image classification.

Phi Silica requires a few more setup steps than Foundry Local, but it unlocks on-device vision capabilities that no cloud-dependent workflow can match. Phi Silica is Microsoft’s on-device model accessed through the Windows AI APIs, supporting on-device language and multimodal scenarios available on Copilot+ PCs. For the full walkthrough, see Get started with Phi Silica and the Phi Silica tutorial.

Unlike Foundry Local (standard REST API), Phi Silica is a Windows API requiring: MSIX (Microsoft’s modern app packaging format) packaging with the systemAIModels restricted capability, a LAF (Limited Access Feature) token tied to your Package Family Name, and model provisioning on-device. Development builds may relax LAF enforcement; production deployments should always assume a valid token is required. See Windows AI API troubleshooting for LAF details.

Minimum Viable Checklist

• ☐ MSIX-packaged app with systemAIModels capability
• ☐ Phi Silica model readied on-device (AI Toolkit Playground or AI Dev Gallery)
• ☐ App checks GetReadyState() before calling vision APIs
• ☐ LAF token from Microsoft
• ☐ Health endpoint for fallback detection

The Sidecar Pattern

Encapsulate the Windows API complexity in a small C# ASP.NET Core service rather than MSIX-packaging your entire app:

Flask app (localhost:5000)  →  Vision Service (localhost:5100)  →  Phi Silica on NPU
                                      ↑
                               MSIX packaged, LAF token

The Vision Service exposes /health, /classify, /describe, /extract-text. Your primary app stays a standard Python web application.

If there’s one architecture principle to take from this entire post, it’s this: never let the app fail silently. Models will hang. Drivers will update. The NPU will occasionally just... not cooperate.

Build these tiers from the start. They cost almost nothing to implement and they save everything when it matters.

Foundry Local is in preview, so treat everything here as a framework that will evolve. That said, if you’re thinking about fleet deployment, these are the things that will bite you if you don’t plan for them.

Model acquisition and offline readiness. Models download on first use and cache locally (typically %LOCALAPPDATA%\.foundry\cache\models). Cache is per-user context, so plan pre-provisioning accordingly.

Update cadence. No SLA, no backward compatibility guarantee. Test in rings (dev → pilot → broad). Pin runtime versions. Revalidate after every Foundry Local or Windows update.

Logging. Track inference latency, token counts, fallback tier used, model load times. Do not log raw prompts or responses containing user data or PII (personally identifiable information).

Cloud escalation policy. Make cloud routing an explicit, auditable decision: local-by-default, cloud-only-when-invoked, with a log entry every time data leaves the device. Provide an admin toggle to disable cloud escalation entirely for sensitive environments.

Security. Foundry Local binds to localhost. Only local processes can reach the model endpoint. Don’t proxy to external interfaces without explicit security controls.

Packaging. Runtime installs via winget install Microsoft.FoundryLocal (Intune-compatible). (use --scope machine for machine-wide deployment; Intune-compatible). Stage three components: runtime, pre-cached model(s), and your application. The demo repo’s setup.ps1 validates each.

Fleet heterogeneity. If your fleet spans Intel and Qualcomm, the WMI detection pattern and three-tier fallback architecture are required, not optional. Test on both silicon families before broad deployment.

AI Agent: governed local chat assistant powered by Phi-4 Mini through Foundry Local. Natural language queries, tool-calling, structured responses. This is where most people start.

My Day: takes local calendar, email, and task data and generates a structured morning briefing entirely on-device.

Two-Brain Router: evaluates each request and decides whether the local NPU can handle it or if it needs to escalate to a frontier cloud model. Shows the decision logic in real time, asks for explicit user consent before any data leaves the device.

Field Inspection Copilot: the multimodal showcase. Five NPU capabilities in a single workflow:

• Voice: speak inspection findings, NPU transcribes and extracts structured fields (location, issue type, severity)
• Camera: photograph the issue, Phi Silica classifies it on-device (water damage, structural crack, mold, equipment fault)
• Pen: annotate photos with the Surface Pen, with local handwriting recognition
• Report generation: NPU synthesizes voice, photos, and annotations into a formatted inspection report
• Translation: one tap to translate the full report into Spanish entirely on-device

Target industries: construction, insurance claims, utilities, manufacturing QA, property management, OSHA compliance. The offline capability is load-bearing because these environments often have poor or zero connectivity.

The dashboard closes every session with the demo numbers: 5 local AI tasks, 0 cloud calls, ~520 tokens consumed, $0.00 in cloud inference costs, 0 bytes transmitted off-device.

github.com/frankcx1/surface-npu-demo

Clone it. Run setup.ps1. Open http://localhost:5000. It auto-detects your silicon, selects the right model, and you’re running.

Fork it and build your own use case on top of it. Add a tab for your industry workflow. Swap in a different model from the catalog. If you build something cool, submit a PR... I’ll merge it. This is a living project.

Just as important as knowing what to build... knowing what not to build. I learned some of these the hard way.

• Not for model training. NPUs are inference accelerators.
• Not for long-running agent loops. Small models lose coherence. Design for single-shot endpoints.
• Not for unbounded conversation history. Manage state explicitly within the ~4K context window.
• Not a cloud replacement. On-device handles the routine majority so your cloud budget goes to the tasks that need it.

Built on Surface Copilot+ PCs. Repo: github.com/frankcx1/surface-npu-demo