Build an AI-Powered Chat App in Minutes with AI Toolkit

AI Toolkit VS Code extension is a great place for developers to experiment, prototype, and learn about AI workflows. Developers exploring AI often start in "Model Playground" — experimenting with models, testing prompts, and iterating on ideas. But turning those experiments into real applications can take time.

That’s why we partnered with the AI Toolkit team to introduce a new project scaffold experience. With just a few clicks, you can now generate a complete AI-powered chat app.

From playground to real app — in one click

AI Toolkit helps developers learn, prototype, and experiment with AI workflows. Until now, it provided code snippets — great for exploration, but still a few steps away from a runnable project.

To bridge that gap, the latest release introduces project scaffolding. Once you’ve explored model responses in Model Playground, click View Code. You’ll now see a new option:

View Code -> OpenAI SDK. The support for other SDKs are coming.

Select "Scaffold a chat application"

After choosing a local folder, AI Toolkit will scaffold a full project — including both backend and frontend code and you’re ready to run.

What’s inside the chat app

The scaffolded project, named AI Chat, provides a complete end-to-end example of an AI + real-time chat application.

VS Code on the left and the UI of the chat app on the right

Key features:

• Multi-room, real-time chat,
• AI bot replies powered by GitHub LLM models via your token,
• Same React frontend and Python backend code in both local or cloud modes,
• (Optional) Azure Web PubSub integration for scale and message broadcasting.

You can start locally in minutes — no Azure setup required. When you’re ready to scale, deploy to Azure with no code changes.

Run locally in minutes

Prerequisites:

• Python 3.12+
• Node.js 18+
• GitHub Personal Access Token with Models – Read permission

pip install -r requirements.txt
export GITHUB_TOKEN=<your_pat>

python start_dev.py

Then open http://localhost:5173 — and you’re chatting!

You can open a second browser window to see real-time message streaming between rooms.

I use Edge Browser "Split screen" feature here.

From local to cloud: scale without rewrites

One of the best parts about this sample is its flexibility.

You can run it entirely locally, with no Azure setup or dependencies. Everything — frontend, backend, and real-time messaging — works out of the box. This makes it perfect for experimentation, quick demos, or internal prototypes.

But when you’re ready to go beyond local, Azure steps in to take care of scalability, reliability, and lifecycle management — with no code changes.

Why run it on Azure?

Deploying to Azure offers several advantages:

• Built-in scalability — Move from a handful of users to thousands of concurrent connections without changing your architecture,
• Managed infrastructure — Azure App Service, Azure Web PubSub, Azure Storage are fully managed services; you don’t manage servers or maintain WebSocket connections,
• Security and access control — Use Azure identity integration for production-grade protection,
• Dev-friendly automation — Provision everything with a single command using Azure Developer CLI (azd).

To deploy the sample app to Azure, you only need one command. 

azd up

Everything — including Azure App Service, Azure Web PubSub and Azure Storage — is provisioned automatically.

Real-time, managed: Azure Web PubSub

At the heart of the cloud setup is Azure Web PubSub, a fully managed service for building real-time, bi-directional messaging applications using WebSockets. Developers can focus on application logic and leaving infra-related concerns to the service.

In the AI Chat Demo, Azure Web PubSub powers the real-time messaging and multi-room architecture, while LLMs via GitHub Models handle the intelligence layer. Specifically, Azure Web PubSub handles:

• Message broadcasting across chat rooms,
• Group management (join, leave, and isolate rooms),
• Event handling through CloudEvents for flexible server integration,
• Client negotiation via tokens for secure, scoped access.

This means your chat app can support large numbers of simultaneous users and global traffic — without you managing connection state or scaling infrastructure.

Next Steps

1. Try the new project scaffold in AI Toolkit VS Code,
2. Explore advanced options in docs/ADVANCED.md,
3. Deploy your app with:

azd up

4. Learn more about Azure Web PubSub

Explore, build and scale

The new AI Toolkit + Azure Web PubSub experience helps developers go from model exploration to real-time AI application in minutes — no boilerplate, no setup friction.

Start experimenting today and bring your AI chat ideas to life.