Build an AI-Powered Chat App in Minutes with AI Toolkit
AI Toolkit VS Code extension is a great place for developers to experiment, prototype, and learn about AI workflows. Developers exploring AI often start in "Model Playground" — experimenting with models, testing prompts, and iterating on ideas. But turning those experiments into real applications can take time. That’s why we partnered with the AI Toolkit team to introduce a new project scaffold experience. With just a few clicks, you can now generate a complete AI-powered chat app. From playground to real app — in one click AI Toolkit helps developers learn, prototype, and experiment with AI workflows. Until now, it provided code snippets — great for exploration, but still a few steps away from a runnable project. To bridge that gap, the latest release introduces project scaffolding. Once you’ve explored model responses in Model Playground, click View Code. You’ll now see a new option: > OpenAI SDK. The support for other SDKs are coming. After choosing a local folder, AI Toolkit will scaffold a full project — including both backend and frontend code and you’re ready to run. What’s inside the chat app The scaffolded project, named AI Chat, provides a complete end-to-end example of an AI + real-time chat application. Key features: Multi-room, real-time chat, AI bot replies powered by GitHub LLM models via your token, Same React frontend and Python backend code in both local or cloud modes, (Optional) Azure Web PubSub integration for scale and message broadcasting. You can start locally in minutes — no Azure setup required. When you’re ready to scale, deploy to Azure with no code changes. Run locally in minutes Prerequisites: Python 3.12+ Node.js 18+ GitHub Personal Access Token with Models – Read permission pip install -r requirements.txt export GITHUB_TOKEN=<your_pat> python start_dev.py Then open http://localhost:5173 — and you’re chatting! You can open a second browser window to see real-time message streaming between rooms. From local to cloud: scale without rewrites One of the best parts about this sample is its flexibility. You can run it entirely locally, with no Azure setup or dependencies. Everything — frontend, backend, and real-time messaging — works out of the box. This makes it perfect for experimentation, quick demos, or internal prototypes. But when you’re ready to go beyond local, Azure steps in to take care of scalability, reliability, and lifecycle management — with no code changes. Why run it on Azure? Deploying to Azure offers several advantages: Built-in scalability — Move from a handful of users to thousands of concurrent connections without changing your architecture, Managed infrastructure — Azure App Service, Azure Web PubSub, Azure Storage are fully managed services; you don’t manage servers or maintain WebSocket connections, Security and access control — Use Azure identity integration for production-grade protection, Dev-friendly automation — Provision everything with a single command using Azure Developer CLI (azd). To deploy the sample app to Azure, you only need one command. azd up Everything — including Azure App Service, Azure Web PubSub and Azure Storage — is provisioned automatically. Real-time, managed: Azure Web PubSub At the heart of the cloud setup is Azure Web PubSub, a fully managed service for building real-time, bi-directional messaging applications using WebSockets. Developers can focus on application logic and leaving infra-related concerns to the service. In the AI Chat Demo, Azure Web PubSub powers the real-time messaging and multi-room architecture, while LLMs via GitHub Models handle the intelligence layer. Specifically, Azure Web PubSub handles: Message broadcasting across chat rooms, Group management (join, leave, and isolate rooms), Event handling through CloudEvents for flexible server integration, Client negotiation via tokens for secure, scoped access. This means your chat app can support large numbers of simultaneous users and global traffic — without you managing connection state or scaling infrastructure. Next Steps Try the new project scaffold in AI Toolkit VS Code, Explore advanced options in docs/ADVANCED.md, Deploy your app with: azd up Learn more about Azure Web PubSub Explore, build and scale The new AI Toolkit + Azure Web PubSub experience helps developers go from model exploration to real-time AI application in minutes — no boilerplate, no setup friction. Start experimenting today and bring your AI chat ideas to life.Building Real-Time AI Apps with Model Context Protocol (MCP) and Azure Web PubSub
Overview Model Context Protocol (MCP) is an open, standardized protocol that allows Large Language Models (LLMs) to interact with external tools and data sources in a consistent, extensible way. It gives models access to capabilities beyond their training data—such as calling APIs or querying databases. MCP builds on the idea of function calling, which lets LLMs invoke external operations by generating JSON payloads that match predefined schemas. While function calling was initially tied to proprietary formats (like OpenAI’s schemas), MCP unifies these patterns under a common JSON-RPC-based protocol. This makes it easier for developers to write tools once and expose them to many LLMs, regardless of vendor. In short: Function calling gave LLMs actions. MCP gives those actions structure, discoverability, and interoperability. In this article, we demonstrate how to give LLMs the ability to broadcast messages to connected web clients through MCP. As going through the article, you will discover that when LLMs have to the ability to publish messages it unlocks useful, real-time app scenarios. An LLM as a collaborative partner in co-editing applications (The app scenario this article explains). An LLM as a proactive, smart data analyst that publishes time-sensitive analysis or alerts. About the whiteboard app The whiteboard this article is based on lets multiple users draw with basic shape tools on a shared canvas. The support for integration with LLMs via MCP even allows users to invite AI as an active participant, unlocking new possibilities for interactive teamwork. Technologies used Web frontend: Vue.js app Backend: Node app using Express Sync drawing activities among web clients: Azure Web PubSub MCP server: Implemented in JavaScript MCP host: VS Code (What this article uses, but you are not limited to it.) For the complete code, please visit GitHub repo. Requirements for following along Node.js, VS Code, An Azure Web PubSub resource. Follow this link to create a resource if you don't have one already. Roadmap ahead Considering the many concepts and moving parts, we break down the article into two parts. Part 1: Run the whiteboard app locally Set up and run the whiteboard app (single user) Run the whiteboard app (multiple users) Part 2: Add MCP support Set up a whiteboard MCP server Configure VS Code to discover the MCP server Part 1: Run the whiteboard app locally Run the whiteboard with a single user Clone the repo, change directory into the whiteboard sample, install app dependencies and build the project git clone https://github.com/Azure/azure-webpubsub.git cd azure-webpubsub/samples/javascript/whiteboard npm install npm run build Provide Azure Web PubSub connection string This applications uses Azure Web PubSub to sync drawing activities among web clients. More on that later. For now, since it's a dependency of the app, we need to supply it. The app uses a connection string to authenticate with Azure Web PubSub. Locate your Azure Web PubSub resource on Azure Portal to find the "connection string" under "Settings > Keys". On Linux, set the environment variable: export Web_PubSub_ConnectionString="<connection_string>" Or, on Windows: SET Web_PubSub_ConnectionString="<connection_string>" Start the app npm run start If you inspect the `server.js` file, you will see that this app is an Express project which serves the web frontend at `port:8080` and handles syncing whiteboard drawing activities among users using Azure Web PubSub. We will explain the syncing part shortly. But for now, you can open your web browser and visit `localhost:8080`, you should see the web frontend. After entering a username, you can play with the whiteboard by drawing a few shapes. Syncing drawing activities When the web frontend is successfully loaded, it goes to the `/negotiate` endpoint to fetch an access token from the Express server to connect with Azure Web PubSub service. The underlying connection between a web client and Azure Web PubSub is a WebSocket connection. This persistent connection allows the web client to send and receive messages to and from Azure Web PubSub service. When you draw on the whiteboard, the client code is written in such a way that it sends every action as a message to the `draw` group in Azure Web PubSub service. Upon receiving the message, Azure Web PubSub broadcasts it to all the connected clients in the `draw` group. In effect, Azure Web PubSub syncs the drawing states of the whiteboard for all the users. // Client code snippets from public/src.index.js // The code uses the `sendToGroup` API offered by Azure Web PubSub diagram.onShapeUpdate((i, m) => ws.sendToGroup('draw', { name: 'updateShape', data: [author, i, m] }, "json")); diagram.onShapePatch((i, d) => ws.sendToGroup('draw', { name: 'patchShape', data: [author, i, d] }, "json", {fireAndForget: true})); In the browser's network panel, you can inspect the WebSocket messages. Run the whiteboard with multiple users To simulate multiple users collaborating on the whiteboard locally, you can open another browser tab. You would expect that the drawing activities in one browser tab should be synced on the other since both whiteboard apps or web clients are connected with Azure Web PubSub service. However, you don't see the syncing happening at the moment. There's nothing wrong with your expectations. The gotcha here is that Azure Web PubSub being a cloud service doesn't know how to reach your web frontend which is running on `localhost`. This is a common pain point experienced by Azure Web PubSub customers. To ensure a smoother developer experience, the team introduced a command line tool to expose localhost so that it's reachable from the internet. Install the tool npm install -g /web-pubsub-tunnel-tool Configure the tool Locate your Web PubSub resource on Azure portal and under "Settings > Settings" to create a new hub named `sample_draw` and configure an event handler as such. URL Template: `tunnel:///eventhandler/{event}` User Event Pattern: `message` System Events: `connect`, `connected`, `disconnected` You should see something like this when you're finished with the configuration. Run the awps-tunnel tool export WebPubSubConnectionString="<connection_string>" awps-tunnel run --hub sample_draw --upstream http://localhost:8080 Now when draw on one whiteboard, the drawing activities should be synced between the two browser tabs. Part 2: Add MCP support Now that we managed to get the collaborative whiteboard running, let's invite an LLM as another collaborator. For an LLM to participate on the whiteboard, the LLM needs the drawing capabilities that human users have access to. As mentioned earlier, MCP makes it easy to provide these capabilities to LLMs. MCP follows the familiar client-server architecture. The server offers capabilities for the clients to request. The MCP client and server communicate following a specially defined protocol. Another important concept is MCP host. An MCP host contains a MCP client and has access to LLMs. The MCP host we are going to use is VS Code Copilot. The flow for the whiteboard looks like the following: We use VS Code Copilot as the MCP host. (No work on our end.) We ask an LLM to draw on the whiteboard. (Using natural language, for example, "draw a house with many trees around it".) The LLM decodes the user's intent, looks at the capabilities included in the prompt as additional context, comes up with a drawing sequence. (No work on our end. VS Code Copilot handles the interactions with LLMs.) The MCP client receives the drawing sequence from the LLM. (No work on our end.) The MCP client requests the MCP server to carry out the drawing sequence. (No work on our end.) The MCP server draws on the whiteboard using Azure Web PubSub. (We need to create the MCP server.) As you can see from the items above, the bulk of the work is done for us by VS Code Copilot's support for MCP. We only need to create the MCP server whose responsibility is to fulfill MCP client requests. Set up MCP server In the whiteboard directory, you will find a `mcpserver` directory. Make sure you are at the root of the whiteboard directory. cd mcpserver npm install If you inspect `mcpserver/index.js`, you will see the server makes one tool, `add_or_update_shape`, available. // Code snippet from `mcpserver/index.js` server.tool( "add_or_update_shape", "Add or update a shape on the whiteboard" // Code omitted for brevity ) The callback provided when registering this tool is the code that will get run when a MCP client requests `add_or_udpate_shape`. It uses Azure Web PubSub's `sendToGroup` API, the same as we saw in Part 1. ws.sendToGroup( "draw", { name: "updateShape", data: ["AI", id, message], }, "json", ); That's it. After installing the dependencies, this server is ready to run and serve MCP clients. Since VS Code Copilot can automatically start the server for us, we don't need to do anything other than installing the dependencies. Configure VS Code to discover the whiteboard MCP server VS Code has great support for MCP. In the command panel, follow the UI to add an MCP server. Select `Command (stdio)` as the type of MCP server to add. In our example, the MCP server and MCP client communicates over standard input/output. As you can see from the dropdown, you can also add MCP servers that communicate with MCP clients over HTTP. But that's beyond the scope of this article. Paste in the full path to `mcpserver/index.js`. When you are finished, you will see a new file is created for you. VS Code Copilot will use this file to discover MCP servers and run them if necessary. Since the whiteboard MCP server is a Node project, the command is set to `node` with the path to the server code file as an argument. Open VS Code Copilot panel and switch to agent mode. MCP is available under agent mode. The model I chose is `GPT-4o`. Click on the wrench icon to configure tools. You will see a dropdown appearing from the command panel, which lists the tools VS Code has discovered. After clicking "OK", VS Code will start the MCP server for you if it's not already. You can verify that the whiteboard MCP server is running by listing the servers. Enjoy the fruit of labor If you've come this far in the article, it's time to enjoy our fruit of labor. One thing to highlight is that the actual drawing actions are performed by the MCP servers and the drawing instructions are delivered through Azure Web PubSub service which maintains a persistent connection with every whiteboard web client. VS Code Copilot as the MCP host facilitates the communication among model, MCP Client and MCP Server. Here's a drawing GPT-4o produced for the ask "Draw a meadow with many trees and sheep. Also, make sure there's a winding river running through it." Impressive! Recap In this article, we demonstrated a scenario where an LLM can be invited by human users to participate on a whiteboard. But one can imagine that LLM can take a more proactive role or as people like to say to exhibit more agentic behaviors. Take a typical monitoring system as an example. Existing monitoring systems are based on “fixed metrics” - when certain pre-defined rules are met, alerts are sent out as notifications or to a dashboard. An agent can complem ent the fixed metric approach with a more flexible and potentially more powerful workflow – intelligently analyzing fresh and historical data, deriving insights or alerts, and delivering them to users in real-time.Announcing Serverless Support for Socket.IO in Azure Web PubSub service
Socket.IO Serverless Mode in Azure Web PubSub service is now available. This new mode eliminates the need for developers to maintain persistent connections on their application servers, offering a more streamlined and scalable approach. In addition to the existing default mode, developers can now deploy Socket.IO applications in a serverless environment using Azure FunctionsAnnouncing MQTT Support in Azure Web PubSub Service (public preview)
We're excited to announce the long-requested release of MQTT support in Azure Web PubSub service, now available in preview! This added capability allows developers to leverage the lightweight and widely used MQTT protocol for messaging, making it easier to connect and communicate with devices that have constrained resources.Don't miss the 2024 Azure Developers JavaScript Day!
Do you want to discover the latest services and features in Azure designed specifically for JavaScript developers? Are you looking for cutting-edge cloud development techniques that can save you time and money, while providing your customers with the best experience possible?Socket.IO support on Azure
Socket.IO is a wildly popular open-source library for client-server real-time communication. While Socket.IO user love the intuitive APIs, scaling out a Socket.IO app is not something developers are particularly fond of. I am happy to share that with the input from the open-source community we brought support for Socket.IO on Azure. With this support, Azure hosts and manages client connections so that Socket.IO developers don't have to. For developers who are familiar with Socket.IO, it means you dont need to develop, deploy or maintainer an "adapter" component. Getting a Socket.IO app running locally to Azure takes only a few lines of code.With geo-replica, Web PubSub resources are fault tolerant and can communicate across regions
Azure Web Pub announces the GA status of the geo-replia feature. Having a failover strategy in place is a must for enterprise-grade applications. What used to take much effort and time can now be configured quicky with a few clicks. Plus, end users from geographically distant locations can benefit from low latency they expect from a real-time system.Native support for Socket.IO on Azure, scalability issue no more
This article talks about a popular open-source library called “Socket.IO”. It’s often used to build real-time web applications, like multi-player games and co-editing, co-creation applications. It explores the relationship between WebSocket API, Socket.IO library and an Azure service. WebSocket API – provides the transport-level plumbing of bi-directional communication between clients and server. Socket.IO library – builds on top of WebSocket API and provides application-level features that are common when developing real-time web apps. Azure Web PubSub for Socket.IO – a feature from an Azure service that provides the infrastructure for massive scalability of Socket.IO apps.Azure Web PubSub for Socket.IO is now generally available
TL;DR Socket.IO library is natively supported on Azure. Since we public previewed this feature, we received positive feedback from users. Now we are happy to share that Web PubSub for Socket.IO is generally available, which means that Azure customers can expect stable APIs, SLAs customer support and it’s suitable for use in production. Follow this quickstarts guide to try out the feature. Check out the repo of a collaborative whiteboard app that showcases the use of Socket.IO APIs and how Azure handles scalability challenges.
