OpenAI Agent SDK Integration with Azure Durable Functions
Picture this: Your agent authored with the OpenAI Agent SDK is halfway through analyzing 10,000 customer reviews when it hits a rate limit and dies. All that progress? Gone. Your multi-agent workflow that took 30 minutes to orchestrate? Back to square one because of a rate limit throttle. If you've deployed AI agents in production, you probably know this frustration first-hand. Today, we're announcing a solution that makes your agents reliable: OpenAI Agent SDK Integration with Azure Durable Functions. This integration provides automatic state persistence, enabling your agents to survive any failure and continue exactly where they stopped. No more lost progress, no more starting over, just reliable agents that work. The Challenge with AI Agents Building AI agents that work reliably in production environments has proven to be one of the most significant challenges in modern AI development. As agent sophistication increases with complex workflows involving multiple LLM calls, tool executions, and agent hand-offs, the likelihood of encountering failures increases. This creates a fundamental problem for production AI systems where reliability is essential. Common failure scenarios include: Rate Limiting: Agents halt mid-process when hitting API rate limits during LLM calls Network Timeouts: workflows terminate due to connectivity issues System Crashes: Multi-agent systems fail when individual components encounter errors State Loss: Complex workflows restart from the beginning after any interruption Traditional approaches force developers to choose between building complex retry logic with significant code changes or accepting unreliable agent behavior. Neither option is suitable for production-grade AI systems that businesses depend on and that’s why we’re introducing this integration. Key Benefits of the OpenAI Agent SDK Integration with Azure Durable Functions Our solution leverages durable execution value propositions to address these reliability challenges while preserving the familiar OpenAI Agents Python SDK developer experience. The integration enables agent invocations hosted on Azure Functions to run within durable orchestration contexts where both agent LLM calls and tool calls are executed as durable operations. This integration delivers significant advantages for production AI systems such as: Enhanced Agent Resilience- Built-in retry mechanisms for LLM calls and tool executions enable agents to automatically recover from failures and continue from their last successful step Multi-Agent Orchestration Reliability- Individual agent failures don't crash entire multi-agent workflows, and complex orchestrations maintain state across system restarts Built-in Observability- Monitor agent progress through the Durable Task Scheduler dashboard with enhanced debugging and detailed execution tracking (only applicable when using the Durable Task Scheduler as the Durable Function backend). Seamless Developer Experience- Keep using the OpenAI Agents SDK interface you already know with minimal code changes required to add reliability Distributed Compute and Scalability – Agent workflow automatically scale across multiple compute instances. Core Integration Components: These powerful capabilities are enabled through just a few simple additions to your AI application: durable_openai_agent_orchestrator: Decorator that enables durable execution for agent invocations run_sync: Uses an existing OpenAI Agents SDK API that executes your agent with built-in durability create_activity_tool: Wraps tool calls as durable activities with automatic retry capabilities State Persistence: Maintains agentic workflow state across failures and restarts Hello World Example Let's see how this works in practice. Here's what code written using the OpenAI Agent SDK looks like: import asyncio from agents import Agent, Runner async def main(): agent = Agent( name="Assistant", instructions="You only respond in haikus.", ) result = await Runner.run(agent, "Tell me about recursion in programming.") print(result.final_output) With our added durable integration, it becomes: from agents import Agent, Runner @app.orchestration_trigger(context_name="context") @app.durable_openai_agent_orchestrator # Runs the agent invocation in the context of a durable orchestration def hello_world(context): agent = Agent( name="Assistant", instructions="You only respond in haikus.", ) result = Runner.run_sync(agent, "Tell me about recursion in programming.") # Provides synchronous execution with built-in durability return result.final_output rable Task Scheduler dashboard showcasing the agent LLM call as a durable operation Notice how little actually changed. We added app.durable_openai_agent_orchestrator decorator but your core agent logic stays the same. The run_sync* method provides execution with built-in durability, enabling your agents to automatically recover from failures with minimal code changes. When using the Durable Task Scheduler as your Durable Functions backend, you gain access to a detailed monitoring dashboard that provides visibility into your agent executions. The dashboard displays detailed inputs and outputs for both LLM calls and tool invocations, along with clear success/failure indicators, making it straightforward to diagnose and troubleshoot any unexpected behavior in your agent processes. A note about 'run_sync' In Durable Functions, orchestrators don’t usually benefit from invoking code asynchronously because their role is to define the workflow—tracking state, scheduling activities, and so on—not to perform actual work. When you call an activity, the framework records the decision and suspends the orchestrator until the result is ready. For example, when you call run_sync, the deterministic part of the call completes almost instantly, and the LLM call activity is scheduled for asynchronous execution. Adding extra asynchronous code inside the orchestrator doesn’t improve performance; it only breaks determinism and complicates replay. Reliable Tool Invocation Example For agents requiring tool interactions, there are two implementation approaches. The first option uses the @function_tool decorator from the OpenAI Agent SDK, which executes directly within the context of the durable orchestration. When using this approach, your tool functions must follow durable functions orchestration deterministic constraints. Additionally, since these functions run within the orchestration itself, they may be replayed as part of normal operations, making cost-conscious implementation necessary. from agents import Agent, Runner, function_tool class Weather(BaseModel): city: str temperature_range: str conditions: str @function_tool def get_weather(city: str) -> Weather: """Get the current weather information for a specified city.""" print("[debug] get_weather called") return Weather( city=city, temperature_range="14-20C", conditions="Sunny with wind." ) @app.orchestration_trigger(context_name="context") @app.durable_openai_agent_orchestrator def tools(context): agent = Agent( name="Hello world", instructions="You are a helpful agent.", tools=[get_weather], ) result = Runner.run_sync(agent, input="What's the weather in Tokyo?") return result.final_output The second approach uses the create_activity_tool function, which is designed for non-deterministic code or scenarios where rerunning the tool is expensive (in terms of performance or cost). This approach executes the tool within the context of a durable orchestration activity, providing enhanced monitoring through the Durable Task Scheduler dashboard and ensuring that expensive operations are not unnecessarily repeated during orchestration replays. from agents import Agent, Runner, function_tool class Weather(BaseModel): city: str temperature_range: str conditions: str @app.orchestration_trigger(context_name="context") @app.durable_openai_agent_orchestrator def weather_expert(context): agent = Agent( name="Hello world", instructions="You are a helpful agent.", tools=[ context.create_activity_tool(get_weather) ], ) result = Runner.run_sync(agent, "What is the weather in Tokio?") return result.final_output @app.activity_trigger(input_name="city") async def get_weather(city: str) -> Weather: weather = Weather( city=city, temperature_range="14-20C", conditions="Sunny with wind." ) return weather Leveraging Durable Functions Stateful App Patterns Beyond basic durability of agents, this integration provides access to the full Durable Functions orchestration context, enabling developers to implement sophisticated stateful application patterns when needed, such as: External Event Handling: Use context.wait_for_external_event() for human approvals, external system callbacks, or time-based triggers Fan-out/Fan-in: Coordinate multiple tasks (including sub orchestrations invoking agents) in parallel. Long-running Workflows: Implement workflows that span hours, days, or weeks with persistent state Conditional Logic: Build dynamic agent workflows based on runtime decisions and external inputs Human Interaction and Approval Workflows Example For scenarios requiring human oversight, you can leverage the orchestration context to implement approval workflows: .durable_openai_agent_orchestrator def agent_with_approval(context): # Run initial agent analysis agent = Agent(name="DataAnalyzer", instructions="Analyze the provided dataset") initial_result = Runner.run_sync(agent, context.get_input()) # Wait for human approval before proceeding approval_event = context.wait_for_external_event("approval_received") if approval_event.get("approved"): # Continue with next phase final_agent = Agent(name="Reporter", instructions="Generate final report") final_result = Runner.run_sync(final_agent, initial_result.final_output) return final_result.final_output else: return "Workflow cancelled by user" This flexibility allows you to build sophisticated agentic applications that combine the power of AI agents with enterprise-grade workflow orchestration patterns, all while maintaining the familiar OpenAI Agents SDK experience. Get Started Today This article only scratches the surface of what's possible with the OpenAI Agent SDK integration for Durable Functions The combination of familiar OpenAI Agents SDK patterns with added reliability opens new possibilities for building sophisticated AI systems that can handle real-world production workloads. The integration is designed for a smooth onboarding experience. Begin by selecting one of your existing agents and applying the transformation patterns demonstrated above (often requiring just a few lines of code changes). Documentation: https://aka.ms/openai-agents-with-reliability-docs Sample Applications: https://aka.ms/openai-agents-with-reliability-samplesSearch Less, Build More: Inner Sourcing with GitHub CoPilot and ADO MCP Server
Developers burn cycles context‑switching: opening five repos to find a logging example, searching a wiki for a data masking rule, scrolling chat history for the latest pipeline pattern. Organisations that I speak to are often on the path of transformational platform engineering projects but always have the fear or doubt of "what if my engineers don't use these resources". While projects like Backstage still play a pivotal role in inner sourcing and discoverability I also empathise with developers who would argue "How would I even know in the first place, which modules have or haven't been created for reuse". In this blog we explore how we can ensure organisational standards and developer satisfaction without any heavy lifting on either side, no custom model training, no rewriting or relocating of repositories and no stagnant local data. Using GitHub CoPilot + Azure DevOps MCP server (with the free `code_search` extension) we turn the IDE into an organizational knowledge interface. Instead of guessing or re‑implementing, engineers can start scaffolding projects or solving issues as they would normally (hopefully using CoPilot) and without extra prompting. GitHub CoPilot can lean into organisational standards and ensure recommendations are made with code snippets directly generated from existing examples. What Is the Azure DevOps MCP Server + code_search Extension? MCP (Model Context Protocol) is an open standard that lets agents (like GitHub Copilot) pull in structured, on-demand context from external systems. MCP servers contain natural language explanations of the tools that the agent can utilise allowing dynamic decision making of when to implement certain toolsets over others. The Azure DevOps MCP Server is the ADO Product Team's implementation of that standard. It exposes your ADO environment in a way CoPilot can consume. Out of the box it gives you access to: Projects – list and navigate across projects in your organization. Repositories – browse repos, branches, and files. Work items – surface user stories, bugs, or acceptance criteria. Wiki's – pull policies, standards, and documentation. This means CoPilot can ground its answers in live ADO content, instead of hallucinating or relying only on what’s in the current editor window. The ADO server runs locally from your own machine to ensure that all sensitive project information remains within your secure network boundary. This also means that existing permissions on ADO objects such as Projects or Repositories are respected. Wiki search tooling available out of the box with ADO MCP server is very useful however if I am honest I have seen these wiki's go unused with documentation being stored elsewhere either inside the repository or in a project management tool. This means any tool that needs to implement code requires the ability to accurately search the code stored in the repositories themself. That is where the code_search extension enablement in ADO is so important. Most organisations have this enabled already however it is worth noting that this pre-requisite is the real unlock of cross-repo search. This allows for CoPilot to: Query for symbols, snippets, or keywords across all repos. Retrieve usage examples from code, not just docs. Locate standards (like logging wrappers or retry policies) wherever they live. Back every recommendation with specific source lines. In short: MCP connects CoPilot to Azure DevOps. code_search makes that connection powerful by turning it into a discovery engine. What is the relevance of CoPilot Instructions? One of the less obvious but most powerful features of GitHub CoPilot is its ability to follow instructions files. CoPilot automatically looks for these files and uses them as a “playbook” for how it should behave. There are different types of instructions you can provide: Organisational instructions – apply across your entire workspace, regardless of which repo you’re in. Repo-specific instructions – scoped to a particular repository, useful when one project has unique standards or patterns. Personal instructions – smaller overrides layered on top of global rules when a local exception applies. (Stored in .github/copilot-instructions.md) In this solution, I’m using a single personal instructions file. It tells CoPilot: When to search (e.g., always query repos and wikis before answering a standards question). Where to look (Azure DevOps repos, wikis, and with code_search, the code itself). How to answer (responses must cite the repo/file/line or wiki page; if no source is found, say so). How to resolve conflicts (prefer dated wiki entries over older README fragments). As a small example, a section of a CoPilot instruction file could look like this: # GitHub Copilot Instructions for Azure DevOps MCP Integration This project uses Azure DevOps with MCP server integration to provide organizational context awareness. Always check to see if the Azure DevOps MCP server has a tool relevant to the user's request. ## Core Principles ### 1. Azure DevOps Integration - **Always prioritize Azure DevOps MCP tools** when users ask about: - Work items, stories, bugs, tasks - Pull requests and code reviews - Build pipelines and deployments - Repository operations and branch management - Wiki pages and documentation - Test plans and test cases - Project and team information ### 2. Organizational Context Awareness - Before suggesting solutions, **check existing organizational patterns** by: - Searching code across repositories for similar implementations - Referencing established coding standards and frameworks - Looking for existing shared libraries and utilities - Checking architectural decision records (ADRs) in wikis ### 3. Cross-Repository Intelligence - When providing code suggestions: - **Search for existing patterns** in other repositories first - **Reference shared libraries** and common utilities - **Maintain consistency** with organizational standards - **Suggest reusable components** when appropriate ## Tool Usage Guidelines ### Work Items and Project Management When users mention bugs, features, tasks, or project planning: ``` ✅ Use: wit_my_work_items, wit_create_work_item, wit_update_work_item ✅ Use: wit_list_backlogs, wit_get_work_items_for_iteration ✅ Use: work_list_team_iterations, core_list_projects The result... To test this I created 3 ADO Projects each with between 1-2 repositories. The repositories were light with only ReadMe's inside containing descriptions of the "repo" and some code snippets examples for usage. I have then created a brand-new workspace with no context apart from a CoPilot instructions document (which could be part of a repo scaffold or organisation wide) which tells CoPilot to search code and the wikis across all ADO projects in my demo environment. It returns guidance and standards from all available repo's and starts to use it to formulate its response. In the screenshot I have highlighted some key parts with red boxes. The first being a section of the readme that CoPilot has identified in its response, that part also highlighted within CoPilot chat response. I have highlighted the rather generic prompt I used to get this response at the bottom of that window too. Above I have highlighted CoPilot using the MCP server tooling searching through projects, repo's and code. Finally the largest box highlights the instructions given to CoPilot on how to search and how easily these could be optimised or changed depending on the requirements and organisational coding standards. How did I implement this? Implementation is actually incredibly simple. As mentioned I created multiple projects and repositories within my ADO Organisation in order to test cross-project & cross-repo discovery. I then did the following: Enable code_search - in your Azure DevOps organization (Marketplace → install extension). Login to Azure - Use the AZ CLI to authenticate to Azure with "az login". Create vscode/mcp.json file - Snippet is provided below, the organisation name should be changed to your organisations name. Start and enable your MCP server - In the mcp.json file you should see a "Start" button. Using the snippet below you will be prompted to add your organisation name. Ensure your CoPilot agent has access to the server under "tools" too. View this setup guide for full setup instructions (azure-devops-mcp/docs/GETTINGSTARTED.md at main · microsoft/azure-devops-mcp) Create a CoPilot Instructions file - with a search-first directive. I have inserted the full version used in this demo at the bottom of the article. Experiment with Prompts – Start generic (“How do we secure APIs?”). Review the output and tools used and then tailor your instructions. Considerations While this is a great approach I do still have some considerations when going to production. Latency - Using MCP tooling on every request will add some latency to developer requests. We can look at optimizing usage through copilot instructions to better identify when CoPilot should or shouldn't use the ADO MCP server. Complex Projects and Repositories - While I have demonstrated cross project and cross repository retrieval my demo environment does not truly simulate an enterprise ADO environment. Performance should be tested and closely monitored as organisational complexity increases. Public Preview - The ADO MCP server is moving quickly but is currently still public preview. We have demonstrated in this article how quickly we can make our Azure DevOps content discoverable. While their are considerations moving forward this showcases a direction towards agentic inner sourcing. Feel free to comment below how you think this approach could be extended or augmented for other use cases! Resources MCP Server Config (/.vscode/mcp.json) { "inputs": [ { "id": "ado_org", "type": "promptString", "description": "Azure DevOps organization name (e.g. 'contoso')" } ], "servers": { "ado": { "type": "stdio", "command": "npx", "args": ["-y", "@azure-devops/mcp", "${input:ado_org}"] } } } CoPilot Instructions (/.github/copilot-instructions.md) # GitHub Copilot Instructions for Azure DevOps MCP Integration This project uses Azure DevOps with MCP server integration to provide organizational context awareness. Always check to see if the Azure DevOps MCP server has a tool relevant to the user's request. ## Core Principles ### 1. Azure DevOps Integration - **Always prioritize Azure DevOps MCP tools** when users ask about: - Work items, stories, bugs, tasks - Pull requests and code reviews - Build pipelines and deployments - Repository operations and branch management - Wiki pages and documentation - Test plans and test cases - Project and team information ### 2. Organizational Context Awareness - Before suggesting solutions, **check existing organizational patterns** by: - Searching code across repositories for similar implementations - Referencing established coding standards and frameworks - Looking for existing shared libraries and utilities - Checking architectural decision records (ADRs) in wikis ### 3. Cross-Repository Intelligence - When providing code suggestions: - **Search for existing patterns** in other repositories first - **Reference shared libraries** and common utilities - **Maintain consistency** with organizational standards - **Suggest reusable components** when appropriate ## Tool Usage Guidelines ### Work Items and Project Management When users mention bugs, features, tasks, or project planning: ``` ✅ Use: wit_my_work_items, wit_create_work_item, wit_update_work_item ✅ Use: wit_list_backlogs, wit_get_work_items_for_iteration ✅ Use: work_list_team_iterations, core_list_projects ``` ### Code and Repository Operations When users ask about code, branches, or pull requests: ``` ✅ Use: repo_list_repos_by_project, repo_list_pull_requests_by_repo ✅ Use: repo_list_branches_by_repo, repo_search_commits ✅ Use: search_code for finding patterns across repositories ``` ### Documentation and Knowledge Sharing When users need documentation or want to create/update docs: ``` ✅ Use: wiki_list_wikis, wiki_get_page_content, wiki_create_or_update_page ✅ Use: search_wiki for finding existing documentation ``` ### Build and Deployment When users ask about builds, deployments, or CI/CD: ``` ✅ Use: pipelines_get_builds, pipelines_get_build_definitions ✅ Use: pipelines_run_pipeline, pipelines_get_build_status ``` ## Response Patterns ### 1. Discovery First Before providing solutions, always discover organizational context: ``` "Let me first check what patterns exist in your organization..." → Search code, check repositories, review existing work items ``` ### 2. Reference Organizational Standards When suggesting code or approaches: ``` "Based on patterns I found in your [RepositoryName] repository..." "Following your organization's standard approach seen in..." "This aligns with the pattern established in [TeamName]'s implementation..." ``` ### 3. Actionable Integration Always offer to create or update Azure DevOps artifacts: ``` "I can create a work item for this enhancement..." "Should I update the wiki page with this new pattern?" "Let me link this to the current iteration..." ``` ## Specific Scenarios ### New Feature Development 1. **Search existing repositories** for similar features 2. **Check architectural patterns** and shared libraries 3. **Review related work items** and planning documents 4. **Suggest implementation** based on organizational standards 5. **Offer to create work items** and documentation ### Bug Investigation 1. **Search for similar issues** across repositories and work items 2. **Check related builds** and recent changes 3. **Review test results** and failure patterns 4. **Provide solution** based on organizational practices 5. **Offer to create/update** bug work items and documentation ### Code Review and Standards 1. **Compare against organizational patterns** found in other repositories 2. **Reference coding standards** from wiki documentation 3. **Suggest improvements** based on established practices 4. **Check for reusable components** that could be leveraged ### Documentation Requests 1. **Search existing wikis** for related content 2. **Check for ADRs** and technical documentation 3. **Reference patterns** from similar projects 4. **Offer to create/update** wiki pages with findings ## Error Handling If Azure DevOps MCP tools are not available or fail: 1. **Inform the user** about the limitation 2. **Provide alternative approaches** using available information 3. **Suggest manual steps** for Azure DevOps integration 4. **Offer to help** with configuration if needed ## Best Practices ### Always Do: - ✅ Search organizational context before suggesting solutions - ✅ Reference existing patterns and standards - ✅ Offer to create/update Azure DevOps artifacts - ✅ Maintain consistency with organizational practices - ✅ Provide actionable next steps ### Never Do: - ❌ Suggest solutions without checking organizational context - ❌ Ignore existing patterns and implementations - ❌ Provide generic advice when specific organizational context is available - ❌ Forget to offer Azure DevOps integration opportunities --- **Remember: The goal is to provide intelligent, context-aware assistance that leverages the full organizational knowledge base available through Azure DevOps while maintaining development efficiency and consistency.**Announcing the Public Preview of Azure Container Apps Azure Monitor dashboards with Grafana
We’re thrilled to announce the public preview of Azure Container Apps Azure Monitor Dashboards with Grafana, a major step forward in simplifying observability for your apps and environments. With this new integration, you can view Grafana dashboards directly within your app or environment in the Azure portal, with no extra setup or cost required. What’s new? Azure Monitor Dashboards with Grafana bring the power of Grafana’s visualization capabilities to your Azure resources. Dashboards with Grafana enable you to create and edit Grafana dashboards directly in the Azure portal without additional cost and less administrative overhead compared to self-hosting Grafana or using managed Grafana services. For Azure Container Apps, this means you can access two new pre-built dashboards: Container App View: View key metrics like CPU usage, memory usage, request rates, replica restarts, and more. Environment View: See all your apps in one view with details like latest revision name, minimum and maximum replicas, CPU and memory allocations, and more for each app. These dashboards are designed to help you quickly identify issues, optimize performance, and ensure your applications are running smoothly. Benefits Drill into key metrics: Stop switching between multiple tools or building dashboards from scratch. Start from the environment dashboard to get a high-level view of all of your apps, then drill into individual app dashboards. Customize your views: Tailor the dashboards to your team’s needs using Grafana’s flexible visualization options. Full compatibility with open-source Grafana: Dashboards created in Azure Monitor are portable across any Grafana instance. Share dashboards across your team with Azure Role-Based Access Control (RBAC): Dashboards are native Azure resources, so you can securely share them using RBAC. Get started today For Azure Container Apps, you can experience these dashboards directly from either your environment or an individual app: Navigate to your Azure Container App environment or a specific Container App in the Azure portal. Open the Monitoring section and select the “Dashboards with Grafana (Preview)” blade. View your metrics or customize the dashboard to meet your needs. For detailed guidance, see aka.ms/aca/grafana Want more? Explore the Grafana Gallery Looking for additional customization or inspiration? Visit the Grafana Dashboard Gallery to explore thousands of community dashboards. If you prefer to use Azure Managed Grafana, here are direct links to Azure Container Apps templates: Azure / Container Apps / Container App View Azure / Container Apps / Aggregate View You can also view other published Azure dashboards here.Announcing Early Preview: BYO Remote MCP Server on Azure Functions
If you’ve already built Model Context Protocol (MCP) servers with the MCP SDKs and wished you could turn them into world class Remote MCP servers using a hyperscale, serverless platform, then this one’s for you! We’ve published samples showing how to host bring‑your-own (BYO) Remote MCP servers on Azure Functions, so you can run the servers you’ve already built with the MCP SDKs—Python, Node, and .NET—with minimal changes and full serverless goodness. Why this is exciting Keep your code. If you’ve already implemented servers with the MCP SDKs (Python, Node, .NET), deploy them to Azure Functions as remote MCP servers with just one line of code change. Serverless scale when you need it. Functions on the Flex Consumption plan handles bursty traffic, scales out and back to zero automatically, and gives you serverless billing. Secure by default. Your remote server endpoint is protected with function keys out-of- the-box, with option to layer on Azure API Management for added authorization flow. BYO vs. Functions Remote MCP extension—pick the path that fits The BYO option complements the existing Azure Functions MCP extension: Build and host with Functions MCP extension: You can build stateful MCP servers with the MCP tool trigger and binding and host them on Functions. Support for SSE is available today with streamable HTTP coming soon. Host BYO remote MCP Server (this announcement): If you already have a server built with the MCP SDKs, or you prefer those SDKs’ ergonomics, host it as‑is on Functions and keep your current codebase. Either way, you benefit from Functions’ serverless platform: secure access & auth, burst scale, event-driven scale from 0 to N, and pay-for-what-you‑use. What’s supported in this early preview Servers built with the Python, Node, and .NET SDKs Debug locally with func start on Visual Studio or Visual Studio Code; deploy with the Azure Developer CLI (azd up) to get your remote MCP server quickly deployed to Azure Functions Stateless servers using the streamable HTTP transport, with guidance coming soon for stateful servers Hosting on Flex Consumption plan Try it now! Python: https://github.com/Azure-Samples/mcp-sdk-functions-hosting-python Node: https://github.com/Azure-Samples/mcp-sdk-functions-hosting-node .NET: https://github.com/Azure-Samples/mcp-sdk-functions-hosting-dotnet Each repo includes the sample weather MCP server implemented with the MCP SDK for that language. You’ll find instructions on how to run the server locally with Azure Functions Core Tools and deploy with azd up in minutes. Once deployed, you can connect to the remote server from an MCP client. The samples use Visual Studio Code, but other clients like Claude can also be used. Provide feedback to shape feature Tell us what you need next - identity flows, diagnostics, more languages, or any other features. Your feedback will shape how we take this early preview to the next level!Announcing Native Azure Functions Support in Azure Container Apps
Azure Container Apps is introducing a new, streamlined method for running Azure Functions directly in Azure Container Apps (ACA). This integration allows you to leverage the full features and capabilities of Azure Container Apps while benefiting from the simplicity of auto-scaling provided by Azure Functions. With the new native hosting model, you can deploy Azure Functions directly onto Azure Container Apps using the Microsoft.App resource provider by setting “kind=functionapp” property on the container app resource. You can deploy Azure Functions using ARM templates, Bicep, Azure CLI, and the Azure portal. Get started today and explore the complete feature set of Azure Container Apps, including multi-revision management, easy authentication, metrics and alerting, health probes and many more. To learn more, visit: https://aka.ms/fnonacav2Building Real-Time AI Apps with Model Context Protocol (MCP) and Azure Web PubSub
Overview Model Context Protocol (MCP) is an open, standardized protocol that allows Large Language Models (LLMs) to interact with external tools and data sources in a consistent, extensible way. It gives models access to capabilities beyond their training data—such as calling APIs or querying databases. MCP builds on the idea of function calling, which lets LLMs invoke external operations by generating JSON payloads that match predefined schemas. While function calling was initially tied to proprietary formats (like OpenAI’s schemas), MCP unifies these patterns under a common JSON-RPC-based protocol. This makes it easier for developers to write tools once and expose them to many LLMs, regardless of vendor. In short: Function calling gave LLMs actions. MCP gives those actions structure, discoverability, and interoperability. In this article, we demonstrate how to give LLMs the ability to broadcast messages to connected web clients through MCP. As going through the article, you will discover that when LLMs have to the ability to publish messages it unlocks useful, real-time app scenarios. An LLM as a collaborative partner in co-editing applications (The app scenario this article explains). An LLM as a proactive, smart data analyst that publishes time-sensitive analysis or alerts. About the whiteboard app The whiteboard this article is based on lets multiple users draw with basic shape tools on a shared canvas. The support for integration with LLMs via MCP even allows users to invite AI as an active participant, unlocking new possibilities for interactive teamwork. Technologies used Web frontend: Vue.js app Backend: Node app using Express Sync drawing activities among web clients: Azure Web PubSub MCP server: Implemented in JavaScript MCP host: VS Code (What this article uses, but you are not limited to it.) For the complete code, please visit GitHub repo. Requirements for following along Node.js, VS Code, An Azure Web PubSub resource. Follow this link to create a resource if you don't have one already. Roadmap ahead Considering the many concepts and moving parts, we break down the article into two parts. Part 1: Run the whiteboard app locally Set up and run the whiteboard app (single user) Run the whiteboard app (multiple users) Part 2: Add MCP support Set up a whiteboard MCP server Configure VS Code to discover the MCP server Part 1: Run the whiteboard app locally Run the whiteboard with a single user Clone the repo, change directory into the whiteboard sample, install app dependencies and build the project git clone https://github.com/Azure/azure-webpubsub.git cd azure-webpubsub/samples/javascript/whiteboard npm install npm run build Provide Azure Web PubSub connection string This applications uses Azure Web PubSub to sync drawing activities among web clients. More on that later. For now, since it's a dependency of the app, we need to supply it. The app uses a connection string to authenticate with Azure Web PubSub. Locate your Azure Web PubSub resource on Azure Portal to find the "connection string" under "Settings > Keys". On Linux, set the environment variable: export Web_PubSub_ConnectionString="<connection_string>" Or, on Windows: SET Web_PubSub_ConnectionString="<connection_string>" Start the app npm run start If you inspect the `server.js` file, you will see that this app is an Express project which serves the web frontend at `port:8080` and handles syncing whiteboard drawing activities among users using Azure Web PubSub. We will explain the syncing part shortly. But for now, you can open your web browser and visit `localhost:8080`, you should see the web frontend. After entering a username, you can play with the whiteboard by drawing a few shapes. Syncing drawing activities When the web frontend is successfully loaded, it goes to the `/negotiate` endpoint to fetch an access token from the Express server to connect with Azure Web PubSub service. The underlying connection between a web client and Azure Web PubSub is a WebSocket connection. This persistent connection allows the web client to send and receive messages to and from Azure Web PubSub service. When you draw on the whiteboard, the client code is written in such a way that it sends every action as a message to the `draw` group in Azure Web PubSub service. Upon receiving the message, Azure Web PubSub broadcasts it to all the connected clients in the `draw` group. In effect, Azure Web PubSub syncs the drawing states of the whiteboard for all the users. // Client code snippets from public/src.index.js // The code uses the `sendToGroup` API offered by Azure Web PubSub diagram.onShapeUpdate((i, m) => ws.sendToGroup('draw', { name: 'updateShape', data: [author, i, m] }, "json")); diagram.onShapePatch((i, d) => ws.sendToGroup('draw', { name: 'patchShape', data: [author, i, d] }, "json", {fireAndForget: true})); In the browser's network panel, you can inspect the WebSocket messages. Run the whiteboard with multiple users To simulate multiple users collaborating on the whiteboard locally, you can open another browser tab. You would expect that the drawing activities in one browser tab should be synced on the other since both whiteboard apps or web clients are connected with Azure Web PubSub service. However, you don't see the syncing happening at the moment. There's nothing wrong with your expectations. The gotcha here is that Azure Web PubSub being a cloud service doesn't know how to reach your web frontend which is running on `localhost`. This is a common pain point experienced by Azure Web PubSub customers. To ensure a smoother developer experience, the team introduced a command line tool to expose localhost so that it's reachable from the internet. Install the tool npm install -g /web-pubsub-tunnel-tool Configure the tool Locate your Web PubSub resource on Azure portal and under "Settings > Settings" to create a new hub named `sample_draw` and configure an event handler as such. URL Template: `tunnel:///eventhandler/{event}` User Event Pattern: `message` System Events: `connect`, `connected`, `disconnected` You should see something like this when you're finished with the configuration. Run the awps-tunnel tool export WebPubSubConnectionString="<connection_string>" awps-tunnel run --hub sample_draw --upstream http://localhost:8080 Now when draw on one whiteboard, the drawing activities should be synced between the two browser tabs. Part 2: Add MCP support Now that we managed to get the collaborative whiteboard running, let's invite an LLM as another collaborator. For an LLM to participate on the whiteboard, the LLM needs the drawing capabilities that human users have access to. As mentioned earlier, MCP makes it easy to provide these capabilities to LLMs. MCP follows the familiar client-server architecture. The server offers capabilities for the clients to request. The MCP client and server communicate following a specially defined protocol. Another important concept is MCP host. An MCP host contains a MCP client and has access to LLMs. The MCP host we are going to use is VS Code Copilot. The flow for the whiteboard looks like the following: We use VS Code Copilot as the MCP host. (No work on our end.) We ask an LLM to draw on the whiteboard. (Using natural language, for example, "draw a house with many trees around it".) The LLM decodes the user's intent, looks at the capabilities included in the prompt as additional context, comes up with a drawing sequence. (No work on our end. VS Code Copilot handles the interactions with LLMs.) The MCP client receives the drawing sequence from the LLM. (No work on our end.) The MCP client requests the MCP server to carry out the drawing sequence. (No work on our end.) The MCP server draws on the whiteboard using Azure Web PubSub. (We need to create the MCP server.) As you can see from the items above, the bulk of the work is done for us by VS Code Copilot's support for MCP. We only need to create the MCP server whose responsibility is to fulfill MCP client requests. Set up MCP server In the whiteboard directory, you will find a `mcpserver` directory. Make sure you are at the root of the whiteboard directory. cd mcpserver npm install If you inspect `mcpserver/index.js`, you will see the server makes one tool, `add_or_update_shape`, available. // Code snippet from `mcpserver/index.js` server.tool( "add_or_update_shape", "Add or update a shape on the whiteboard" // Code omitted for brevity ) The callback provided when registering this tool is the code that will get run when a MCP client requests `add_or_udpate_shape`. It uses Azure Web PubSub's `sendToGroup` API, the same as we saw in Part 1. ws.sendToGroup( "draw", { name: "updateShape", data: ["AI", id, message], }, "json", ); That's it. After installing the dependencies, this server is ready to run and serve MCP clients. Since VS Code Copilot can automatically start the server for us, we don't need to do anything other than installing the dependencies. Configure VS Code to discover the whiteboard MCP server VS Code has great support for MCP. In the command panel, follow the UI to add an MCP server. Select `Command (stdio)` as the type of MCP server to add. In our example, the MCP server and MCP client communicates over standard input/output. As you can see from the dropdown, you can also add MCP servers that communicate with MCP clients over HTTP. But that's beyond the scope of this article. Paste in the full path to `mcpserver/index.js`. When you are finished, you will see a new file is created for you. VS Code Copilot will use this file to discover MCP servers and run them if necessary. Since the whiteboard MCP server is a Node project, the command is set to `node` with the path to the server code file as an argument. Open VS Code Copilot panel and switch to agent mode. MCP is available under agent mode. The model I chose is `GPT-4o`. Click on the wrench icon to configure tools. You will see a dropdown appearing from the command panel, which lists the tools VS Code has discovered. After clicking "OK", VS Code will start the MCP server for you if it's not already. You can verify that the whiteboard MCP server is running by listing the servers. Enjoy the fruit of labor If you've come this far in the article, it's time to enjoy our fruit of labor. One thing to highlight is that the actual drawing actions are performed by the MCP servers and the drawing instructions are delivered through Azure Web PubSub service which maintains a persistent connection with every whiteboard web client. VS Code Copilot as the MCP host facilitates the communication among model, MCP Client and MCP Server. Here's a drawing GPT-4o produced for the ask "Draw a meadow with many trees and sheep. Also, make sure there's a winding river running through it." Impressive! Recap In this article, we demonstrated a scenario where an LLM can be invited by human users to participate on a whiteboard. But one can imagine that LLM can take a more proactive role or as people like to say to exhibit more agentic behaviors. Take a typical monitoring system as an example. Existing monitoring systems are based on “fixed metrics” - when certain pre-defined rules are met, alerts are sent out as notifications or to a dashboard. An agent can complem ent the fixed metric approach with a more flexible and potentially more powerful workflow – intelligently analyzing fresh and historical data, deriving insights or alerts, and delivering them to users in real-time.Azure Functions Flex Consumption is now generally available
We are excited to announce that Azure Functions Flex Consumption is now generally available. This hosting plan provides the highest performance for Azure Functions with concurrency-based scaling for both HTTP and non-HTTP triggers, scale from zero to 1000 instances, and no cold start with the Always Ready feature. Flex Consumption also allows you to enjoy seamless integration with your virtual network at no extra cost, ensuring secure and private communication, with no considerable impact to your app’s scale out performance. Learn more about How to achieve high HTTP scale with Azure Functions Flex Consumption, the engineering innovation behind it, and project Legion, the platform behind Flex Consumption. In addition to the fast scaling based on per-instance concurrency, you can choose between 2048MB and 4096MB instance sizes. As the function app receives requests it will automatically scale from zero to as many instances of that instance size as needed based on per instance concurrency, and back to zero for cost efficiency when there’s no more requests to process. You can also take advantage of the built-in integration with Azure Load Testing and the Performance Optimizer to optimize your HTTP functions for performance and cost. Flex Consumption is now generally available for .NET 8 on the isolated worker model, Java 11, Java 17, Node 20, PowerShell 7.4, Python 3.10, and Python 3.11 in Australia East, East Asia, East US, North Europe, Southeast Asia, Sweden Central, UK South, and West US 2, and in preview in East US 2, South Central US, and West US 3. By December 9th 2024, .NET 9 will also generally available in Australia East, East Asia, East US, North Europe, Southeast Asia, Sweden Central, and UK South. Besides the currently supported DevOps and dev tools like VS Code, Java tooling, Azure Pipeline tasks, and GitHub Actions, you can now use the latest Visual Studio 2022 v17.12 update or newer to create and publish to Flex Consumption apps. The Flex Consumption plan offers competitive pricing with flexible options to fit your needs, with GA pricing taking effect on December 1, 2024. For detailed pricing information, please refer to the pricing page. Customer adoption and scenarios We have been working with several internal and external customers during the public preview period, with hundreds of external customers actively using Flex Consumption. “ At Yggdrasil, we immediately started adopting Flex Consumption functions when they went into public preview, as they offer the combination of cost-efficiency, scalability, and security features we need to run our company. We already have 100 Flex Consumption functions running in production, and expect to move at least another 50 functions, now that the product has reached GA. We migrated to Flex from consumption to have VNet integration and private endpoints. – Andreas Strandfelt, Partner & Senior Cloud Specialist at Yggdrasil Commodities ApS “ What really matters to us is that the app scales up and down based on demand. Azure Functions Flex Consumption is very appealing to us because of how it dynamically scales based on the number of messages that are queued up in Azure Event Hubs – Stephan Miehe, GitHub Senior Director. Public case study “ Microsoft AI We had a need to process a large queue, representing a significant volume of data with inconsistent availability. Azure Functions Flex Consumption dramatically simplified the code footprint needed to perform this embarrassingly parallel task and helped us complete it in a much shorter timeframe that we had expected. – Craig Presti, Office of the CTO, Microsoft AI project “ Going Forward In the upcoming months we look forward to rolling out even more features to Flex Consumption, including: Availability zones: Enabling availability zones will be possible for new and existing Flex Consumption apps 512 MB instance size: We will introduce a new, smaller instance size for more granular control Enhanced tooling support: PowerShell modules and Terraform AzureRM support New language versions: Support for the latest language versions like Node 22, Python 3.12, and Java 21 Expanded regional availability: The number of regions will continue to expand in early 2025 with UAE North, Centra US, West US 3, South Central US, East US 2, West US, Canada Central, France Central, and Norway East coming first Metrics support: Full Azure Monitor metrics support for Flex Consumption apps Deployment improvements: Zero-downtime deployment to ensure no disruption to running executions More triggers: Kafka and SQL triggers Closing features: Addressing the limitations identified in Considerations. Please let us know which ones are most important to you! Get Started! Explore our reference samples, quickstarts, and comprehensive documentation to get started with the Azure Functions Flex Consumption hosting plan today!Announcing Public Preview of Azure Functions Flex Consumption
We're thrilled to announce the Public Preview of Azure Functions Flex Consumption, a new Linux-based hosting plan which offers the features in Consumption that you have been waiting for: networking and fast scaling features on a serverless billing model!
