Upgrade your voice agent with Azure AI Voice Live API
Today, we are excited to announce the general availability of Voice Live API, which enables real-time speech-to-speech conversational experience through a unified API powered by generative AI models. With Voice Live API, developers can easily voice-enable any agent built with the Azure AI Foundry Agent Service. Azure AI Foundry Agent Service, enables the operation of agents that make decisions, invoke tools, and participate in workflows across development, deployment, and production. By eliminating the need to stitch together disparate components, Voice Live API offers a low latency, end-to-end solution for voice-driven experiences. As always, a diverse range of customers provided valuable feedback during the preview period. Along with announcing general availability, we are also taking this opportunity to address that feedback and improve the API. Following are some of the new features designed to assist developers and enterprises in building scalable, production-ready voice agents. More natively integrated GenAI models including GPT-Realtime Voice Live API enables developers to select from a range of advanced AI models designed for conversational applications, such as GPT-Realtime, GPT-5, GPT-4.1, Phi, and others. These models are natively supported and fully managed, eliminating the need for developers to manage model deployment or plan for capacity. These natively supported models may each have a distinct stage in their life cycle (e.g. public preview, generally available) and be subject to varying pricing structures. The table below lists the models supported in each pricing tier. Pricing Tier Generally Available In Public Preview Voice Live Pro GPT-Realtime, GPT-4.1, GPT-4o GPT-5 Voice Live Standard GPT-4o-mini, GPT-4.1-mini GPT-4o-Mini-Realtime, GPT-5-mini Voice Live Lite NA Phi-4-MM-Realtime, GPT-5-Nano, Phi-4-Mini Extended speech languages to 140+ Voice Live API now supports speech input in over 140 languages/locales. View all supported languages by configuration. Automatic multilingual configuration is enabled by default, using the multilingual model. Integrated with Custom Speech Developers need customization to better manage input and output for different use cases. Besides the support for Custom Voice released in May 2025, Voice Live now supports seamless integration with Custom Speech for improved speech recognition results. Developers can also improve speech input accuracy with phrase lists and refine speech synthesis pronunciation using custom lexicons, all without training a model. Learn how to customize speech and voice models for Voice Live API. Natural HD voices upgraded Neural HD voices in Azure AI Speech are contextually aware and engineered to provide a natural, expressive experience, making them ideal for voice agent applications. The latest V2 upgrade enhances lifelike qualities with features such as natural pauses, filler words, and seamless transitions between speaking styles, all available with Voice Live API. Check out the latest demo of Ava Neural HD V2. Improved VAD features for interruption detection Voice Live API now features semantic Voice Activity Detection (VAD), enabling it to intelligently recognize pauses and filler word interruptions in conversations. In the latest en-US evaluation on Multilingual filler words data, Voice Live API achieved ~20% relative improvement from previous VAD models. This leap in performance is powered by integrating semantic VAD into the n-best pipeline, allowing the system to better distinguish meaningful speech from filler noise and enabling more accurate latency tracking and cleaner segmentation, especially in multilingual and noisy environments. 4K avatar support Voice Live API enables efficient integration with streaming avatars. With the latest updates, avatar options offer support for high-fidelity 4K video models. Learn more about the avatar update. Improved function calling and integration with Azure AI Foundry Agent Service Voice Live API enables function calling to assist developers in building robust voice agents with their chosen generative AI models. This release improves asynchronous function calls and enhances integration with Azure AI Foundry Agent Service for agent creation and operation. Learn more about creating a voice live real-time voice agent with Azure AI Foundry Agent Service. More developer resources and availability in more regions Developer resources are available in C# and Python, with more to come. Get started with Voice Live API. Voice Live API is available in more regions now including Australia East, East US, Japan East, and UK South, besides the previously supported regions such as Central India, East US 2, South East Asia, Sweden Central, and West US 2. Check the features supported in each region. Customers adopting Voice Live In healthcare, patient experience is always the top priority. With Voice Live, eClinicalWorks’ healow Genie contact center solution is now taking healthcare modernization a step further. healow is piloting Voice Live API for Genie to inform patients about their upcoming appointments, answer common questions, and return voicemails. Reducing these routine calls saves healthcare staff hours each day and boosts patient satisfaction through timely interactions. “We’re looking forward to using Azure AI Foundry Voice Live API so that when a patient calls, Genie can detect the question and respond in a natural voice in near-real time,” said Sidd Shah, Vice President of Strategy & Business Growth at healow. “The entire roundtrip is all happening in Voice Live API.” If a patient asks about information in their medical chart, Genie can also fetch data from their electronic health record (EHR) and provide answers. Read the full story here. “If we did multiple hops to go across different infrastructures, that would add up to a diminished patient experience. The Azure AI Foundry Voice Live API is integrated into one single, unified solution, delivering speech-to-text and text-to-speech in the same infrastructure.” Bhawna Batra, VP of Engineering at eClinicalWorks Capgemini, a global business and technology transformation partner, is reimagining its global service desk managed operations through its Capgemini Cloud Infrastructure Services (CIS) division. The first phase covers 500,000 users across 45 clients, which is only part of the overall deployment base. The goal is to modernize the service desk to meet changing expectations for speed, personalization, and scale. To drive this transformation, Capgemini launched the “AI-Powered Service Desk” platform powered by Microsoft technologies including Dynamics 365 Contact Center, Copilot Studio, and Azure AI Foundry. A key enhancement was the integration of Voice Live API for real-time voice interactions, enabling intelligent, conversational support across telephony channels. The new platform delivers a more agile, truly conversational, AI-driven service experience, automating routine tasks and enhancing agent productivity. With scalable voice capabilities and deep integration across Microsoft’s ecosystem, Capgemini is positioned to streamline support operations, reduce response times, and elevate customer satisfaction across its enterprise client base. "Integrating Microsoft’s Voice Live API into our platform has been transformative. We’re seeing measurable improvements in user engagement and satisfaction thanks to the API’s low-latency, high-quality voice interactions. As a result, we are able to deliver more natural and responsive experiences, which have been positively received by our customers.” Stephen Hilton, EVP Chief Operating Officer at CIS Capgemini Astra Tech, a fast-growing UAE-based technology group part of G42, is bringing Voice Live API to its flagship platform, botim, a fintech-first and AI-native platform. Eight out of 10 smartphone users in the UAE already rely on the app. The company is now reshaping botim from a communications tool into a fintech-first service, adding features such as digital wallets, international remittances, and micro-loans. To achieve its broader vision, Astra Tech set out to make botim simpler, more intuitive, and more human. “Voice removes a lot of complexity, and it’s the most natural way to interact,” says Frenando Ansari, Lead Product Manager at Astra Tech. “For users with low digital literacy or language barriers, tapping through traditional interfaces can be difficult. Voice personalizes the experience and makes it accessible in their preferred language.” " The Voice Live API acts as a connective tissue for AI-driven conversation across every layer of the app. It gives us a standardized framework so that different product teams can incorporate voice without needing to hire deep AI expertise.” Frenando Ansari, Lead Product Manager at Astra Tech “The most impressive thing about the Voice Live API is the voice activity detection and the noise control algorithm.” Meng Wang, AI Head at Astra Tech Get started Voice Live API is transforming how developers build voice-enabled agent systems by providing an integrated, scalable, and efficient solution. By combining speech recognition, generative AI, and text-to-speech functionalities into a unified interface, it addresses the challenges of traditional implementations, enabling faster development and superior user experiences. From streamlining customer service to enhancing education and public services, the opportunities are endless. The future of voice-first solutions is here—let’s build it together! Voice Live API introduction (video) Try Voice Live in Azure AI Foundry Voice Live API documents Voice Live quickstart Voice Live Agent code sample in GitHub
Azure AI Foundry: Advancing OpenTelemetry and delivering unified multi-agent observability
Microsoft is enhancing multi-agent observability by introducing new semantic conventions to OpenTelemetry, developed collaboratively with Outshift, Cisco’s incubation engine. These additions—built upon OpenTelemetry and W3C Trace Context—establish standardized practices for tracing and telemetry within multi-agent systems, facilitating consistent logging of key metrics for quality, performance, safety, and cost. This systematic approach enables more comprehensive visibility into multi-agent workflows, including tool invocations and collaboration. These advancements have been integrated into Azure AI Foundry, Microsoft Agent Framework, Semantic Kernel, and Azure AI packages for LangChain, LangGraph, and the OpenAI Agents SDK, enabling customers to get unified observability for agentic systems built using any of these frameworks with Azure AI Foundry. The additional semantic conventions and integration across different frameworks equip developers to monitor, troubleshoot, and optimize their AI agents in a unified solution with increased efficiency and valuable insights. “Outshift, Cisco's Incubation Engine, worked with Microsoft to add new semantic conventions in OpenTelemetry. These conventions standardize multi-agent observability and evaluation, giving teams comprehensive insights into their AI systems.” Giovanna Carofiglio, Distinguished Engineer, Cisco Multi-agent observability challenges Multi-agent systems involve multiple interacting agents with diverse roles and architectures. Such systems are inherently dynamic—they adapt in real time by decomposing complex tasks into smaller, manageable subtasks and distributing them across specialized agents. Each agent may invoke multiple tools, often in parallel or sequence, to fulfill its part of the task, resulting in emergent workflows that are non-linear and highly context dependent. Given the dynamic nature of multi-agent systems and the management across agents, observability becomes critical for debugging, performance tuning, security, and compliance for such systems. Multi-agent observability presents unique challenges that traditional GenAI telemetry standards fail to address. Traditional observability conventions are optimized for single-agent reasoning path visibility and lack the semantic depth to capture collaborative workflows across multiple agents. In multi-agent systems, tasks are often decomposed and distributed dynamically, requiring visibility into agent roles, task hierarchies, tool usage, and decision-making processes. Without standardized task spans and a unified namespace, it's difficult to trace cross-agent coordination, evaluate task outcomes, or analyze retry patterns. These gaps hinder white-box observability, making it hard to assess performance, safety, and quality across complex agentic workflows. Extending OpenTelemetry with multi-agent observability Microsoft has proposed new spans and attributes to OpenTelemetry semantic convention for GenAI agent and framework spans. They will enrich insights and capture the complexity of agent, tool, tasks and plans interactions in multi-agent systems. Below is a list of all the new additions proposed to OpenTelemetry. New Span/Trace/Attributes Name Purpose New span execute_task Captures task planning and event propagation, providing insights into how tasks are decomposed and distributed. New child spans under “invoke_agent” agent_to_agent_interaction Traces communication between agents agent.state.management Effective context, short or long term memory management agent_planning Logs the agent’s internal planning steps agent orchestration Capture agent-to-agent orchestration New attributes in invoke_agent span tool_definitions Describes the tool’s purpose or configuration llm_spans Records model call spans New attributes in “execute_tool” span tool.call.arguments Logs the arguments passed during tool invocation tool.call.results Records the results returned by the tool New event Evaluation - attributes (name, error.type, label) Enables structured evaluation of agent performance and decision-making More details can be found in the following pull-requests merged into OpenTelemetry Add tool definition plus tool-related attributes in invoke-agent, inference, and execute-tool spans Capture evaluation results for GenAI applications Azure AI Foundry delivers unified observability for Microsoft Agent Framework, LangChain, LangGraph, OpenAI Agents SDK Agents built with Azure AI Foundry (SDK or portal) get out-of-the box observability in Foundry. With the new addition, agents built on different frameworks including Microsoft Agent Framework, Semantic Kernel, LangChain, LangGraph and OpenAI Agents SDK can use Foundry for monitoring, analyzing, debugging and evaluation with full observability. Agents built on Microsoft Agent Framework and Semantic Kernel get out-of-box tracing and evaluations support in Foundry Observability. Agents built with LangChain, LangGraph and OpenAI Agents SDK can use the corresponding packages and detailed instructions listed in the documentation to get tracing and evaluations support in Foundry Observability. Customer benefits With standardized multi-agent observability and support across multiple agent frameworks, customers get the following benefits: Unified Observability Platform for Multi-agent systems: Foundry Observability is the unified multi-agent observability platform for agents built with Foundry SDK or other agent frameworks like Microsoft Agent Framework, LangGraph, Lang Chain, OpenAI SDK. End-to-End Visibility into multi-agent systems: Customers can see not just what the system did, but how and why—from user request, through agent collaboration, tool usage, to final output. Faster Debugging & Root Cause Analysis: When something goes wrong (e.g., wrong answer, safety violation, performance bottleneck), customers can trace the exact path, see which agent/tool/task failed, and why. Quality & Safety Assurance: Built-in metrics and evaluation events (e.g. task success and validation scores) help customers monitor and improve the quality and safety of their AI workflows. Cost & Performance Optimization: Detailed metrics on token usage, API calls, resource consumption, and latency help customers optimize efficiency and cost. Get Started today with end-to-end agent observability with Foundry Azure AI Foundry Observability is a unified solution for evaluating, monitoring, tracing, and governing the quality, performance, and safety of your AI systems end-to-end— all built into your AI development loop and backed by the power of Azure Monitor for full stack observability. From model selection to real-time debugging, Foundry Observability capabilities empower teams to ship production-grade AI with confidence and speed. It’s observability, reimagined for the enterprise AI era. With the above OpenTelemetry enhancements Azure AI Foundry now provides detailed multi-agent observability for agents built with different frameworks including Azure AI Foundry, Microsoft Agent Framework, LangChain, LangGraph and OpenAI Agents SDK. Learn more about Azure AI Foundry Observability and get end-to-end agent observability today!Transforming Emergency Response: How AI is reshaping public safety
Brand new released Smart City Trend Report: Discover how AI is transforming emergency response and public safety in cities worldwide. In an era of escalating climate events, urban complexity, and rising public expectations, emergency response systems are under pressure like never before. From wildfires and floods to public health crises and infrastructure failures, cities must respond faster, smarter, and more collaboratively. The newly released Transform Emergency Response Trend Report offers a compelling roadmap for how artificial intelligence (AI) is helping cities meet these challenges head-on, by modernizing operations, improving situational awareness, and building resilient, resident-centered safety ecosystems. As Dave Williams, Director of Global Public Safety and Justice at Microsoft, puts it: AI models are increasingly embedded in public safety workflows to enhance both anticipation and real-time awareness. Predictive analytics are used to forecast crime hotspots, traffic incidents, and natural disasters by analyzing historical and real-time data, enabling proactive resource deployment and faster response times. This transformation is not theoretical, it’s happening now. And at the upcoming Smart City Expo World Congress in Barcelona, November 4–6, Microsoft and leading technology innovators will showcase how AI is driving real-world impact across emergency services, law enforcement, and city operations. Government AI Transformation in Action: Oklahoma City Fire Department: Digitizing Operations for Faster Response Serving over 700,000 residents, the Oklahoma City Fire Department (OKCFD) faced mounting challenges due to outdated, paper-based workflows. From rig inspections to fuel logging, manual processes slowed response times and increased risk. Partnering with AgreeYa Solutions and leveraging Microsoft Power Platform, OKCFD built 15+ custom mobile-first apps to digitize core operations. The results were transformative: Helped drive a 40% reduction in manual tasks Real-time dashboards for leadership visibility Improved data accuracy and faster emergency response This modernization not only boosted internal efficiency but also strengthened community trust by ensuring timely, reliable service delivery. North Wales Fire and Rescue Service: Empowering Remote Teams with Secure Access With 44 stations and a mix of full-time and on-call firefighters, North Wales Fire and Rescue Service (NWFRS) needed a better way to support staff across a wide geographic area. Their legacy on-premises systems limited remote access to critical data. By deploying a SharePoint-based intranet integrated with Microsoft 365 tools, NWFRS enabled secure, mobile access to documents, forms, and departmental updates. Improved communication and workflow efficiency Reduced travel time for on-call staff Enhanced compliance and data security This shift empowered firefighters to stay informed and prepared—no matter where they were. San Francisco Police Department: Real-Time Vehicle Recovery Reporting Managing thousands of stolen vehicle cases annually, the San Francisco Police Department (SFPD) struggled with a slow, manual reporting process that delayed updates and eroded public trust. Using Microsoft Power Apps, SFPD built RESTVOS (Returning Stolen Vehicle to Owner System), allowing officers to update vehicle status in real time from the field. Helped reduce reporting time from 2 hours to 2 minutes Supported 500 officer hours saved per month Improved resident experience and reduced mistaken stops This digital leap not only streamlined operations but also reinforced transparency and accountability. Join Us in Barcelona: See Emergency Response in Action At Smart City Expo World Congress 2025, Microsoft and our AI transformations partners will showcase emergency response AI transformation with immersive demos, theater sessions, and roundtable discussions. Transform Emergency Response will be a central focus, showcasing how AI, cloud platforms, and agentic solutions are enabling cities to: Modernize emergency operation centers Enable real-time situational awareness Foster community engagement and trust Featured AI demos from innovative partners: 3AM Innovations Disaster Tech PRATUS Sentient Hubs Tomorrow.io Unified Emergency Response with Microsoft Fabric and Copilot These solutions are not just about technology, they’re about outcomes. They help cities cut response times, improve coordination, and build public trust. Why This Matters Now As Dave Williams emphasizes, the future of emergency response is not just faster, it’s smarter and more resilient: Modern emergency response increasingly relies on unified data platforms that integrate inputs from IoT sensors, satellite imagery, social media, and agency databases. AI-powered analytics systems synthesize this data to support real-time decision-making and resource allocation across agencies. Cities must also invest in governance frameworks, ethical AI policies, and inclusive design to ensure these technologies serve all residents fairly. Let’s Connect Whether you’re a city CIO, emergency services leader, or public safety innovator, we invite you to join us at Smart City Expo World Congress in Barcelona, November 4–6. Explore how Microsoft and its partners are helping cities transform emergency response, and build safer, more resilient communities. Visit our booth at Hall 3, Stand #3D51, attend our theater sessions, and see demos from AI transformation partners delivering demos on Transform Emergency Response. Together, we can reimagine public safety for the challenges of today and the possibilities of tomorrow.Unlock Your Competitive Edge with Azure AI: The Strategic Roadmap You Can’t Afford to Miss
AI isn’t the future—it’s the now. Today, AI is a business imperative, and the organizations that master it will shape tomorrow’s industries. But here’s the truth: most teams don’t fail at AI for lack of ambition—they fail because they don’t have a structured plan. That’s a gap you can close—starting now. Microsoft Learn’s milestone-based approach gives you the structure, tools, and confidence to turn AI ambition into real results. Azure Essentials: A Foundation for AI Success Every successful cloud and AI journey starts with a strong foundation: governance, security, and operational excellence. The Azure Essentials framework is your blueprint, visualized as a continuous cycle: Readiness & Foundation: Prepare your organization with the right infrastructure, security, and strategic alignment. Design & Govern: Architect solutions and implement governance for compliance, scalability, and business alignment. Manage & Optimize: Continuously monitor, manage costs, and tune performance for operational excellence. This iterative model ensures your AI initiatives are launched, sustained, and optimized—perfectly aligned with Microsoft Learn’s milestone-based journey. Microsoft Learn Plan: Unlock Business Potential with AI Apps and Agents Ready to move from ambition to action? The Unlock business potential with AI Apps and agents Official Plan guides you from exploration to enterprise-grade execution. Whether you’re a business leader, developer, or innovator, this hands-on path gives you the structure, skills, and real-world context to build secure, scalable, and impactful AI solutions. Milestone 1: Building an AI-Ready Organization Before you build AI, you need to build trust and readiness. Milestone 1 is all about strategy, security, and organizational capability - grounded in Azure Essentials. Learn how to secure AI systems with the AI Security Fundamentals path. Establish governance and alignment with the AI Center of Excellence (ACoE). How the AI CoE assists in planning adoption of AI - Training Modernize your infrastructure with the Infrastructure for the Era of AI series. Preparing your data for AI innovation Maximize Data Value Refresh Why it matters: Fueled by an AI Center of Excellence, organizations with a well-established AI CoE unlock faster time-to-value, reduce risk, and foster a culture of innovation. They can move beyond isolated pilots to enterprise-grade AI transformation, enabling every team, from leadership to frontline users, to harness AI securely and effectively. AI Adoption: A People-First Transformation Once your organization is technically prepared, the real transformation begins—with your people. Every wave of innovation, from cloud to GenAI, reshapes how teams work, learn, and deliver value. The AI Center of Excellence (CoE) guides teams forward, focusing on trust, leadership, and collaboration. As shown in the diagram below, successful AI adoption is built on essential pillars—talent strategy, operating model, technology and data strategy, and business strategy—all supported by trust and effective change management. Key Practices for AI Readiness Build trust in technology across all levels: AI adoption guidance. Ensure leaders are visible and actively champion AI: Establish an AI Center of Excellence Upskill teams and foster cross-functional collaboration: AI learning hub Use Microsoft’s change management strategy to support adoption. Redesign talent systems for continuous learning and GenAI readiness: AI Readiness Assessment. Common Challenges Shifting from decision makers to transformation champions Bridging technical and non-technical collaboration Keeping pace with rapid change and new skills Actionable Tips Start with people, not tools. Co-create solutions to build trust. Make reskilling a priority. Measure real transformation, not just technical output. Real-World Example: A leading engineering firm saw adoption of its own GenAI-powered “Smart Analyzer” soar by 65% in five months after engaging sponsors and subject matter experts in co-creation and hands-on enablement. Milestone 2: Turn Strategy into Action This is where vision becomes velocity. Assess your AI skills with the AI Learning Journey. Learn how to embed AI into your business with the Transform Your Business with Microsoft AI path. Get inspired by real-world customer stories. Build foundational skills with the Get Started with AI Fundamentals module. Why it matters: This milestone turns your AI strategy into real-world impact. It helps you move from planning to piloting—transforming AI from a conceptual ambition into a tangible capability. Imagine a healthcare startup with a brilliant AI idea but no roadmap. Without direction, even the best ideas can stall. Microsoft Learn’s milestone-based journey provides that roadmap, guiding you step-by-step to build secure, scalable solutions that align with your mission and deliver real impact. Milestone 3: Optimize for Efficiency and Sustainability AI is powerful—but it must also be practical. For a practical roadmap to adopting and managing AI, explore the AI adoption guidance in the Cloud Adoption Framework. Learn how to manage AI costs. Design efficient, scalable solutions with the Azure Well-Architected Framework. Why it matters: This milestone ensures your AI initiatives are innovative, sustainable, and aligned with your business goals. It helps you design solutions that are cost-effective, scalable, and resilient. Without a clear roadmap, AI projects often stall, budgets spiral out of control, and organizational trust erodes. By focusing on operational excellence and cost optimization, you set up your AI strategy for long-term success. Milestone 4: Build Intelligent Apps and Agents Now it’s time to build. Set up your environment with the Plan and Prepare for Azure AI Development module. What is AI? Learn how to build autonomous, intelligent agents with AI Agent Fundamentals. Why it matters: This is where you go from learning to launching—building real AI apps and agents that start to solve your organization’s challenges. By the end of this journey, your team will be equipped to build secure, scalable AI apps and agents. You’ll have a strategy, a toolkit, and library of real-world use cases to draw insights and inspiration from, empowering you to deliver secure, scalable, and impactful AI solutions. AI in Action: Industry Use Cases AI is transforming every industry. Here’s how: Industry Use Cases Strategic Value Healthcare Diagnostics, triage, personalized care Better outcomes, lower costs Finance Fraud detection, risk modeling, automation Stronger compliance, faster service Retail Inventory, personalization, demand forecasting Higher loyalty, smarter supply chains Manufacturing Predictive maintenance, quality control Less downtime, better products Sustainability Emissions tracking, energy optimization ESG alignment, cost savings Nonprofit Donor engagement, service delivery Greater impact, improved transparency Quick Win: Explore Transform your business with Microsoft AI from Milestone 2 to see six key sectors of curated real-world AI use cases of how leading organizations are using AI to gain a competitive edge and deliver greater value; and how you can apply those insights to your own innovation journey. Your AI Journey Starts Now This isn’t just a learning path. It’s a launchpad. Microsoft Learn’s milestone-based journey gives you the tools, structure, and inspiration to turn AI ambition into AI advantage. Whether you’re just starting or scaling, this is your moment. Ready to build the future? Explore the curated Plan on Microsoft Learn: Unlock business potential with AI Apps and agents and begin your skilling journey to build the future—one milestone at a time. Pro Tip: Start small, celebrate quick wins, and scale with confidence. The future of AI is yours to build—one milestone at a time. Where are you on your AI journey? Share your story in the comments or connect with me on LinkedIn Barbara Andrews LinkedIn with #AzureAIJourney, #MicrosoftLearn #SkilledByMTT
The Future of AI: An Intern’s Adventure Turning Hours of Video into Minutes of Meaning
This blog post, part of The Future of AI series by Microsoft’s AI Futures team, follows an intern’s journey in developing AutoHighlight—a tool that transforms long-form video into concise, narrative-driven highlight reels. By combining Azure AI Content Understanding with OpenAI reasoning models, AutoHighlight bridges the gap between machine-detected moments and human storytelling. The post explores the challenges of video summarization, the technical architecture of the solution, and the lessons learned along the way.
Organising the AI Foundry: A Practical Guide for Enterprise Readiness
Purpose of the document provide overview of AI Foundry and how it can be set up and organised for at scale for an enterprise. This document should be considered as guidance and recommendations, but individual organisations should treat and consider other factors such as security, policy, governance and number of business units. AI Foundry Resource: Azure AI Foundry is Microsoft’s unified platform for building, customising, and managing AI applications and agents—designed to accelerate innovation and operationalise AI at scale. It brings together: Data, models, and operations into a single, integrated environment. A developer-first experience with native support for GitHub, Visual Studio, and Copilot Studio. A low-code portal and code-first SDKs/APIs for flexibility across skill levels. Key capabilities include: Model Catalogue: Access and customise top-tier LLM models (e.g. OpenAI, Hugging Face, Meta, Phi, Mistral, etc) Agent development: Build multi-agent systems using prebuilt templates and orchestration tools. Enterprise-grade governance: Identity based authentication, Role-based access (RBAC), quota management, and compliance tooling. AI Foundry offer centralised management; project workspaces will remain the primary environment for AI developers. Organization of AI Foundry Resource: The new version of AI Foundry Resource and its high-level component view: AI Foundry Resource serves as the foundational building block that defines the scope, configuration, and monitoring of deployments. AI Foundry Projects act like containers (child or sub resource of AI Foundry Resource), helping to organize work and resources within the context of an AI Foundry Resource. AI Foundry Project also provide access to Foundry’s developer APIs and tools. Organise & Set up AI Foundry: The following considerations can guide the design and establishment of an AI Foundry within an enterprise: Team Structure: Teams such as Data Science, AI Innovation, and Generative AI are structured and collaborate around specific business use cases. o AI Foundry Resource per Team: Separate resources are aligned to individual team who works multiple project / products o AI Foundry Resource per Product/Project: Separate resources are aligned to individual customer projects or products. o Single AI Foundry Resource: A single resource supports multiple teams or projects, depending on scale and maturity. Environment Setup: Environments are integral to the development lifecycle of Generative AI use cases, supporting the transition from experimentation to operationalisation through model deployment. Typical environment stages include: o Development, Testing, Production Each environment should include an instance of the AI Foundry resource to effectively support the full lifecycle of Generative AI deployment Team Structure: To address manageability and governance needs, organisations typically implement one or a combination of the following AI Foundry setup patterns. AI Foundry Resource per Team (Business Unit or Group within org): Each team is provisioned with a single AI Foundry Resource instance. Team members can use this shared resource to work on multiple use cases. Within this setup, each AI Foundry Project represents a distinct use case. These projects act as containers that organize all relevant components—such as agents and files—specific to that application. While projects inherit baseline security settings from the parent resource, they also support their own access controls, data integrations, and governance configurations, enabling fine-grained management at the project level. Each team ie Data Science, AI Innovation, and Generative AI is provisioned with a dedicated AI Foundry Resource instance. This instance acts as the primary workspace, allowing teams to create and manage multiple projects within a cohesive environment. Teams are typically organised by business unit or line of business, ensuring alignment with specific organizational goals. Centralized Governance: The AI Foundry Resource serves as the central control for each team, enabling unified access to data, shared resources, and consistent policy enforcement across all associated projects. Access Management: Security configurations and access controls defined at the resource level are inherited by all associated projects, ensuring consistency and simplifying team level administration. While these global settings are inherited, individual projects can define their own RBAC rules to address specific security and collaboration needs. Shared Connections: Connections established at the AI Foundry Resource level—such as links to data sources, tools, Azure OpenAI, or other Foundry resources—are accessible across all projects within the resource. It improves the team members productivity by having easily access, explore, and reuse the connections. Project Level Isolation: For projects handling sensitive data or subject to regulatory constraints, isolated connections can be configured at the project level to prevent sharing with other projects under the same Foundry Resource instance. Cost & Consumption Tracking: This approach streamlines cost management at the team level. As experimentation and trial runs can scale rapidly, consolidating activities within a single AI Foundry Resource per team helps maintain clear ownership and keeps operations well organised. This setup is recommended for enterprise-scale deployments where teams share similar needs—such as consistent data access, comparable experimentation workflows, or common asset usage—offering greater flexibility, seamless collaboration, and strong governance across the organization. AI Foundry Resource per Product/Project: Using an AI Foundry Resource at the product level is recommended when there is a need to fully isolate data and assets within a specific customer’s project or product. This setup is tailored for product-centric collaboration and sharing, ensuring that only the relevant product team or group has access. It enables controlled reuse of assets strictly within the boundaries of that product, supporting secure and focused development. Isolation & Ownership governance: All data, assets, and resources are isolated under a product scope, ensuring exclusive access and controlled reuse within the product by the designated team or group. For example, when multiple teams are involved in developing a single product, instead of provisioning separate AI Foundry Resources for each team, a single AI Foundry Resource can be established. Within this shared resource, individual AI Foundry projects can be created to support the development of sub-products, maintaining isolation while promoting coordinated collaboration. Access Management: In addition to product scope data isolation, Role-Based Access Control (RBAC) can be configured at the individual project level, allowing the product team to tightly manage permissions and maintain secure access to assets. Cost & Consumption Tracking: Budgeting, billing, and usage can be monitored and managed at the product level. Enables transparency and cost optimisation per product or project. Sharing Limitations: Challenges in sharing assets and connections outside the AI Foundry resource. For instance, fine-tuned models and their associated datasets are often tightly coupled to a specific product context. May require additional governance or integration mechanisms for cross-product collaboration. This setup is ideal when high levels of isolation, data security, and asset control are required. It supports projects that demand clear ownership, regulatory compliance, and product-specific governance Single AI Foundry Resource: This setup is ideal for non-project-specific or non-team-specific experimentation, where resources are shared across users without being tied to a particular initiative. It simplifies management, reduces admin overhead, and is best suited for sandbox environments. Ownership & Governance: Designed for general-purpose use with no team or project ownership. Enables user to run experiment without needing dedicated resources. Cost & Resource Efficiency: Costs are not attributed to any specific team or project. Helps minimise Azure resource sprawl and reduce management overhead. Simplified Management: Operates as a single unit for accessing all data and assets. Reduces the complexity of maintaining multiple isolated environments. Potential Challenges: Lack of isolation can lead to clutter and resource mismanagement. Difficult to manage access, data governance, and asset lifecycle as usage grows. Consolidating all projects / teams under a single AI Foundry resource can lead to disorder and governance challenges over time. This set up is recommended for Sandbox environments where flexibility and ease of access are prioritised over control and isolation. Environment Setup: Following environment deployment approaches are common: Single environment deployment: A single AI Foundry Resource is deployed without separating production and non-production data. This model is best suited for sandbox or experimental use cases where data isolation is not a priority and simplicity is preferred. Multi environments (e.g., Dev, Test, Prod) are established to segregate data and access controls. This setup supports both inner and outer loops of the GenAIOps lifecycle, enabling smooth promotion of code and assets from development to production. Recommended for enterprise-grade implementations requiring structured governance and lifecycle management. Isolated environment deployment: Environments are strictly separated based on data sensitivity. For example, the development environment accesses only non-production data, while the production environment handles live and historical data. This model ensures compliance, data security, and controlled access, making it suitable for regulated industries or sensitive workloads. Multi Environment Deployment The proposed multi environment approach aligns with the recommended model of assigning an AI Foundry Resource per team. Each environment contains separate subscriptions for different teams (Team A and Team B), which house individual AI Foundry resources and projects. These resources are connected to additional services for data, monitoring, and AI, ensuring the integration of security and content safety measures. By adopting a GenAIOps approach, any Generative AI or agent-related development can be efficiently promoted across environments—from development through to production—ensuring a smooth and consistent lifecycle. The shared subscription serves as a centralised platform for deploying AI Foundry assets such as models, domain knowledge, common data sources, and MCP servers that are universally applicable within a specific environment (ex: development). This centralised shared subscription approach ensures that governance, security, and control measures, such as policies prohibiting the use of certain LLM models, are comprehensively managed. Models and policies within the shared subscription can be uniformly applied across various projects. This setup not only facilitates strict governance and uniform policy across all projects but also enables inter-team collaboration within the same environment. For example, Team A in the development environment can leverage AI Foundry models, common “AI services” within the shared subscription and can connect with Team B's resources for additional AI functionalities (thro “connected resources”) such as other specific “AI services”. Access to these models for application purposes is mediated through an APIM gateway, which serves as a single-entry point for all LLM models consumption in the given environment. Each environment is recommended to have its own dedicated shared subscription to maintain organised and secure management of AI assets. Regions: AI Foundry Resource instances can be deployed across multiple regions based on organizational requirements such as data residency, compliance, and security. Associated resources can also span multiple regions to support workloads while still being centrally managed under a single AI Foundry Resource instance. Furthermore, LLM models can be deployed in various regions to accommodate different data zones, global standard, Batch and PTU. These regional and global deployed models can be accessed via APIs and keys, allowing seamless integration across geographies. Cost: Azure AI Foundry Resource integrates multiple Azure services, and its pricing structure varies based on the chosen setup and the number of AI Foundry projects created under a single resource instance. Costs are influenced by architectural decisions, resource usage, and the provisioning of associated components and services. It’s important to account for all related cost when planning deployments in Azure AI Foundry. To ensure cost efficiency and scalability, it is recommended to perform sizing and cost estimation based on the specific use cases and workloads being considered. IaC - Templates: Automate the AI Foundry Resource by using IaC templates ARM templates or Bicep templates to automate environment provision and secure deployments Terraform templates Conclusion In summary, Microsoft’s Azure AI Foundry offers a comprehensive and unified platform for organisations aiming to operationalise GenAI at scale. By providing a flexible structure that accommodates various team, product, and project requirements, AI Foundry empowers enterprises to tailor their setup according to business needs, security considerations, and governance standards. Selecting the right organisational model—whether by team, product, or through a single resource—enables alignment with business objectives, cost management, and collaboration. The recommended practices for environment setup, cost estimation, and automation with Infrastructure as Code streamline adoption and ongoing management. Ultimately, by following these guidelines and considering the unique context of each enterprise, organisations can maximise the value of AI Foundry, accelerating innovation whilst maintaining robust security, compliance, and operational efficiency.Azure OpenAI: GPT-5-Codex Availability?
Greetings everyone! I just wanted to see if there's any word as to when/if https://openai.com/index/introducing-upgrades-to-codex/ will make it's way to the AI Foundry. It was released on September 15th, 2025, but I have no idea how long Azure tends to follow behind OpenAI's releases. It doesn't really seem like there's any source of information to view whenever new models drop as to what Azure is going to do with them, if any. Any conversation around this would be helpful and appreciated, thanks!The Future of AI: Evaluating and optimizing custom RAG agents using Azure AI Foundry
This blog post explores best practices for evaluating and optimizing Retrieval-Augmented Generation (RAG) agents using Azure AI Foundry. It introduces the RAG triad metrics—Retrieval, Groundedness, and Relevance—and demonstrates how to apply them using Azure AI Search and agentic retrieval for custom agents. Readers will learn how to fine-tune search parameters, use end-to-end evaluation metrics and golden retrieval metrics like XDCG and Max Relevance, and leverage Azure AI Foundry tools to build trustworthy, high-performing AI agents.
