Empowering the Physical World with AI

insagiv

Microsoft

May 19, 2025

Unlocking AI at the Edge with Azure Arc

The integration of AI into the physical environment is revolutionizing ways we interact with and navigate the world around us. By embedding intelligence into edge devices, AI is not just processing data—it is defining how machines perceive, reason, and act autonomously in real-world scenarios. AI at the edge is transforming how we interact with our environment, driven by critical factors such as data sensitivity, local regulations, compliance, low latency requirements, limited network connectivity, and cost considerations. Added to this, the emergence of new, powerful agentic AI capabilities enables autonomous and adaptive real-time operations, making AI an indispensable tool in reshaping the physical world.

Customers’ Use Cases

By embedding AI into edge operations, industries are unlocking transformative efficiencies and innovations. In manufacturing, edge-powered AI enables real-time quality control and predictive maintenance, minimizing downtime and maximizing productivity. In retail, AI enhances customer experiences with personalized recommendations and streamlined inventory management. Similarly, finance leverages AI's capabilities for robust fraud detection and advanced risk management. Moreover, sectors like government and defense are increasingly adopting edge AI for safety-critical applications, enabling autonomous, real-time surveillance and response solutions that are both efficient and resilient. These advancements are paving the way for scalable, adaptive solutions that meet the unique demands of diverse operational environments.

Azure’s Adaptive Cloud Approach enabling AI from cloud to edge

Building on the promise to unify cloud and edge, Azure’s adaptive cloud approach is empowering teams to develop and scale AI workloads seamlessly across diverse environments. By enabling a unified suite of services tailored for modern AI applications, whether deployed in public clouds or distributed locations, Azure Arc enables streamlined operations with enhanced security and resilience.

Central to extending AI services to the edge is our commitment to adaptive, scalable, and efficient solutions tailored to diverse operational needs. Azure Arc plays a key role in this vision by facilitating seamless deployment and management of AI workloads across various environments. This week, we’re excited to share that a subset of Microsoft Azure AI Foundry models, such as Phi and Mistral have been rigorously validated to run on Azure Local enabled by Azure Arc.

Our investments are reflected in two primary areas:

Foundational tools for MLOps and developer frameworks, which empower teams to build robust AI applications
Intuitive, end-to-end low-code experiences designed for data analysts and solution developers. These low-code tools prioritize user-friendly interfaces and rapid deployment, enabling the creation of solutions with just a few clicks.

This dual focus ensures enterprises can fully harness the potential of edge AI while maintaining flexibility and operational efficiency.

Image 1: This high-level diagram illustrates our vision for the cloud to edge AI workloads, enabled by Azure Arc. Some components (agents and integration with AI Foundry and Foundry Local) are still under development, while others are more advanced and have been released to the market.

Build 2025: New Capabilities and Releases

This strategic vision is now being realized through a wave of new capabilities unveiled at Build 2025. These innovations are designed to accelerate edge AI adoption and simplify the developer experience—making it easier than ever to build, deploy, and manage intelligent applications across hybrid environments.

Announcements related to developer Building blocks:

Kubernetes AI Toolchain Orchestrator (KAITO), enabled by Azure Arc (public preview)
Foundry Local (public preview) for Windows apps to be deployed on any client device
read more here.
Workload orchestration (public preview)
Application development tools for Kubernetes enabled by Arc (public preview)

Refer to this blog to read more: https://aka.ms/AdaptiveCloudBuild2025

Announcements related to End-to-end experiences:

Edge RAG, enabled by Azure Arc is now available in public preview.
Azure AI Video Indexer for recorded files, enabled by Arc is generally available since April 2025.
Azure AI Video Indexer for live video analysis, enabled by Arc is available in private preview, for limited set of customers

Customer scenarios: enabling search and retrieval for on-premises data on Azure Local

Edge RAG targets customers who have data that needs to stay on premises due to data gravity, security and compliance, or latency requirements. We have observed significant and consistent interest from highly regulated sectors. These entities are exploring the use of RAG capabilities in disconnected environments through Azure Local.

DataON is a hybrid cloud computing company for enterprises of all sizes, with a focus on educational institutions and local government agencies. Recently, they have worked with the their customers to successfully deploy our RAG solution on CPU and GPU clusters and begin testing with sample end-customer data.

“DataON has been actively exploring how Edge RAG can enhance our Microsoft Azure Local solutions by providing more efficient data retrieval and decision-making capabilities. It’s exciting to be part of the private preview program and see firsthand how Edge RAG is shaping the future of data-driven insights.”

Howard Lo | VP, Sales & Marketing | DataON

This capability brings generative AI and RAG to on-premises data. Edge RAG was validated on AKS running on Azure Local.

Based on DataON and other customer feedback, we have expanded the version to include new features:

Model Updates: Ability to use any model compatible with OpenAI Inferencing standard APIs
Multi-lingual support: 100+ common languages for document ingestion and question-answer sessions
Multi-modal support: Support for image ingestion & retrieval during question-answer sessions
Search Types: Support for Text, Vector, Hybrid Text & Hybrid Text+Image searches
Ingestion Scale-out: Integration with KEDA for fully parallelized, high-throughput ingestion pipeline
Evaluation Workflow with RAG Metrics: Integrated workflow with built-in or customer-provided sample dataset

Read more about Edge RAG in this blog: https://aka.ms/AzureEdgeAISearchenabledbyArc.

AI Workloads for Disconnected Operations

In fully disconnected (air-gapped or non-internet) environments, such as those often found in government and defense sectors, technologies like RAG, can be deployed on-premises or in secure private clouds. This is currently available with limited access.

Use Cases:

Video analysis: Automatically analyzes video and audio content to extract metadata such as objects and scenes. Use cases include live video and analysis, mission debriefing and training, and modern safety.

Models consumption: A central repository for securely managing, sharing, and deploying AI/ML models. Use cases: model governance, rapid deployment of mission-specific models, and inter-agency collaboration.
Retrieval-Augmented Generation (RAG): Combines LLMs with a document retrieval system to generate accurate, context-aware responses based on internal knowledge bases. Use cases include field briefings, legal and policy compliance, and cybersecurity incident response.

Transforming Industries with AI: Real-World Stories from the Edge

Across industries, organizations are embracing AI to solve complex challenges, enhance operations, and deliver better outcomes. From healthcare to manufacturing, retail to energy, and even national security, Azure AI solutions are powering innovation at scale.

In the manufacturing sector, a global company sought to optimize production and reduce costly downtime. Azure AI Video Indexer monitored video feeds from production lines to catch defects early, while custom predictive maintenance models from the Model Catalog helped prevent equipment failures. RAG provided real-time insights into operations, empowering managers to make smarter decisions by asking questions. These tools collectively boosted efficiency, minimized downtime, and improved product quality.

At Airports, Azure AI helped enhance passenger experience and safety. From monitoring queue lengths and tracking vehicles to detecting falls and identifying restricted area breaches, the combination of Azure Local, Video Indexer, Azure IoT for Operations, and custom AI created a smarter, safer airport environment.

Retailers, too, are reaping the benefits. A major retail chain used Azure AI to understand in-store customer behavior through video analytics, optimize inventory with demand forecasting models, and personalize shopping experiences using RAG. These innovations led to better customer engagement, streamlined inventory management, and increased sales.

In Healthcare, a leading provider operating multiple hospitals and clinics nationwide faced the daunting task of analyzing massive volumes of patient data—from medical records and imaging to real-time feeds from wearable devices. With strict privacy regulations in play, they turned to Azure AI. Using Azure AI Video Indexer, they analyzed imaging data like X-rays and MRIs to detect anomalies. The Model Catalog enabled predictive analytics to identify high-risk patients and forecast readmissions. Meanwhile, Retrieval-Augmented Generation (RAG) gave doctors instant access to patient histories and relevant medical literature. The result? More accurate diagnoses, better patient care, and full regulatory compliance.

These stories highlight how Azure Arc enabled AI workloads are not just a set of tools—they are a catalyst for transformation. Whether it’s saving lives, improving safety, or driving business growth, the impact is real, measurable, and growing every day.

Learn More

Whether you are tuning in online or joining us in person, we wish you a fun and exciting Build 2025! The advancements in AI at the edge are set to revolutionize how we build, deploy, and manage applications, providing greater speed, agility, and security for businesses around the world.

Recommended Build Sessions: