Built a Real-Time Azure AI + AKS + DevOps Project – Looking for Feedback
Hi everyone, I recently completed a real-time project using Microsoft Azure services to build a cloud-native healthcare monitoring system. The key services used include: Azure AI (Cognitive Services, OpenAI) Azure Kubernetes Service (AKS) Azure DevOps and GitHub Actions Azure Monitor, Key Vault, API Management, and others The project focuses on real-time health risk prediction using simulated sensor data. It's built with containerized microservices, infrastructure as code, and end-to-end automation. GitHub link (with source code and documentation): https://github.com/kavin3021/AI-Driven-Predictive-Healthcare-Ecosystem I would really appreciate your feedback or suggestions to improve the solution. Thank you!
EOL of Azure Linux 2.0 on Azure Kubernetes Service enabled by Azure Arc
Azure Linux 2.0 will reach its End of Life (EOL) in July 2025 Azure Linux 2.0 (formerly CBL-Mariner) will reach its official End of Life (EOL) on July 31, 2025. After this date, it will no longer receive updates, security patches, or support from the Azure Linux team. Starting with the Azure Local 2507 release, Azure Kubernetes Service enabled by Azure Arc will ship Azure Linux 3.0 images for all supported Kubernetes versions. This change applies to all AKS enabled by Azure Arc deployments, as we have used Azure Linux 2.0 as the base image in the past. To maintain security compliance and ensure continued support, all AKS Arc customers must plan on migrating to Azure Linux 3.0 at the earliest by upgrading their Azure Local instances to the 2507 release, when it is available. What's new in Azure Linux 3.0 Approximately every three years Azure Linux releases a new version of its operating system with upgrades to major components. Azure Linux 3.0 offers increased package availability and versions, an updated kernel, and improvements to performance, security, and tooling and developer experience. Some of the major components upgraded from Azure Linux 2.0 to 3.0 include: Component Azure Linux 3.0 Azure Linux 2.0 Release Notes Linux Kernel v6.6 (Latest LTS) V5.15 (Previous LTS) Linux 6.6 Containerd v1.7.13, but will also offer v2.0 once it becomes stable 1.6.26 Containerd Releases SystemD v255 V250 Systemd Releases OpenSSL v3.3.0 V1.1.1k OpenSSL 3.3 For more details on the key features and updates in Azure Linux 3.0 see the 3.0 GitHub release notes. Upgrading to Azure Linux 3.0 Once the Azure Local 2507 release is available, update to 2507 . Once your Azure Local instance has upgraded, you can then upgrade your Kubernetes clusters You can choose to the remain on the same Kubernetes version and provide the same version number in the aksarc upgarde command. Once the upgrade is completed, you should be able to check the kernel version on your Linux nodes. Kernel version v6.6 is the latest Azure Linux 3.0 version. Sample command kubectl --kubeconfig /path/to/aks-cluster-kubeconfig get nodes -o wide Sample output NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME moc-lsbe393il9d Ready control-plane 3h14m 100.72.248.133 100.72.248.133 <none> CBL-Mariner/Linux 6.6.92.2 Containerd://1.6.26 moc-lzwagtkjah5 Ready control-plane 3h12m 00.72.248.134 v1.30.4 <none> CBL-Mariner/Linux 6.6.92.2 Containerd://1.6.26 FAQs Is Azure Linux same as Mariner? Yes, Mariner was rebranded to Azure Linux. We will slowly update our documentation and VM/container image tags to reflect this name change When did Azure Linux 3.0 GA? Azure Linux 3.0 became generally available in August 2024. When will Azure Linux 3.0 reach End of Life (EOL)? We currently support each major version for 3 years after it becomes generally available. Azure Linux 3.0 will reach EOL in Summer 2027. How to keep in touch with the AKS Arc team For updates, feedback, and feature requests related to AKS Arc: Ask questions & submit feedback via AKS Arc GitHub Issues Partners with support questions can reach out to aks-hci-talk@microsoft.com
Deploy a Kubernetes Application Programmatically Using Terraform and CLI
In our previous blog post, we explored Kubernetes apps benefits along with an introduction into how to programmatically deploy Kubernetes Apps. Today we will cover deploying a Kubernetes application programmatically using Terraform and CLI. These deployment methods can streamline your workflow and automate repetitive tasks. Deploying your Kubernetes Application using Terraform This walkthrough assumes you have previous knowledge of Terraform. For additional information and guidance on using Terraform to provision a cluster, please refer here. Prerequisites Before we begin, ensure you have the following: Terraform Azure CLI Sample Location You can find the Terraform sample we will be using at this location: Terraform Sample Prepare the Environment First, initialize Terraform in the current directory where you have copied the k8s-extension-install sample by running the following command: terraform init In the directory, you will find two example tfvars files. These files can be used to deploy the application with different configurations: azure-vote-without-config.tfvars - Deploy the application with the default configuration for azure-vote. azure-vote-with-config.tfvars - Deploy/update the application with a custom configuration for azure-vote. Before you test run the sample tfvars files, update the following in the tfvars files: cluster_name - The name of the AKS cluster. resource_group_name - The name of the resource group where the AKS cluster is located. subscription_id - The subscription ID where the AKS cluster is located. Deploy the Application To deploy the application with the default configuration for azure-vote, run: terraform apply -var-file="azure-vote-without-config.tfvars" To deploy or update the application with a custom configuration for azure-vote, use: terraform apply -var-file="azure-vote-with-config.tfvars" Conclusion And that's it! You've successfully deployed your Kubernetes application programmatically using Terraform. This process can drastically reduce the time and effort involved in managing and scaling your applications. By using Terraform, you can ensure that your deployment is consistent and repeatable, making it easier to maintain your infrastructure as code. Deploying a Kubernetes Application from Azure CLI Deploying a Kubernetes application using Azure CLI can seem daunting, but we’re here to make it simple and accessible. Follow these steps, and you’ll have your azure-vote application up and running in no time! Prerequisites Before we get started, ensure you have the following: Azure CLI installed on your machine Deploying the Sample Azure-Vote Application from the Marketplace Step 1: Log in to Azure Open your terminal and log in to your Azure account by running: az login Step 2: Set Your Subscription Specify the subscription you want to use with: az account set --subscription Step 3: Deploy the Azure-Vote Application Now, deploy the azure-vote application to your Kubernetes cluster with the following command: az k8s-extension create --name azure-vote --scope cluster ` --cluster-name <clusterName> --resource-group <resourceGroupName> --cluster-type managedClusters ` --extension-type commercialMarketplaceServices.AzureVote ` --plan-name azure-vote-paid ` --plan-product azure-vote-final-1 ` --plan-publisher microsoft_commercial_marketplace_services ` --configuration-settings title=VoteAnimal value1=Cats value2=Dogs Updating Configuration Settings If you want to update the configuration settings of the azure-vote application, you can do so easily. Use the following command to change the configuration settings: az k8s-extension update --name azure-vote ` --cluster-name <clusterName> --resource-group <resourceGroupName> --cluster-type managedClusters ` --configuration-settings value1=Elephant value2=Horse And there you have it! By following these steps, you can deploy and update the azure-vote application on your Kubernetes cluster using Azure CLI. Conclusion Deploying Kubernetes applications using Azure CLI is a powerful way to manage and scale your applications. The process described above helps ensure your deployments are consistent and repeatable, simplifying the maintenance of your infrastructure as code.😄Announcing general availability of workload orchestration: simplifying edge deployments at scale
We’re excited to announce the General Availability of workload orchestration, a new Azure Arc capability that simplifies how enterprises deploy and manage Kubernetes-based applications across distributed edge environments. Organizations across industries, such as manufacturing, retail, healthcare, face challenges in managing varied site-specific configurations. Traditional methods often require duplicating app variants—an error-prone, costly, and hard-to-scale approach. Workload orchestration solves this with a centralized, template-driven model: define configurations once, deploy them across all sites, and allow local teams to adjust within guardrails. This ensures consistency, improves speed, reduces errors, and scales with your CI/CD workflows—whether you’re supporting 200+ factories, offline retail clusters, or regionally-compliant hospital apps. Fig 1.0: Workload orchestration – Key features Key benefits of workload orchestration include: Solution Configuration & Template Reuse Define solutions, environments, and multiple hierarchy levels using reusable templates. Key-value stores and schema-driven inputs allow flexible configurations, validations with role-based access to maintain control. Context-Aware Deployments Automatically generate deployable artifacts based on selected environments (Dev, QA, Prod) and push changes safely through a git ops flow — enabling controlled rollouts and staged testing across multiple environments. Deploying at Scale in Constrained Environments Deploy workloads across edge and cloud environments with built-in dependency management and preloading of container images (a.k.a Staging) to minimize downtime during narrow maintenance windows. Bulk Deployment and Git Ops-Based Rollouts Execute large-scale deployments — including shared or dependent applications — across multiple sites using Git-based CI/CD pipelines that validate configurations and enforce policy compliance before rollout. End to End Observability K8 diagnostics in workload orchestration provide full-stack observability by capturing container logs, Kubernetes events, system logs, and deployment errors—integrated with Azure Monitor and Open Telemetry pipelines for proactive troubleshooting across edge and cloud environments. Who Is It For? Workload orchestration supports two primary user personas: IT Admins and DevOps Engineers: Responsible for initial setup and application configuration via CLI. OT Operators: Use the portal for day-to-day activities like monitoring deployments and adjusting configurations. Resources for You to Get Started You can start using workload orchestration by visiting the Azure Arc portal and following the documentation. We encourage you to try it with a small application deployed to a few edge sites. Create a template, define parameters like site name or configuration toggles, and run a deployment. As you grow more comfortable, expand to more sites or complex applications.Empowering the Physical World with AI
Unlocking AI at the Edge with Azure Arc The integration of AI into the physical environment is revolutionizing ways we interact with and navigate the world around us. By embedding intelligence into edge devices, AI is not just processing data—it is defining how machines perceive, reason, and act autonomously in real-world scenarios. AI at the edge is transforming how we interact with our environment, driven by critical factors such as data sensitivity, local regulations, compliance, low latency requirements, limited network connectivity, and cost considerations. Added to this, the emergence of new, powerful agentic AI capabilities enables autonomous and adaptive real-time operations, making AI an indispensable tool in reshaping the physical world. Customers’ Use Cases By embedding AI into edge operations, industries are unlocking transformative efficiencies and innovations. In manufacturing, edge-powered AI enables real-time quality control and predictive maintenance, minimizing downtime and maximizing productivity. In retail, AI enhances customer experiences with personalized recommendations and streamlined inventory management. Similarly, finance leverages AI's capabilities for robust fraud detection and advanced risk management. Moreover, sectors like government and defense are increasingly adopting edge AI for safety-critical applications, enabling autonomous, real-time surveillance and response solutions that are both efficient and resilient. These advancements are paving the way for scalable, adaptive solutions that meet the unique demands of diverse operational environments. Azure’s Adaptive Cloud Approach enabling AI from cloud to edge Building on the promise to unify cloud and edge, Azure’s adaptive cloud approach is empowering teams to develop and scale AI workloads seamlessly across diverse environments. By enabling a unified suite of services tailored for modern AI applications, whether deployed in public clouds or distributed locations, Azure Arc enables streamlined operations with enhanced security and resilience. Central to extending AI services to the edge is our commitment to adaptive, scalable, and efficient solutions tailored to diverse operational needs. Azure Arc plays a key role in this vision by facilitating seamless deployment and management of AI workloads across various environments. This week, we’re excited to share that a subset of Microsoft Azure AI Foundry models, such as Phi and Mistral have been rigorously validated to run on Azure Local enabled by Azure Arc. Our investments are reflected in two primary areas: Foundational tools for MLOps and developer frameworks, which empower teams to build robust AI applications Intuitive, end-to-end low-code experiences designed for data analysts and solution developers. These low-code tools prioritize user-friendly interfaces and rapid deployment, enabling the creation of solutions with just a few clicks. This dual focus ensures enterprises can fully harness the potential of edge AI while maintaining flexibility and operational efficiency. Image 1: This high-level diagram illustrates our vision for the cloud to edge AI workloads, enabled by Azure Arc. Some components (agents and integration with AI Foundry and Foundry Local) are still under development, while others are more advanced and have been released to the market. Build 2025: New Capabilities and Releases This strategic vision is now being realized through a wave of new capabilities unveiled at Build 2025. These innovations are designed to accelerate edge AI adoption and simplify the developer experience—making it easier than ever to build, deploy, and manage intelligent applications across hybrid environments. Announcements related to developer Building blocks: Kubernetes AI Toolchain Orchestrator (KAITO), enabled by Azure Arc (public preview) Foundry Local (public preview) for Windows apps to be deployed on any client device read more here. Workload orchestration (public preview) Application development tools for Kubernetes enabled by Arc (public preview) Refer to this blog to read more: https://aka.ms/AdaptiveCloudBuild2025 Announcements related to End-to-end experiences: Edge RAG, enabled by Azure Arc is now available in public preview. Azure AI Video Indexer for recorded files, enabled by Arc is generally available since April 2025. Azure AI Video Indexer for live video analysis, enabled by Arc is available in private preview, for limited set of customers Customer scenarios: enabling search and retrieval for on-premises data on Azure Local Edge RAG targets customers who have data that needs to stay on premises due to data gravity, security and compliance, or latency requirements. We have observed significant and consistent interest from highly regulated sectors. These entities are exploring the use of RAG capabilities in disconnected environments through Azure Local. DataON is a hybrid cloud computing company for enterprises of all sizes, with a focus on educational institutions and local government agencies. Recently, they have worked with the their customers to successfully deploy our RAG solution on CPU and GPU clusters and begin testing with sample end-customer data. “DataON has been actively exploring how Edge RAG can enhance our Microsoft Azure Local solutions by providing more efficient data retrieval and decision-making capabilities. It’s exciting to be part of the private preview program and see firsthand how Edge RAG is shaping the future of data-driven insights.” Howard Lo | VP, Sales & Marketing | DataON This capability brings generative AI and RAG to on-premises data. Edge RAG was validated on AKS running on Azure Local. Based on DataON and other customer feedback, we have expanded the version to include new features: Model Updates: Ability to use any model compatible with OpenAI Inferencing standard APIs Multi-lingual support: 100+ common languages for document ingestion and question-answer sessions Multi-modal support: Support for image ingestion & retrieval during question-answer sessions Search Types: Support for Text, Vector, Hybrid Text & Hybrid Text+Image searches Ingestion Scale-out: Integration with KEDA for fully parallelized, high-throughput ingestion pipeline Evaluation Workflow with RAG Metrics: Integrated workflow with built-in or customer-provided sample dataset Read more about Edge RAG in this blog: https://aka.ms/AzureEdgeAISearchenabledbyArc. AI Workloads for Disconnected Operations In fully disconnected (air-gapped or non-internet) environments, such as those often found in government and defense sectors, technologies like RAG, can be deployed on-premises or in secure private clouds. This is currently available with limited access. Use Cases: Video analysis: Automatically analyzes video and audio content to extract metadata such as objects and scenes. Use cases include live video and analysis, mission debriefing and training, and modern safety. Models consumption: A central repository for securely managing, sharing, and deploying AI/ML models. Use cases: model governance, rapid deployment of mission-specific models, and inter-agency collaboration. Retrieval-Augmented Generation (RAG): Combines LLMs with a document retrieval system to generate accurate, context-aware responses based on internal knowledge bases. Use cases include field briefings, legal and policy compliance, and cybersecurity incident response. Transforming Industries with AI: Real-World Stories from the Edge Across industries, organizations are embracing AI to solve complex challenges, enhance operations, and deliver better outcomes. From healthcare to manufacturing, retail to energy, and even national security, Azure AI solutions are powering innovation at scale. In the manufacturing sector, a global company sought to optimize production and reduce costly downtime. Azure AI Video Indexer monitored video feeds from production lines to catch defects early, while custom predictive maintenance models from the Model Catalog helped prevent equipment failures. RAG provided real-time insights into operations, empowering managers to make smarter decisions by asking questions. These tools collectively boosted efficiency, minimized downtime, and improved product quality. At Airports, Azure AI helped enhance passenger experience and safety. From monitoring queue lengths and tracking vehicles to detecting falls and identifying restricted area breaches, the combination of Azure Local, Video Indexer, Azure IoT for Operations, and custom AI created a smarter, safer airport environment. Retailers, too, are reaping the benefits. A major retail chain used Azure AI to understand in-store customer behavior through video analytics, optimize inventory with demand forecasting models, and personalize shopping experiences using RAG. These innovations led to better customer engagement, streamlined inventory management, and increased sales. In Healthcare, a leading provider operating multiple hospitals and clinics nationwide faced the daunting task of analyzing massive volumes of patient data—from medical records and imaging to real-time feeds from wearable devices. With strict privacy regulations in play, they turned to Azure AI. Using Azure AI Video Indexer, they analyzed imaging data like X-rays and MRIs to detect anomalies. The Model Catalog enabled predictive analytics to identify high-risk patients and forecast readmissions. Meanwhile, Retrieval-Augmented Generation (RAG) gave doctors instant access to patient histories and relevant medical literature. The result? More accurate diagnoses, better patient care, and full regulatory compliance. These stories highlight how Azure Arc enabled AI workloads are not just a set of tools—they are a catalyst for transformation. Whether it’s saving lives, improving safety, or driving business growth, the impact is real, measurable, and growing every day. Learn More Whether you are tuning in online or joining us in person, we wish you a fun and exciting Build 2025! The advancements in AI at the edge are set to revolutionize how we build, deploy, and manage applications, providing greater speed, agility, and security for businesses around the world. Recommended Build Sessions: Breakout session (BRK188): Power your AI apps across cloud and edge with Azure Arc Breakout session (BRK183): Improving App Health with Health Modeling and Chaos Engineering Breakout session (BRK 195): Inside Azure innovations with Mark Russinovich Breakout session (BRK 168): AI and Agent Observability in Azure AI Foundry and Azure MonitorComparing feature sets for AKS enabled by Azure Arc deployment options
This article shows a comparison of features available for the different deployment options under AKS enabled by Azure Arc. AKS on Azure Stack HCI, version 23H2 AKS Edge Essentials AKS on Windows Server and AKS on Azure Stack HCI 22H2 Supported infrastructure where the Kubernetes clusters are hosted Azure Stack HCI, version 23H2 Windows 10/11 IoT Enterprise Windows 10/11 Enterprise Windows 10/11 Pro Windows Server 2019/2022 Azure Stack HCI 22H2 Windows Server 2019 Windows Server 2022 CNCF conformant? Yes Yes Yes K8s cluster lifecycle management tools (create, scale, upgrade and delete clusters) Az CLI Az PowerShell Azure Portal ARM templates PowerShell PowerShell Windows Admin Center Kubernetes cluster management plane Kubernetes clusters are managed by Arc Resource Bridge that runs as part of infrastructure components on the Azure Stack HCI cluster. Kubernetes clusters are self-managed, to preserve resources. Kubernetes clusters are managed using a “management cluster”, that is installed using PowerShell before Kubernetes workload clusters can be created. Can you use kubectl and other open-source Kubernetes tools? Yes Yes Yes Supported Kubernetes versions. Supports K8s only. Continuous updates to supported Kubernetes versions. For latest version support, run az aksarc get-versions. Supports K3s and K8s. Continuous updates to supported Kubernetes versions. For the latest version, visit steps to prepare your machine for AKS Edge Essentials. Supports K8s only. Continuous updates to supported Kubernetes versions. For latest version support, visit AKS hybrid releases on GitHub. Azure Fleet Manager integration No No No Terraform integration Not yet No No Azure Monitor integration Yes, via Arc extensions Yes, via Arc extensions Yes, via Arc extensions The following is a comparison between node pool capabilities for AKS enabled by Azure Arc deployment options: AKS on Azure Stack HCI, version 23H2 AKS Edge Essentials AKS on Windows Server and Azure Stack HCI 22H2 Windows nodepool support Yes Windows Server 2019 Datacenter Windows Server 2022 Datacenter Yes Windows Server 2022 Datacenter (Core) Yes Windows Server 2019 Datacenter Windows Server 2022 Datacenter Linux OS options CBL-Mariner CBL-Mariner CBL-Mariner Container Runtime Containerd for Linux and Windows nodes. Containerd for Linux and Windows nodes. Containerd for Linux and Windows nodes. Node pool auto-scalar Yes No (manually add nodes) Yes Horizontal pod scalar Yes No Yes GPU support Yes No Yes Azure container registry Yes Yes Yes The following is a comparison between networking features for AKS enabled by Azure Arc deployment options: AKS on Azure Stack HCI, version 23H2 AKS Edge Essentials AKS on Windows Server and Azure Stack HCI 22H2 Network creation and management You need to create the network in Azure Stack HCI 23H2 before creating an AKS cluster. You also need to ensure the network has the right connectivity and IP address availability for a successful cluster creation and operation. You need to provide the IP address range for node IPs and Service IPs, that is available and has the right connectivity. The network configuration needed for the cluster is handled by AKS. Read AKS Edge Essentials networking. You need to create the network in Windows Server before creating an AKS cluster. You also need to ensure the Read network has the right connectivity and IP address availability for a successful cluster creation and operation. Supported networking options Static IP networks with/without VLAN ID Static IP address or use reserved IPs when using DHCP DHCP networks with/without VLAN ID Static IP networks with/without VLAN ID SDN support No No Yes Supported CNIs Calico Calico (K8s) Flannel (K3s) Calico Load balancer MetalLB Arc extension Bring your own load balancer (BYOLB) KubeVIP MetalLB Arc extension Bring your own load balancer (BYOLB) HAProxy MetalLB Arc extension SDN load balancer Bring your own load balancer (BYOLB) The following is a comparison between storage features for AKS enabled by Azure Arc deployment options: AKS on Azure Stack HCI, version 23H2 AKS Edge Essentials AKS on Windows Server and Azure Stack HCI 22H2 Types of supported persistent volumes Read Write Once Read Write Many PVC using local storage Read Write Once Read Write Many Container Storage Interface (CSI) support Yes Yes Yes CSI drivers Disk and Files (SMB and NFS) drivers installed by default. Support for SMB and NFS storage drivers. Support for SMB and NFS storage drivers. Dynamic provisioning support Yes Yes Yes Volume resizing support Yes Yes Yes The following is a comparison between security and authentication options in AKS and AKS enabled by Azure Arc: AKS on Azure Stack HCI, version 23H2 AKS Edge Essentials AKS on Windows Server and Azure Stack HCI 22H2 Access to Kubernetes clusters Kubectl Kubectl Kubectl Kubernetes cluster authentication Certificate based Kubeconfig Microsoft Entra ID Certificate based Kubeconfig Microsoft Entra ID Certificate based Kubeconfig Microsoft Entra ID Active Directory SSO Kubernetes cluster authorization (RBAC) Kubernetes RBAC Azure RBAC Kubernetes RBAC Kubernetes RBAC Support for network policies No No Yes – only for Linux containers Limit source networks that can access API server Yes Yes Yes Certificate rotation and encryption Yes Yes Yes Secrets store CSI driver Yes Yes Yes gMSA support No Yes Yes Azure policy Yes, via Arc extensions Yes, via Arc extensions Yes, via Arc extensions Azure Defender No Yes, via Arc extensions (preview) Yes, via Arc extensions (preview) The following is a comparison between pricing and SLA for AKS and AKS enabled by Azure Arc: AKS on Azure Stack HCI, version 23H2 AKS Edge Essentials AKS on Windows Server and Azure Stack HCI 22H2 Pricing Pricing is based on the number of workload cluster vCPUs. Control plane node nodes are free. Azure Stack HCI, version 23H2 is priced a $10/physical core and AKS workload VMs is $24/vcpu/month. $2.50 per device per month. Pricing is based on the number of workload cluster vCPUs. Control plane nodes & load balancer VMs are free. Azure Stack HCI, version 23H2 is priced a $10/physical core and AKS workload VMs is $24/vcpu/month. Azure hybrid benefit support Yes No Yes SLA No SLA offered since the Kubernetes cluster is running on-premises. No SLA offered since the Kubernetes cluster is running on-premises. No SLA offered since the Kubernetes cluster is running on-premises.Public Preview: Deploy OSS Large Language Models with KAITO on AKS on Azure Local
Announcement Along with Kubernetes AI Toolchain Operator (KAITO) on AKS GA release, we are thrilled to announce Public Preview refresh for KAITO on AKS on Azure Local. Customers can now enable KAITO as a cluster extension on AKS enabled by Azure Arc as part of cluster creation or day 2 using Az CLI. The seamless enablement experience makes it easy to get started with LLM deployment and fully consistent with AKS in the cloud. We also invest heavily to reduce frictions in LLM deployment such as recommending the right GPU SKU, validating preset models with GPUs and avoiding Out of Memory errors, etc. KAITO Use Cases Many of our lighthouse customers are exploring exciting opportunities to build, deploy and run AI Apps at the edge. We’ve seen many interesting scenarios like Pipeline Leak detection, Shrinkage detection, Factory line optimization or GenAI Assistant across many industry verticals. All these scenarios need a local AI model with edge data to satisfy low latency or regulatory requirements. With one simple command, customers can quickly get started with LLM in the edge-located Kubernetes cluster, and ready to deploy OSS models with OpenAI-compatible endpoints. Deploy & fine-tune LLM declaratively With KAITO extension, customers can author a simple YAML for inference workspace in Visual Studio Code or any text editor and deploy a variety of preset models ranging from Phi-4, Mistral, to Qwen with kubectl on any supported GPUs. In addition, customers can deploy any vLLM compatible text generation model from Hugging Face or even private weights models by following custom integration instructions. You can also customize base LLMs in the edge Kubernetes with Parameter Efficient Fine Tuning (PEFT) using qLoRA or LoRA method, just like the inference workspace deployment with YAML file. For more details, please visit the product documentation and KAITO Jumpstart Drops for more details. Compare and evaluate LLMs in AI Toolkit Customers can now use AI Toolkit, a popular extension in Visual Studio Code, to compare and evaluate LLMs whether it’s local or remote endpoint. With AI Toolkit playground and Bulk Run features, you can test and compare LLMs side by side and find out which model fits the best for your edge scenario. In addition, there are many built-in LLM Evaluators such as Coherence, Fluency, or Relevance that can be used to analyze model performance and generate numeric scores. For more details, please visit AI Toolkit Overview document. Monitor inference metrics in Managed Grafana The KAITO extension defaults to vLLM inference runtime. With vLLM runtime, customers can now monitor and visualize inference metrics with Azure Managed Prometheus and Azure Managed Grafana. Within a few configuration steps, e.g., enabling the extensions, labeling inference workspace, creating Service Monitor, the vLLM metrics will show up in Azure Monitor Workspace. To visualize them, customers can link the Grafana dashboard to Azure Monitor Workspace and view the metrics using the community dashboard. Please view product document and vLLM metric reference for more details. Get started today The landscape of LLM deployment and application is evolving at lightning speed - especially in the world of Kubernetes. With the KAITO extension, we're aiming to supercharge innovation around LLMs and streamline the journey from ideation to model endpoints to real-world impact. Dive into this blog as well as KAITO Jumpstart Drops to explore how KAITO can help you get up and running quickly on your own edge Kubernetes cluster. We’d love to hear your thoughts - drop your feedback or suggestions in the KAITO OSS Repo!Public Preview: Workload orchestration simplifying edge deployment at scale
Public Preview Announcement - workload orchestration Introduction: As enterprises continue to scale their edge infrastructure, IT teams face growing complexity in deploying, managing, and monitoring workloads across distributed environments. Today, we are excited to announce the Public Preview of workload orchestration — a purpose-built platform that redefines configuration and deployment management across enterprise environments. Workload orchestration is designed to help you centrally manage configurations for applications deployed in diverse locations (from factories and retail stores to restaurants and hospitals) while empowering on-site teams with flexibility. Modern enterprises increasingly deploy Kubernetes-based applications at the edge, where infrastructure diversity and operational constraints are the norm. Managing these with site-specific configurations traditionally requires creating and maintaining multiple variants of the same application for different sites – a process that is costly, error-prone, and hard to scale. Workload orchestration addresses this challenge by introducing a centralized, template-driven approach to configuration. With this platform, central IT can define application configurations once and reuse them across many deployments, ensuring consistency and compliance, while still allowing site owners to adjust parameters for their local needs within controlled guardrails. The result is a significantly simplified deployment experience that maintains both central governance and localized flexibility. Key features of workload orchestration The public preview release of workload orchestration includes several key innovations and capabilities designed to simplify how IT manages complex workload deployments: Powerful Template Framework & Schema Inheritance: Define application configurations and schemas one time and reuse or extend them for multiple deployments. Workload orchestration introduces a templating framework that lets central IT teams create a single source of truth for app configurations, which can then be inherited and customized by different sites as needed. This ensures consistency across deployments and streamlines the authoring process by eliminating duplicate work. Dependent Application Management: Manage and deploy interdependent applications seamlessly using orchestrated workflows. The platform supports configuring and deploying apps with dependencies via a guided CLI or an intuitive portal experience, reducing deployment friction and minimizing errors when rolling out complex, multi-tier applications. Custom Validation Rules: Ensure every configuration is right before it’s applied. Administrators can define pre-deployment validation expressions (rules) that automatically check parameter inputs and settings. This means that when site owners customize configurations, all inputs are validated against predefined rules to prevent misconfigurations, helping to reduce rollout failures. External Validation Rules: External validation enables you to verify the solution template through an external service, such as an Azure Function or a webhook. The external validation service receives events from the workload orchestration service and can execute custom validation logic. This design pattern is commonly used when customers require complex validation rules that exceed data type and expression-based checks. It allows the implementation of business-specific validation logic, thereby minimizing runtime errors. Integrated Monitoring & Unified Control: Track and manage deployments from a single pane of glass. Workload orchestration includes an integrated monitoring dashboard that provides near real-time visibility into deployment progress and the health of orchestrated workloads. From this centralized interface, you can pause, retry, or roll back deployments as needed, with full logging and compliance visibility for all actions. Enhanced Authoring Experience (No-Code UI with RBAC): We’ve built a web-based orchestration portal that offers a no-code configuration authoring experience. Configuration managers can easily define or update application settings via an intuitive UI – comparing previous configuration revisions side by side, copying values between versions, and defining hierarchical parameters with just a few clicks. This portal is secured with role-based access control (RBAC) and full audit logging, so non-developers and local operators can safely make approved adjustments without risking security or compliance. CLI and Automation Support: For IT admins and DevOps engineers, workload orchestration provides a command-line interface (CLI) optimized for automation. This enables scripted deployments and environment bootstrapping. Power users can integrate the orchestration into CI/CD pipelines or use it to programmatically manage application lifecycles across sites, using familiar CLI commands to deploy or update configurations in bulk. Fast Onboarding and Setup: Getting started with orchestrating your edge environments is quick. The platform offers guided setup workflows to configure your organizational hierarchy of edge sites, define user roles, and set up access policies in minutes. This means you can onboard your team and prepare your edge infrastructure for orchestration without lengthy configuration processes. Architecture & Workflow: Workload orchestration is a service built with cloud and edge components. At a high level, the cloud control plane of workload orchestration provides customers and opportunity to use a dedicated resource provider to define templates centrally which WO edge agents consume and contextualize based on required customization needed at edge locations. The overall object model is embedded in Azure Resource Manager thus providing customers fine grained RBAC (Role Based Access Control) for all workload orchestration resources. The key actions to manage WO are governed by an intuitive CLI and portal experience. There is also a simplified no code experience for non-technical onsite staff for authoring, monitoring and deploying solution with contextualized configurations. Important Details & Limitations: Preview Scope: During public preview, workload orchestration supports Kubernetes-based workloads at the edge (e.g., AKS edge deployments or Arc-enabled Kubernetes clusters). Support for other types of workloads or cloud VMs is coming soon. Regions and Availability: The service is available in East US and East US2 regions during preview. Integration Requirements: Using workload orchestration with your edge Kubernetes clusters require them to be connected (e.g., via Azure Arc) for full functionality. Getting Started with workload orchestration Availability: Workload orchestration is available in public preview starting 19 th May, 2025. For access to public preview, please complete the form to get access for your subscription or share your subscription details over email at configmanager@service.microsoft.com. Once you have shared the details, the team will get back to you with an update on your request! Try it Out: We encourage you to try workload orchestration with one of your real-world scenarios. A great way to start is to pick a small application that you typically deploy to a few edge sites and use the orchestration to deploy it. Create a template for that app, define a couple of parameters (like a site name or a configuration toggle), and run a deployment to two or three test sites. This hands-on trial will let you experience first-hand how the process works and the value it provides. As you grow more comfortable, you can expand to more sites or more complex applications. Because this is a preview, feel free to experiment — you can deploy to non-production clusters or test environments to see how the orchestration fits your workflow. Feedback and Engagement We’d love to hear your feedback! As you try out workload orchestration, please share your experiences, questions, and suggestions. You can leave a comment below this blog post – our team will be actively monitoring and responding to comments throughout the preview. Let us know what worked well, what could be improved, and any features you’d love to see in the future. Your insights are incredibly valuable to us and will help shape the product as we progress toward General Availability. If you encounter any issues or have urgent feedback, you can also engage with us through the following channels: Email at configmanager@service.microsoft.com or fill up the form at WOfeedback for feedback Email at configmanager@service.microsoft.com or fill up the form at WOReportIssuees for reporting issues Contact your Microsoft account representative or support channel and mention “workload orchestration Public Preview” – they can route your feedback to us as well. Occasionally, we may reach out to select preview customers for deeper feedback sessions or to participate in user research. If you’re interested in that, please mention it in your comment or forum post. We truly consider our preview users as co-creators of the product. Many of the features and improvements in workload orchestration have been influenced by early customer input. So, thank you in advance for sharing your thoughts and helping us ensure that this platform meets your needs! (Reminder: Since this is a public preview, it is not meant for production use yet. If you do decide to use it in a production scenario, do so with caution and be aware of the preview limitations. We will do our best to assist with any issues during preview). Learn More To help you get started and dive deeper into workload orchestration, we’ve prepared a set of resources: Workload orchestration Documentation – Overview and how-to guides: Learn about the architecture, concepts, and step-by-step instructions for using workload orchestration in our official docs. [WO documentation] Quick Start: Deploy Your First Application – Tutorial: Follow a guided tutorial to create a template and deploy a sample application to a simulated edge cluster using workload orchestration. [Quickstart] CLI Reference – Command reference: Detailed documentation of all workload orchestration CLI commands with examples. [CLI reference] Conclusion: We’re thrilled for you to explore workload orchestration and see how it can transform your edge deployment strategy. This public preview is a major step towards simplifying distributed workload management, and your participation and feedback are key to its success.Unlocking AI Apps Across Boundaries with Azure
As we open the doors to Microsoft Build 2025, I’m thrilled to share the newest releases in our effort to enable teams to more rapidly develop and scale applications across boundaries: app development tools for Kubernetes (public preview), Kubernetes AI Toolchain Orchestrator [KAITO] (public preview), Foundry Local (public preview), workload orchestration (public preview) and Retrieval-Augmented Generation (RAG) capabilities on Azure Local (public preview). With our adaptive cloud approach, we offer a unified set of capabilities to enable your AI applications—whether they’re deployed to the public cloud, in hybrid environments, or at distributed edge locations. These capabilities include tools developers use every day, such as Visual Studio Code, to help build AI applications faster, better, and with greater security and resilience than ever before. Microsoft's Adaptive cloud approach to more rapidly developing and scaling applications across boundaries These new additions complement existing capabilities from Azure Arc for Kubernetes and Azure Kubernetes Service (AKS) enabled by Azure Arc, that support the hosting of containerized workloads, now with key capabilities designed to help expedite the creation of AI applications from model selection to edge-ready cluster provisioning (with GPU nodes), automated model deployment, lifecycle management and more. By combining KAITO with Azure Arc and Foundry Local in your workflow, Microsoft provides you with a more unified, flexible platform for building and running intelligent applications across boundaries. Learn more about our Arc-enabled AI story here. To help accelerate your adoption of cloud-native capabilities in distributed environments, Kubernetes-based app development tools extend essential services—such as container storage and secrets synchronization—to edge-located clusters. And we plan to expand this set of services in the future. This integration simplifies the deployment and management of applications across hybrid and multi-cloud environments. By unifying infrastructure and application lifecycle management, it empowers teams to move faster while maintaining consistency, security, and visibility. More details on each of these releases below. Here’s a glimpse of what they can mean for you, your workflow and your company. Many of these services are already making a difference for application teams at customers like Domino’s, Coles, Chevron, and Dick’s Sporting Goods. Providing them with greater speed and agility, as they build the solutions their customers and teams need. As customers continue to modernize their applications across hybrid, multi-cloud and distributed environments, many rely on trusted solutions from independent software vendors (ISVs). This is designed to help accelerate this journey—enabling partners to build, validate, and publish Arc-enabled Kubernetes applications directly to the Azure Marketplace. Building on the momentum from our initial launch at last year's Ignite, I'm excited to introduce a new wave of partner solutions to the Azure Arc ISV Partner Program. This latest expansion brings not only new partners, but also entirely new solution categories to the Azure Marketplace—including Security, Networking & Service Mesh, API Infrastructure & Management, and Monitoring & Observability. With just a few clicks, customers can now deploy enterprise-grade tools like HashiCorp Vault Enterprise, Istio by Solo.io, Traefik’s API stack, and Dynatrace Operator directly onto their Arc-enabled Kubernetes clusters. These additions to the Azure Arc ISV Partner Program reflect our commitment to supporting the full spectrum of cloud-native application needs. Explore the growing ecosystem of Arc-enabled solutions in the Azure Marketplace. RELEASES Here’s a recap of some of our newest feature releases that support our Adaptive cloud approach. App development tools for Kubernetes | Public Preview Kubernetes clusters enabled by Azure Arc helps power our adaptive cloud strategy. We are extending a set of fundamental, services that are fully validated, managed and deployed by Arc. The initial set of these services includes Azure Container Storage enabled by Azure Arc and Azure Key Vault . In the future we will be expanding and adding more of these foundational services. In addition, a Visual Studio Code extension is available for developers to kick start Kubernetes application development and turn their Kubernetes apps into Arc-enabled applications. This toolkit provides code samples and an environment to build, test and deploy Kubernetes applications. Figure 1: app development tools for Kubernetes in Azure Arc Retrieval-Augmented Generation (RAG) capabilities on Azure Local | Public Preview Edge RAG on Azure Local is a is a turnkey service, Azure Arc-enabled solution that brings Retrieval-Augmented Generation (RAG) capabilities to on-premises environments. It can help customers to build, evaluate, and deploy generative AI applications—like custom chat assistants—directly on their local data, without sending it to the cloud. This release is especially valuable for industries like manufacturing and healthcare, where data sovereignty, low latency, and IP protection are important. By supporting customer local deployment of language models, more secure data ingestion, and built-in tools for prompt engineering and evaluation, these capabilities help empower organizations to unlock AI insights while maintaining more control over their data. KAITO extension for AKS on Azure Local | Public Preview Kubernetes AI Toolchain Operator (KAITO) enabled by Azure Arc is designed to help simplify and scale AI model deployment across hybrid and edge environments. It enables developers to declaratively deploy AI models—whether from Microsoft’s AI Foundry, third-party hubs like Hugging Face, or customer-provided sources—on Arc-enabled Kubernetes clusters. It helps customers bring cloud-native AI capabilities to the edge, enable low-latency inference, more consistent lifecycle management, and operational control across diverse infrastructure. Try it out today using the “KAITO & AKS Arc” Jumpstart Drop! Figure 2: Deploy AI models on AKS in hybrid and edge environments using KAITO Workload orchestration | Public Preview Workload orchestration provides a centralized, template-driven platform for managing application configurations across distributed edge environments. It enables IT teams to define reusable templates, manage interdependent applications, and enforce custom validation rules—both built-in and external. It also includes integrated monitoring, a no-code authoring portal with RBAC, and CLI support for automation and CI/CD integration. Workload orchestration simplifies complex edge deployments by unifying configuration management and governance, empowering teams to scale faster with consistency, security, and flexibility. Foundry Local | Public Preview Foundry Local is the high-performance local AI runtime stack that helps bring Azure AI Foundry’s power to client devices. It includes CLI, SDK, and a local REST API for model inference, and integrates with the Azure AI Foundry catalog for model access and deployment. It can help provide performance optimizations for Windows and Apple Silicon, and the SDK enables code portability between local and cloud environments. Foundry Local, now available in preview on Windows and macOS, enables the creation and deployment of cross-platform AI applications that help operate models, tools, and agents directly on-device. This eliminates reliance on cloud connectivity and offers more enhanced control and flexibility. FIND US AT BUILD Breakout session (BRK188): Build and Scale your AI apps with Kubernetes and Azure Arc Breakout session (BRK183): Improving App Health with Health Modeling and Chaos Engineering Breakout session (BRK 195): Inside Azure innovations with Mark Russinovich Breakout session (BRK 168): AI and Agent Observability in Azure AI Foundry and Azure Monitor You can also come talk to us about building, deploying and managing applications for the Adaptive cloud at the Expert Meet Up Area. Whether you are tuning in online or joining us in person, I wish you a fun and exciting Build 2025!!Welcoming the Next Wave at Build: New Partners Join the Azure Arc ISV Program
We are thrilled to announce the second round of partners joining the Azure Arc ISV Partner Program for Microsoft Build. Following its successful launch at Ignite last fall, this innovative program continues to grow, enabling partners to publish their offers on the Azure Marketplace for deployment to Arc-enabled Kubernetes clusters. With this new wave, we’re also expanding the solution landscape by introducing four new categories—Security, Networking & Service Mesh, API Infrastructure & Management, and Monitoring & Observability. These additions reflect the evolving needs of hybrid and multi-cloud environments and highlight the breadth of innovation our partners bring to the Azure Arc ecosystem. This new wave of collaborations marks a significant milestone in our journey to foster a vibrant ecosystem of innovation and excellence. This expansion marks a significant step forward in building a dynamic and innovative ecosystem that drives success for both customers and partners alike. What is Azure Arc? Azure Arc is the bridge that extends Azure to on-premises, edge, or even multi-cloud environments. It simplifies governance and management by delivering the consistency of the Azure platform. The ability to create offerings for Azure Arc in the marketplace is a significant benefit to our partners, allowing them to integrate with Azure services and tools and access a large and diverse customer base. Azure Arc enables partners to validate their applications and offer them to customers so they can manage their Kubernetes cluster on Azure. Edge developers can leverage these building blocks to develop their enterprise applications, and we aim to provide them with a one-stop shop in Azure Marketplace. Meet our partners The Azure Arc ISV Partner Program is focusing on expanding categories such as security, networking & service mesh, API infrastructure & management, monitoring & observability. We are excited to introduce our esteemed partners, HashiCorp, Traefik Labs, Solo.io, and Dynatrace, who have Arc-enabled their applications and will now be available on the Azure Marketplace. Here’s a closer look at their offerings: HashiCorp HashiCorp is a leading provider of infrastructure automation and security solutions for modern, dynamic IT environments. HashiCorp Vault Enterprise for Azure Arc enables organizations to manage access to secrets and protect sensitive data using identity-based security principles. As enterprises shift to hybrid and multi-cloud architectures, traditional perimeter-based security models fall short. Vault helps to address this challenge by authenticating every user and application, authorizing access based on identity and policy, encrypting secrets, and injecting just-in-time credentials. It also helps to automate the rotation of secrets, certificates, and encryption keys—reducing operational risk and improving compliance. By integrating with Azure Arc, Vault Enterprise can be deployed and managed alongside other Azure Arc-enabled services. This allows organizations to consistently enforce zero trust security practices—whether workloads run on-premises, in Azure, or in other cloud environments—while benefiting from centralized governance and compliance visibility through the Azure control plane. To deploy HashiCorp Vault Enterprise for Azure Arc, visit aka.ms/HashiCorpForAzureArc. To learn more about HashiCorp Vault Enterprise on Azure Arc, visit HashiCorp Vault Traefik Labs Traefik for Azure Arc empowers organizations to modernize and scale their AI and API runtime infrastructure across any Kubernetes in hybrid and multi-cloud environments. With over 3.3 billion downloads and 250,000+ production nodes globally, Traefik can be deployed in three modular and progressive phases—Application Proxy, API & AI Gateway, and API Management—meeting users where they are on their journey and enabling seamless transitions without vendor lock-in or disruptive migrations. Traefik helps deliver zero-config service discovery across Kubernetes and other orchestrators, efficiently replacing legacy tools with simplified traffic routing and management. As needs grow, they more easily transition to comprehensive AI and API Gateway capabilities with centralized authentication and authorization, semantic caching for AI workloads, and data governance for responsible AI deployments. The final evolution helps introduce complete API governance, observability, self-service developer portals, and instant mock APIs—enabling unified management across both traditional and AI-enabled services without disruptive architectural changes. By combining Azure Arc with Traefik, organizations gain more unified control over API and AI workloads, enhanced by features like semantic caching and content guard. This integration helps bridge fragmented environments, accelerates deployment, and enable clearer versioning boundaries—fundamental for scaling AI and API services across distributed systems. To deploy Traefik for Azure Arc, visit aka.ms/TraefikForAzureArc. To learn more about Traefik for Azure Arc and get started, visit aka.ms/TraefikForArcJumpstart. Solo.io Solo.io is a leading provider of service mesh and API infrastructure solutions for cloud-native applications. Istio for Azure Arc, powered by Solo.io, helps deliver an enterprise-grade service mesh experience through Istio in Ambient Mode—specifically optimized for Azure Arc-enabled Kubernetes clusters. This modern, sidecar-less architecture helps to simplify deployment, reduces operational overhead, and improves resource efficiency while maintaining Istio’s advanced capabilities. The solution provides robust Layer 7 traffic management, zero-trust security with mutual TLS and fine-grained authorization, and deep observability through distributed tracing and logging. It’s ideal for IT operations, DevOps, and security teams managing workloads in regulated industries like finance, healthcare, retail, and technology—where resilience, security, and visibility are important. By using Istio for Azure Arc, organizations can deploy and manage service mesh consistently across hybrid and multi-cloud environments, accelerating application delivery while maintaining control and compliance. To deploy Istio for Azure Arc, visit aka.ms/IstioForAzureArc. To learn more about Istio for Azure Arc, visit Istio by Solo.io. Dynatrace Dynatrace is a leading provider of AI-driven monitoring and performance analytics solutions. Dynatrace Operator helps streamlines your processes, gains insights, and accelerates innovation with its powerful AI-driven platform. Now available through the Microsoft Azure Marketplace, this solution more easily integrates with your Microsoft ecosystem—from Azure to Arc-enabled Kubernetes Service and beyond. With Dynatrace Operator, you can build custom apps and automations tailored to your unique business needs, empowering you to work smarter, not harder. Visualize and fully understand your entire Hybrid cloud ecosystem in real time, plus benefit from automated identification and illustration of application dependencies and their underlying infrastructure, delivering enriched, contextualized data for more informed decisions. Designed to help enterprises automate, analyze, and innovate faster, Dynatrace Operator is your key to unlocking efficiency and growth. By combining Azure Arc with Dynatrace Operator, organizations can deploy and manage monitoring and performance analytics consistently across hybrid and multi-cloud environments, accelerating application delivery while maintaining control and compliance. To deploy Dynatrace Operator for Azure Arc, visit aka.ms/DynatraceOperatorForArc. To learn more about Dynatrace Operator for Azure Arc, visit Dynatrace | Kubernetes monitoring. Become an Arc-enabled Partner These partners have collaborated with Microsoft to join our ISV ecosystem, helping provide resilient and scalable applications more readily accessible for our Azure Arc customers via the Azure Marketplace. Joining forces with Microsoft enables partners to stay ahead of the technological curve, strengthen customer relationships, and contribute to transformative digital changes across industries. We look forward to expanding this program to include more ISVs, enhancing the experience for customers using Arc enabled Kubernetes clusters. As we continue to expand our Azure Arc ISV Partner Program, stay tuned for more blogs on the new partners being published to the Azure Marketplace. To reach out and learn more about the Azure Arc ISV Partner Program visit: What is the Azure Arc ISV Partner program? or reach out to us at https://aka.ms/AzureArcISV.
