Announcing General Availability: Windows Server Management enabled by Azure Arc
Windows Server Management enabled by Azure Arc offers customers with Windows Server licenses that have active Software Assurances or Windows Server licenses that are active subscription licenses the following key benefits: Azure Update Manager Azure Change Tracking and Inventory Azure Machine Configuration Windows Admin Center in Azure for Arc Remote Support Network HUD Best Practices Assessment Azure Site Recovery (Configuration Only) Upon attestation, customers receive access to the following at no additional cost beyond associated networking, compute, storage, and log ingestion charges. These same capabilities are also available for customers enrolled in Windows Server 2025 Pay as you Go licensing enabled by Azure Arc. Learn more at Windows Server Management enabled by Azure Arc - Azure Arc | Microsoft Learn or watch Video: Free Azure Services for Non-Azure Windows Servers Covered by SA Powered by Azure Arc! To get started, connect your servers to Azure Arc, attest for these benefits, and deploy management services as you modernize to Azure's AI-enabled set of server management capabilities across your hybrid, multi-cloud, and edge infrastructure!Announcing Preview of Run Command on Arc-enabled servers
We are excited to announce the Public Preview of Run Command on Azure Arc-enabled servers. This feature is a game-changer for remotely and securely managing your Azure Arc-enabled servers. You can start using Azure CLI or API for Run Command today, without requiring any additional extensions or configurations, and at no additional cost.Ignite 2024: AKS enabled by Azure Arc - New Capabilities and Expanded Workload Support
Microsoft Ignite 2024 has been a showcase of innovation across the Azure ecosystem, bringing forward major advancements in AI, cloud-native applications, and hybrid cloud solutions. This year’s event featured key updates, including enhancements to AKS enabled by Azure Arc, which introduced new capabilities and expanded workload support. These updates reinforce the value and versatility that AKS enabled by Azure Arc brings to organizations looking to scale and optimize their operations. With these advancements, AKS Arc continues to support seamless management, increased scalability, and enhanced workload performance across diverse infrastructures. AKS Enabled by Azure Arc AKS enabled by Azure Arc brings the power of Azure’s managed Kubernetes service to any environment, providing consistent management and security across on-premises, edge, and multi-cloud deployments. It encompasses: AKS on Azure Local: A full-featured Kubernetes platform integrated with Azure Local for comprehensive container orchestration in hybrid setups. Notably, AKS on Azure Local has earned recognition as a leader in the 2024 Gartner Magic Quadrant for Distributed Hybrid Infrastructure, underscoring Microsoft's dedication to delivering comprehensive, enterprise-ready solutions for hybrid cloud deployments. AKS Edge Essentials: A lightweight version designed for edge computing, ensuring operational consistency on constrained hardware. AKS on Azure Local Disconnected Operations: It is now available on Azure Local Disconnected Operations. This latest addition to AKS enabled by Azure Arc portfolio is the support for fully disconnected scenario. It allows AKS enabled by Azure Arc to operate in air-gapped, isolated environments without the need for continuous Azure connectivity. It is crucial for organizations that require secure, self-sufficient Kubernetes operations in highly controlled or remote locations. With this support, businesses can maintain robust Kubernetes functionality while meeting stringent compliance and security standards. Key Features and Expanded Workload Support This year's Ignite announcements unveiled a series of public preview and GA features that enhance the capabilities of AKS enabled by Azure Arc. These advancements reflect our commitment to delivering robust, scalable solutions that meet the evolving needs of our customers. Below are the key highlights that showcase the enhanced capabilities of AKS enabled by Azure Arc: Edge Workload Azure IoT Operations - enabled by Azure Arc: Available on AKS Edge Essentials (AKS-EE) and AKS on Azure Local with public preview support. Azure IoT Operations in the management and scaling of IoT solutions. It provides robust support for deploying and overseeing IoT applications within Kubernetes environments, enhancing operational control and scalability. Organizations can leverage this tool to maintain seamless management of distributed IoT workloads, ensuring consistent performance and simplified scaling across diverse deployment scenarios. Azure Container Storage - enabled by Azure Arc: Available on both AKS Edge Essentials (AKS-EE) and AKS on Azure Local, this support enables seamless integration for persistent storage needs in Kubernetes environments. It provides scalable, reliable, and high-performance storage solutions that enhance data management and support stateful applications running in hybrid and edge deployments. This addition ensures that organizations can efficiently manage their containerized workloads with robust storage capabilities. Azure Key Vault Secret Store extension for Kubernetes: Now available as public preview on AKS Edge Essentials and AKS on Azure Local, this extension automatically synchronizes secrets from an Azure Key Vault to an AKS enabled by Azure Arc cluster for offline access, providing essential tools for proactive monitoring and policy enforcement. It offers advanced security and compliance capabilities tailored for robust governance and regulatory adherence, ensuring that organizations can maintain compliance with industry standards and best practices while safeguarding their infrastructure. Azure Monitor Pipeline: The Azure Monitor pipeline is a data ingestion solution designed to provide consistent, centralized data collection for Azure Monitor. Once deployed for AIO on AKS cluster enabled by Azure Arc, it enables at-scale telemetry data collection and routing at the edge. The pipeline can cache data locally, syncing with the cloud when connectivity is restored, and supports segmented networks where direct data transfer to the cloud isn’t possible. Built on OpenTelemetry Collector, the pipeline’s configuration includes data flows, cache properties, and destination rules defined in the DCR to ensure seamless data processing and transmission to the cloud. Arc Workload Identity Federation: Now available as public preview on AKS Edge Essentials and AKS on Azure Local, providing secure federated identity management to enhance security for customer workloads Arc Gateway: Now available as public preview for AKS Edge Essentials and AKS on Azure Local. Arc Gateway support on AKS enabled by Azure Arc enhances secure connectivity across hybrid environments, reducing required firewall rules and improving security for customer deployments. Azure AI Video Indexer - enabled by Azure Arc: Supported on AKS Edge Essentials and AKS on Azure Local. Arc-enabled Video Indexer enables comprehensive AI-powered video analysis, including transcription, facial recognition, and object detection. It allows organizations to deploy sophisticated video processing solutions within hybrid and edge environments, ensuring efficient local data processing with improved security and minimal latency. MetalLB - Azure Arc Extension: Now supported on AKS Edge Essentials and AKS on Azure Local, MetalLB ensures efficient load balancing capabilities. This addition enhances network resilience and optimizes traffic distribution within Kubernetes environments. Comprehensive AI and Machine Learning Capabilities GPUs for AI Workloads: Now AKS enabled by Azure Arc supports a range of GPUs tailored for AI and machine learning workloads with GPU Partitioning) and GPU Passthrough Virtualization support. These options enable robust performance for resource-intensive AI and machine learning workloads, allowing for efficient use of GPU resources to run complex models and data processing tasks. Arc-enabled Azure Machine Learning: Support on AKS on Azure Local, AML capabilities for running sophisticated AI models. Businesses can leverage Azure’s powerful machine learning tools seamlessly across different environments, enabling them to develop, deploy, and manage machine learning models effectively on-premises and at the edge. Arc-enabled Video Indexer: It extends Azure's advanced video analytics capabilities to AKS enabled by Azure Arc. Organizations can now process and analyze video content in real-time, harnessing Azure's robust video AI tools to enhance video-based insights and operations. This support provides businesses with greater flexibility to conduct video analysis seamlessly in remote or hybrid environments Kubernetes AI Toolchain Orchestrator (Kaito + LoRA + QLoRA): Fully validated and support for fine-tuning and optimizing AI models, Kaito, LoRA and QLoRA are designed for edge deployments such as AKS on Azure Local. This combination enhances the ability to run and refine AI applications effectively in edge environments, ensuring performance and flexibility. Flyte Integration: Now supported on AKS on Azure Local, Flyte offers a scalable orchestration platform for managing machine learning workflows. This capability enables teams to build, execute, and manage complex AI pipelines efficiently, enhancing productivity and simplifying the workflow management process. Enhanced Infrastructure and Operations Management Infrastructure as Code (IaC) with Terraform: Now supported on AKS on Azure Local for both Connected and Air-gapped scenario, providing streamlined deployment capabilities through code. This support enables teams to automate and manage their Kubernetes infrastructure at scale more efficiently with Terraform. Anti-affinity, Pod CIDR, Taints/Labels: Available on AKS on Azure Local, these features provide enhanced infrastructure capabilities by allowing refined workload placement and advanced network configuration. Anti-affinity rules help distribute pods across different nodes to avoid single points of failure, while Pod CIDR simplifies network management by allocating IP ranges to pods. Taints and labels offer greater control over node selection, ensuring that specific workloads run on designated nodes and enhancing the overall efficiency and reliability of Kubernetes operations. Optimized Windows Node Pool Management: AKS enabled by Azure Arc now includes the capability to enable and disable Windows node pools for clusters. This enhancement helps prevent unnecessary binary downloads, benefiting customers with low-speed or limited internet connection. It optimizes resource usage, reduces bandwidth consumption, and enhances overall deployment efficiency, making it ideal for environments with network constraints. Kubernetes Development AKS-WSL: With AKS-WSL, developers can set up a local environment that mimics the experience of working with AKS. This makes it easier for developers to write, debug, and test Kubernetes applications locally before deploying them to a full AKS cluster. AKS-WSL VSCode Extension: The Visual Studio Code extension for AKS-WSL allows developers to write, debug, and deploy Kubernetes applications locally, streamlining development workflows. This setup improves productivity by providing efficient tools and capabilities, making it easier to develop, test, and refine Kubernetes workloads directly from a local machine. Arc Jumpstart: Supported AKS Edge Essentials and AKS on Azure Local. Arc Jumpstart simplifies deployment initiation, providing developers with a streamlined way to set up and start working with Kubernetes environments quickly. It makes it easier for teams to evaluate and experiment with AKS enabled by Azure Arc, offering pre-configured scenarios and comprehensive guidance. By reducing complexity and setup time, Arc Jumpstart enhances the developer experience, facilitating faster prototyping and smoother onboarding for new projects in hybrid and edge settings. Conclusion Microsoft Ignite 2024 has underscored the continued evolution of AKS enabled by Azure Arc, bringing more comprehensive, scalable, and secure solutions to diverse environments. These advancements support organizations in running cloud-native applications anywhere, enhancing operational efficiency and innovation. We welcome your feedback (aksarcfeedback@microsoft.com) and look forward to ongoing collaboration as we continue to evolve AKS enabled by Azure Arc.Public Preview: Deploy OSS Large Language Models with KAITO on AKS on Azure Local
Announcement Along with Kubernetes AI Toolchain Operator (KAITO) on AKS GA release, we are thrilled to announce Public Preview refresh for KAITO on AKS on Azure Local. Customers can now enable KAITO as a cluster extension on AKS enabled by Azure Arc as part of cluster creation or day 2 using Az CLI. The seamless enablement experience makes it easy to get started with LLM deployment and fully consistent with AKS in the cloud. We also invest heavily to reduce frictions in LLM deployment such as recommending the right GPU SKU, validating preset models with GPUs and avoiding Out of Memory errors, etc. KAITO Use Cases Many of our lighthouse customers are exploring exciting opportunities to build, deploy and run AI Apps at the edge. We’ve seen many interesting scenarios like Pipeline Leak detection, Shrinkage detection, Factory line optimization or GenAI Assistant across many industry verticals. All these scenarios need a local AI model with edge data to satisfy low latency or regulatory requirements. With one simple command, customers can quickly get started with LLM in the edge-located Kubernetes cluster, and ready to deploy OSS models with OpenAI-compatible endpoints. Deploy & fine-tune LLM declaratively With KAITO extension, customers can author a simple YAML for inference workspace in Visual Studio Code or any text editor and deploy a variety of preset models ranging from Phi-4, Mistral, to Qwen with kubectl on any supported GPUs. In addition, customers can deploy any vLLM compatible text generation model from Hugging Face or even private weights models by following custom integration instructions. You can also customize base LLMs in the edge Kubernetes with Parameter Efficient Fine Tuning (PEFT) using qLoRA or LoRA method, just like the inference workspace deployment with YAML file. For more details, please visit the product documentation and KAITO Jumpstart Drops for more details. Compare and evaluate LLMs in AI Toolkit Customers can now use AI Toolkit, a popular extension in Visual Studio Code, to compare and evaluate LLMs whether it’s local or remote endpoint. With AI Toolkit playground and Bulk Run features, you can test and compare LLMs side by side and find out which model fits the best for your edge scenario. In addition, there are many built-in LLM Evaluators such as Coherence, Fluency, or Relevance that can be used to analyze model performance and generate numeric scores. For more details, please visit AI Toolkit Overview document. Monitor inference metrics in Managed Grafana The KAITO extension defaults to vLLM inference runtime. With vLLM runtime, customers can now monitor and visualize inference metrics with Azure Managed Prometheus and Azure Managed Grafana. Within a few configuration steps, e.g., enabling the extensions, labeling inference workspace, creating Service Monitor, the vLLM metrics will show up in Azure Monitor Workspace. To visualize them, customers can link the Grafana dashboard to Azure Monitor Workspace and view the metrics using the community dashboard. Please view product document and vLLM metric reference for more details. Get started today The landscape of LLM deployment and application is evolving at lightning speed - especially in the world of Kubernetes. With the KAITO extension, we're aiming to supercharge innovation around LLMs and streamline the journey from ideation to model endpoints to real-world impact. Dive into this blog as well as KAITO Jumpstart Drops to explore how KAITO can help you get up and running quickly on your own edge Kubernetes cluster. We’d love to hear your thoughts - drop your feedback or suggestions in the KAITO OSS Repo!Announcing the General Availability of Jumpstart HCIBox
Almost one year ago the Jumpstart team released the public preview of HCIBox, our self-contained sandbox for exploring Azure Stack HCI capabilities without the need for physical hardware. Feedback from the community has been fantastic, with dozens of feature requests and issues submitted and resolved through our open-source community. Today, the Jumpstart team is excited to announce the general availability of HCIBox!
AKS Arc - Optimized for AI Workloads
Overview Azure is the world’s AI supercomputer providing the most comprehensive AI capabilities ranging from infrastructure, platform services to frontier models. We’ve seen emerging needs among Azure customers to use the same Azure-based solution for AI/ML on the edge with minimized latencies while staying compliant with industry regulation or government requirement. Azure Kubernetes Service enabled by Azure Arc (AKS Arc) is a managed Kubernetes service that empowers customers to deploy and manage containerized workload whether they are in data centers or at edge locations. We want to ensure AKS Arc provides optimal experience for AI/ML workload on the edge, throughout the whole development lifecycle from AI infrastructure, Model deployment, Inference, Fine-tuning, and Application. AI infrastructure AKS Arc supports Nvidia A2, A16, and T4 for compute-intensive workload such as machine learning, deep learning, model training. When GPUs are enabled in Azure Local; AKS Arc customers can provision GPU node pools from Azure and host AI/ML workload in the Kubernetes cluster on the edge. For more details, please visit instructions from GPU Nodepool in AKS Arc. Model deployment and fine tuning Use KAITO for language model deployment, inference and fine tuning Kubernetes AI Toolchain Operator (KAITO) is an open-source operator that automates and simplifies the management of model deployments on a Kubernetes cluster. With KAITO, you can deploy popular open-source language models such as Phi-3 and Falcon, and host them in the cloud or on the edge. Along with the currently supported models from KAITO, you can also onboard and deploy custom language models following this guidance in just a few steps. AKS Arc has been validated with the latest KAITO operator via helm-based installation, and customers can now use KAITO in the edge to: Deploy language models such as Falcon, Phi-3, or their custom models Automate and optimize AI/ML model inferencing for cost-effective deployments, Fine-tune a model directly in a Kubernetes cluster, Perform parameter efficient fine tuning using low-rank adaptation (LoRA) Perform parameter efficient fine tuning using quantized adaptation (QLoRA) You can get started by installing KAITO and deploying a model for inference on your edge GPU nodes with KAITO Quickstart Guidance. You may also refer to KAITO experience in AKS in cloud: Deploy an AI model with the AI toolchain operator (Preview) Use Arc-enabled Machine Learning to train and deploy models in the edge For customers who are already familiar with Azure Machine Learning (AML), Azure Arc-enabled ML extends AML in Azure and enables customers to target any Arc enabled Kubernetes cluster for model training, evaluation and inferencing. With Arc ML extension running in AKS Arc, customers can meet data-residency requirements by storing data on premises during model training and deploy models in the cloud for global service access. To get started with Arc ML extension, please view instructions from Azure Machine Learning document . In addition, AML extension can now be used for a fully automated deployment of a curated list of pre-validated language and traditional AI models to AKS clusters, perform CPU and GPU-based inferencing, and subsequently manage them via Azure ML Studio. This experience is currently in gated preview, please view another Ignite blog for more details. Use Azure AI Services with disconnected container in the edge Azure AI services enable customers to rapidly create cutting-edge AI applications with out-of-the-box and customizable APIs and models. It simplified the developer experience to use APIs and embed the ability to see, hear, speak, search, understand and accelerate decision-making into the application. With disconnected Azure AI service containers, customers can now download the container to an offline environment such as AKS Arc and use the same APIs available from Azure. Containers enable you to run Azure AI services APIs in your own environment and are great for your specific security and data governance requirements. Disconnected containers enable you to use several of these APIs disconnected from the internet. Currently, the following containers can be run in this manner: Speech to text Custom Speech to text Neural Text to speech Text Translation (Standard) Azure AI Vision - Read Document Intelligence Azure AI Language Sentiment Analysis Key Phrase Extraction Language Detection Summarization Named Entity Recognition Personally Identifiable Information (PII) detection To get started with disconnected container, please view instructions at Use Docker containers in disconnected environments . Build and deploy data and machine learning pipelines with Flyte Flyte is an open-source orchestrator that facilitates building production-grade data and ML pipelines. It is a Kubernetes native workflow automation tool. Customers can focus on experimentation and providing business value without being an expert in infrastructure and resource management. Data scientists and ML engineers can use Flyte to create data pipelines for processing petabyte-scale data, building analytics workflow for business or finance, or leveraging it as ML pipeline for industry applications. AKS Arc has been validated with the latest Flyte operator via helm-based installation, customers are welcome to use Flyte for building data or ML pipelines. For more information, please view instructions from Introduction to Flyte - Flyte and Build and deploy data and machine learning pipelines with Flyte on Azure Kubernetes Service (AKS). AI-powered edge applications with cloud-connected control plane Azure AI Video Indexer, enabled by Azure Arc Azure AI Video Indexer enabled by Arc enables video and audio analysis, generative AI on edge devices. It runs as Azure Arc extension on AKS Arc and supports many video formats including MP4 and other common formats. It also supports several languages in all basic audio-related models. The Phi 3 language model is included and automatically connected with your Video Indexer extension. With Arc enabled VI, you can bring AI to the content for cases when indexed content can’t move to the cloud due to regulation or data store being too large. Other use cases include using on-premises workflow to lower the indexing duration latency or pre-indexing before uploading to the cloud. You can find more details from What is Azure AI Video Indexer enabled by Arc (Preview) Search on-premises data with a language model via Arc extension Retrieval Augmented Generation (RAG) is emerging to augment language models with private data, and this is especially important for enterprise use cases. Cloud services like Azure AI Search and Azure AI Studio simplify how customers can use RAG to ground language models in their enterprise data in cloud. The same experience is coming to the edge and now customers can deploy an Arc extension and ask questions about on-premises data within a few clicks. Please note this experience is currently in gated preview and please see another Ignite blog for more details. Conclusion Developing and running AI workload at distributed edges brings clear benefits such as using cloud as universal control plane, data residency, reduced network bandwidth, and low latency. We hope the products and features we developed above can benefit and enable new scenarios in Retail, Manufacturing, Logistics, Energy, and more. As Microsoft-managed Kubernetes on the edge, AKS Arc not only can host critical edge applications but also optimized for AI workload from hardware, runtime to application. Please share your valuable feedback with us (aksarcfeedback@microsoft.com) and we would love to hear from you regarding your scenarios and business impact.
