Public Preview: Deploy OSS Large Language Models with KAITO on AKS on Azure Local
Announcement Along with Kubernetes AI Toolchain Operator (KAITO) on AKS GA release, we are thrilled to announce Public Preview refresh for KAITO on AKS on Azure Local. Customers can now enable KAITO as a cluster extension on AKS enabled by Azure Arc as part of cluster creation or day 2 using Az CLI. The seamless enablement experience makes it easy to get started with LLM deployment and fully consistent with AKS in the cloud. We also invest heavily to reduce frictions in LLM deployment such as recommending the right GPU SKU, validating preset models with GPUs and avoiding Out of Memory errors, etc. KAITO Use Cases Many of our lighthouse customers are exploring exciting opportunities to build, deploy and run AI Apps at the edge. We’ve seen many interesting scenarios like Pipeline Leak detection, Shrinkage detection, Factory line optimization or GenAI Assistant across many industry verticals. All these scenarios need a local AI model with edge data to satisfy low latency or regulatory requirements. With one simple command, customers can quickly get started with LLM in the edge-located Kubernetes cluster, and ready to deploy OSS models with OpenAI-compatible endpoints. Deploy & fine-tune LLM declaratively With KAITO extension, customers can author a simple YAML for inference workspace in Visual Studio Code or any text editor and deploy a variety of preset models ranging from Phi-4, Mistral, to Qwen with kubectl on any supported GPUs. In addition, customers can deploy any vLLM compatible text generation model from Hugging Face or even private weights models by following custom integration instructions. You can also customize base LLMs in the edge Kubernetes with Parameter Efficient Fine Tuning (PEFT) using qLoRA or LoRA method, just like the inference workspace deployment with YAML file. For more details, please visit the product documentation and KAITO Jumpstart Drops for more details. Compare and evaluate LLMs in AI Toolkit Customers can now use AI Toolkit, a popular extension in Visual Studio Code, to compare and evaluate LLMs whether it’s local or remote endpoint. With AI Toolkit playground and Bulk Run features, you can test and compare LLMs side by side and find out which model fits the best for your edge scenario. In addition, there are many built-in LLM Evaluators such as Coherence, Fluency, or Relevance that can be used to analyze model performance and generate numeric scores. For more details, please visit AI Toolkit Overview document. Monitor inference metrics in Managed Grafana The KAITO extension defaults to vLLM inference runtime. With vLLM runtime, customers can now monitor and visualize inference metrics with Azure Managed Prometheus and Azure Managed Grafana. Within a few configuration steps, e.g., enabling the extensions, labeling inference workspace, creating Service Monitor, the vLLM metrics will show up in Azure Monitor Workspace. To visualize them, customers can link the Grafana dashboard to Azure Monitor Workspace and view the metrics using the community dashboard. Please view product document and vLLM metric reference for more details. Get started today The landscape of LLM deployment and application is evolving at lightning speed - especially in the world of Kubernetes. With the KAITO extension, we're aiming to supercharge innovation around LLMs and streamline the journey from ideation to model endpoints to real-world impact. Dive into this blog as well as KAITO Jumpstart Drops to explore how KAITO can help you get up and running quickly on your own edge Kubernetes cluster. We’d love to hear your thoughts - drop your feedback or suggestions in the KAITO OSS Repo!Welcoming the Next Wave at Build: New Partners Join the Azure Arc ISV Program
We are thrilled to announce the second round of partners joining the Azure Arc ISV Partner Program for Microsoft Build. Following its successful launch at Ignite last fall, this innovative program continues to grow, enabling partners to publish their offers on the Azure Marketplace for deployment to Arc-enabled Kubernetes clusters. With this new wave, we’re also expanding the solution landscape by introducing four new categories—Security, Networking & Service Mesh, API Infrastructure & Management, and Monitoring & Observability. These additions reflect the evolving needs of hybrid and multi-cloud environments and highlight the breadth of innovation our partners bring to the Azure Arc ecosystem. This new wave of collaborations marks a significant milestone in our journey to foster a vibrant ecosystem of innovation and excellence. This expansion marks a significant step forward in building a dynamic and innovative ecosystem that drives success for both customers and partners alike. What is Azure Arc? Azure Arc is the bridge that extends Azure to on-premises, edge, or even multi-cloud environments. It simplifies governance and management by delivering the consistency of the Azure platform. The ability to create offerings for Azure Arc in the marketplace is a significant benefit to our partners, allowing them to integrate with Azure services and tools and access a large and diverse customer base. Azure Arc enables partners to validate their applications and offer them to customers so they can manage their Kubernetes cluster on Azure. Edge developers can leverage these building blocks to develop their enterprise applications, and we aim to provide them with a one-stop shop in Azure Marketplace. Meet our partners The Azure Arc ISV Partner Program is focusing on expanding categories such as security, networking & service mesh, API infrastructure & management, monitoring & observability. We are excited to introduce our esteemed partners, HashiCorp, Traefik Labs, Solo.io, and Dynatrace, who have Arc-enabled their applications and will now be available on the Azure Marketplace. Here’s a closer look at their offerings: HashiCorp HashiCorp is a leading provider of infrastructure automation and security solutions for modern, dynamic IT environments. HashiCorp Vault Enterprise for Azure Arc enables organizations to manage access to secrets and protect sensitive data using identity-based security principles. As enterprises shift to hybrid and multi-cloud architectures, traditional perimeter-based security models fall short. Vault helps to address this challenge by authenticating every user and application, authorizing access based on identity and policy, encrypting secrets, and injecting just-in-time credentials. It also helps to automate the rotation of secrets, certificates, and encryption keys—reducing operational risk and improving compliance. By integrating with Azure Arc, Vault Enterprise can be deployed and managed alongside other Azure Arc-enabled services. This allows organizations to consistently enforce zero trust security practices—whether workloads run on-premises, in Azure, or in other cloud environments—while benefiting from centralized governance and compliance visibility through the Azure control plane. To deploy HashiCorp Vault Enterprise for Azure Arc, visit aka.ms/HashiCorpForAzureArc. To learn more about HashiCorp Vault Enterprise on Azure Arc, visit HashiCorp Vault Traefik Labs Traefik for Azure Arc empowers organizations to modernize and scale their AI and API runtime infrastructure across any Kubernetes in hybrid and multi-cloud environments. With over 3.3 billion downloads and 250,000+ production nodes globally, Traefik can be deployed in three modular and progressive phases—Application Proxy, API & AI Gateway, and API Management—meeting users where they are on their journey and enabling seamless transitions without vendor lock-in or disruptive migrations. Traefik helps deliver zero-config service discovery across Kubernetes and other orchestrators, efficiently replacing legacy tools with simplified traffic routing and management. As needs grow, they more easily transition to comprehensive AI and API Gateway capabilities with centralized authentication and authorization, semantic caching for AI workloads, and data governance for responsible AI deployments. The final evolution helps introduce complete API governance, observability, self-service developer portals, and instant mock APIs—enabling unified management across both traditional and AI-enabled services without disruptive architectural changes. By combining Azure Arc with Traefik, organizations gain more unified control over API and AI workloads, enhanced by features like semantic caching and content guard. This integration helps bridge fragmented environments, accelerates deployment, and enable clearer versioning boundaries—fundamental for scaling AI and API services across distributed systems. To deploy Traefik for Azure Arc, visit aka.ms/TraefikForAzureArc. To learn more about Traefik for Azure Arc and get started, visit aka.ms/TraefikForArcJumpstart. Solo.io Solo.io is a leading provider of service mesh and API infrastructure solutions for cloud-native applications. Istio for Azure Arc, powered by Solo.io, helps deliver an enterprise-grade service mesh experience through Istio in Ambient Mode—specifically optimized for Azure Arc-enabled Kubernetes clusters. This modern, sidecar-less architecture helps to simplify deployment, reduces operational overhead, and improves resource efficiency while maintaining Istio’s advanced capabilities. The solution provides robust Layer 7 traffic management, zero-trust security with mutual TLS and fine-grained authorization, and deep observability through distributed tracing and logging. It’s ideal for IT operations, DevOps, and security teams managing workloads in regulated industries like finance, healthcare, retail, and technology—where resilience, security, and visibility are important. By using Istio for Azure Arc, organizations can deploy and manage service mesh consistently across hybrid and multi-cloud environments, accelerating application delivery while maintaining control and compliance. To deploy Istio for Azure Arc, visit aka.ms/IstioForAzureArc. To learn more about Istio for Azure Arc, visit Istio by Solo.io. Dynatrace Dynatrace is a leading provider of AI-driven monitoring and performance analytics solutions. Dynatrace Operator helps streamlines your processes, gains insights, and accelerates innovation with its powerful AI-driven platform. Now available through the Microsoft Azure Marketplace, this solution more easily integrates with your Microsoft ecosystem—from Azure to Arc-enabled Kubernetes Service and beyond. With Dynatrace Operator, you can build custom apps and automations tailored to your unique business needs, empowering you to work smarter, not harder. Visualize and fully understand your entire Hybrid cloud ecosystem in real time, plus benefit from automated identification and illustration of application dependencies and their underlying infrastructure, delivering enriched, contextualized data for more informed decisions. Designed to help enterprises automate, analyze, and innovate faster, Dynatrace Operator is your key to unlocking efficiency and growth. By combining Azure Arc with Dynatrace Operator, organizations can deploy and manage monitoring and performance analytics consistently across hybrid and multi-cloud environments, accelerating application delivery while maintaining control and compliance. To deploy Dynatrace Operator for Azure Arc, visit aka.ms/DynatraceOperatorForArc. To learn more about Dynatrace Operator for Azure Arc, visit Dynatrace | Kubernetes monitoring. Become an Arc-enabled Partner These partners have collaborated with Microsoft to join our ISV ecosystem, helping provide resilient and scalable applications more readily accessible for our Azure Arc customers via the Azure Marketplace. Joining forces with Microsoft enables partners to stay ahead of the technological curve, strengthen customer relationships, and contribute to transformative digital changes across industries. We look forward to expanding this program to include more ISVs, enhancing the experience for customers using Arc enabled Kubernetes clusters. As we continue to expand our Azure Arc ISV Partner Program, stay tuned for more blogs on the new partners being published to the Azure Marketplace. To reach out and learn more about the Azure Arc ISV Partner Program visit: What is the Azure Arc ISV Partner program? or reach out to us at https://aka.ms/AzureArcISV.Troubleshoot the Azure Arc Agent in Azure using Azure Monitor & Log Analytics Workspace
This article explores how to centralize logging from on-premises servers—both physical and virtual—into a single Log Analytics Workspace. The goal is to enhance monitoring capabilities for the Azure Arc Connected Machine Agent running on these servers. Rather than relying on scattered and unstructured .log files on individual machines, this approach enables customers to collect, analyze, and gain insights from multiple agents in one centralized location. This not only simplifies troubleshooting but also unlocks richer observability across the hybrid environment.Arc Jumpstart Newsletter: April 2025 Edition
We’re thrilled to bring you the latest updates from the Arc Jumpstart team in this month’s newsletter. Whether you are new to the community or a regular Jumpstart contributor, this newsletter will keep you informed about new releases, key events, and opportunities to get involved in within the Azure Adaptive Cloud ecosystem. Check back each month for new ways to connect, share your experiences, and learn from others in the Adaptive Cloud community.Announcing Azure Local Public Preview on Microsoft Azure Government Cloud
Today, we are thrilled to announce that Azure Local is now available for Azure Government customers in public preview. Building upon the earlier introduction of Azure Local, we are excited to bring this innovative solution to government agencies. Government customers benefit from the same user experience as they would in the Azure public cloud. Azure Local enables organizations to deploy cloud-connected infrastructure at their own physical locations, under their operational control, while maintaining compliance with stringent regulations and security standards. This innovative approach empowers customers to leverage the benefits of Azure’s robust ecosystem—including compute, networking, storage, and application services—locally, while seamlessly integrating with broader cloud workflows. Key Features of Azure Local Deployment and Management Simplified Azure Local instances can be deployed, configured, updated, and monitored using the Azure portal or infrastructure-as-code tools like ARM templates. This streamlines the process, making it predictable and repeatable while reducing IT operational costs and complexity. Azure Local is now available for deployment in the US Gov Virginia region in preview. It provides a streamlined workflow in the Azure portal, allowing customers to configure clusters, networking, and storage to suit the unique requirements of their environments. Azure Local integrates natively with Azure Monitor for unified observability across cloud resources and distributed locations, enabled by Azure Arc. You can monitor your distributed VMs, Kubernetes clusters, and physical infrastructure from a single pane of glass. Azure Local comes with 60+ standard Metrics, out-of-the-box Insights dashboards, and Alerts rules for the infrastructure stack. including creating virtual machines, Kubernetes clusters and performing cluster updates. You can conveniently view and manage Azure Local updates directly in Azure Update Manager, alongside other cloud resources. Select one or multiple Azure Local instances and apply updates with just a few clicks. Behind the scenes, Azure Local orchestrates moving workloads and updating each physical machine in sequence, to ensure that updates in multi-node environments are non-disruptive (workloads keep running). You always control when to apply updates. Ready for VMs and containers Many critical workloads run as VMs. Azure Local offers general-purpose VMs with flexible sizing and configuration options to meet your application requirements. Specify the specs, networking, and storage you need, and either bring your own custom VM image or conveniently access ones from the Azure Marketplace. Secure by default Azure Local is deployed with a hardened infrastructure security posture by default. To help you detect and correct drift over time, Azure Local integrates with Microsoft Defender for Cloud. This provides unified security tooling for all your resources, across cloud regions and distributed locations, and can protect against existing and evolving threats. The Extended Security Update (ESU) program enables you to get important security patches for legacy Microsoft products that are past the end of support. Getting ESU through Azure Local comes with additional benefits and implementation steps. For details, check Extended Security Updates (ESU) on Azure Local - Azure Local | Microsoft Learn Trusted launch is a security option that hardens VMs against malware-based rootkits and boot kits. Trusted launched VMs get a virtual Trusted Platform Module (vTPM) that enables Secure Boot and guest OS features like BitLocker data encryption. vTPM state is seamlessly preserved when the VM moves around the Azure Local cluster, enabling live migration and automatic failover. How to Get Started Simply follow the workflow in the Azure Government portal to download the latest Azure Local OS image and create your Azure Local instance. Customize your deployment based on your needs, including cluster configuration, networking, and storage options. Azure Local empowers government agencies to harness the power of distributed cloud infrastructure with confidence and ease. Conclusion Azure Local in public preview for Azure Government represents a significant step forward in enabling secure, scalable, and efficient distributed cloud infrastructure for government customers. By combining the power of Azure Arc with on-site operational control, Azure Local ensures that agencies can meet their unique needs while benefiting from a unified cloud experience. We encourage government customers to try out Azure Local during the public preview and explore how this solution can transform their infrastructure operations. Stay tuned for more updates and enhancements as we continue to innovate and expand capabilities to support your mission-critical goals.Arc Jumpstart Newsletter: March 2025 Edition
We’re thrilled to bring you the latest updates from the Arc Jumpstart team in this month’s newsletter. Whether you are new to the community or a regular Jumpstart contributor, this newsletter will keep you informed about new releases, key events, and opportunities to get involved in within the Azure Adaptive Cloud ecosystem. Check back each month for new ways to connect, share your experiences, and learn from others in the Adaptive Cloud community.Announcing Private Preview: ArgoCD through Microsoft GitOps
We're excited to announce the Private Preview for Microsoft GitOps ArgoCD. Delivered as a cluster extension across Azure Kubernetes Service (AKS) and Azure Arc-enabled Kubernetes, Microsoft GitOps delivers a consistent and robust management, security, and deployment experience for ArgoCD across your heterogeneous environments. This capability complements Microsoft GitOps existing support for Flux, which is currently in General Availability. By signing up for the Private Preview, you'll get access to the ArgoCD cluster extension and the opportunity to connect with and provide feedback to the Microsoft GitOps product group. Sign up today at https://aka.ms/MicrosoftGitOpsPreviewSignup. Advantages of the current Microsoft GitOps experience for ArgoCD include: Simplified, templatized deployment as a cluster extension Managed and automated upgrade reducing overhead Official supportability and security for enterprise readiness Integration with Azure identity and authentication We look forward to continuing to deliver on an exceptional Microsoft GitOps experience across ArgoCD and Flux for customers running containerized workloads not only on Azure, but also on on-premises and other public clouds through Azure Arc.Arc Jumpstart Newsletter: February 2025 Edition
We’re thrilled to bring you the latest updates from the Arc Jumpstart team in this month’s newsletter. Whether you are new to the community or a regular Jumpstart contributor, this newsletter will keep you informed about new releases, key events, and opportunities to get involved in within the Azure Adaptive Cloud ecosystem. Check back each month for new ways to connect, share your experiences, and learn from others in the Adaptive Cloud community.Arc Jumpstart Newsletter: January 2025 Edition
We’re thrilled to bring you the latest updates from the Arc Jumpstart team in this month’s newsletter. Whether you are new to the community or a regular Jumpstart contributor, this newsletter will keep you informed about new releases, key events, and opportunities to get involved in within the Azure Adaptive Cloud ecosystem. Check back each month for new ways to connect, share your experiences, and learn from others in the Adaptive Cloud community.
