Fault Tolerant, Durable, Edge Kubernetes Storage with Azure Container Storage enabled by Azure Arc
Azure Container Storage enabled by Azure Arc [Previously named “Edge Storage Accelerator”] – a groundbreaking addition to our Azure storage solutions, designed to revolutionize data handling at the edge. We invite you to explore the capabilities of Edge Volumes and experience the benefits of advanced edge storage solutions firsthand. At the edge, customers have many struggles with data: sharing, resiliency, storage capacity, space management, and cloud connection, among others. We are proud to announce Azure Container Storage enabled by Azure Arc, a first-party Arc Extension designed to solve these customer Kubernetes storage challenges. ACSA offers a fault tolerant, highly available, persistent ReadWriteMany file system using Kubernetes native PVCs. Simply write to an ACSA PVC as if it were your local file system. ACSA offers two main storage configuration options. The Local Shared Edge Volume allows for shared ReadWriteMany storage that remains local to your Kubernetes cluster. This configuration is ideal for persistent application storage, such as databases, data historians, and other data processing scenarios. The Ingest Cloud Edge Volume, ACSA’s second configuration option, uploads data written by applications to Azure storage destinations, such as Blob, ADLSgen2, and OneLake Fabric. Ingest volumes also have user configurable policies for flexibility of data upload throughput and ordering. ACSA follows security best practices by leveraging Managed Identity capabilities and always implementing the latest version of the Blob SDK. Cloud Edge Volumes are disconnection tolerant to losses in network connectivity: ACSA will continue to accept application writes and will automatically upload that data once the connection is reestablished. ACSA accepts all file data you create at the edge, whether it be parquet files, time series data, photos, video, etc. With the option to keep it local to your Kubernetes cluster or send it to a cloud destination, ACSA can handle it for you. ACSA is available as a standard component of the Azure IoT Operations GA release and is suitable for production workloads. Try Out Edge Volumes Today! 📄 Get started by visiting this documentation. Jumpstart Drops make installation a breeze. Try out both Local Shared and Cloud Ingest.Outperform the competition with a unified approach to data and AI
Organizations around the world are excited by the opportunity to leverage data to improve productivity and efficiency while also creating new revenue streams and competitive differentiation. With the increasing availability of AI tools to reason over the massive quantity of data produced each day, these goals have never been closer to reality. However, the ability to effectively harness AI at scale across large, distributed data estate requires a new approach. At Ignite 2024, Microsoft is making a variety of exciting announcements regarding how customers can collect data from any source, process and analyze that data locally and globally for AI-enabled insights anywhere, and securely share those insights with employees and the ecosystem. These announcements include the General Availability of Azure IoT Operations, the General Availability of Fabric Real-Time Intelligence, new capabilities for Azure Video Indexer, enabled by Arc, and a new Arc-enabled service focused on enabling generative AI applications at the edge. These announcements build upon Azure's adaptive cloud approach, a new paradigm to accelerate adoption of cloud innovations in scenarios extending beyond our hyperscale datacenters. The adaptive cloud approach unifies hybrid, multi-cloud, and edge infrastructure with a common operations, innovation, and insights platform, enabled by Azure Arc. Organizations that choose to embrace this methodology can drive consistency in how they manage and secure infrastructure, applications and data, as well as in how they collect and contextualize data. The ability to reduce operational complexity and improve visibility can serve as a powerful combination to help organizations accelerate digital transformation outcomes. Distributed organizations have distributed data Today, it is common for organizations to have a data estate that is distributed across a variety of physical locations. According to global research firm Gartner: "Gartner® predicts that, by 2025, more than 50% of enterprise managed-data will be created and processed outside the data center or cloud.”* There are a variety of reasons why distributed data is the norm versus an exception. Many organizations have a global footprint or have operations that span multiple physical locations. Additionally, organizations in any industry may go through acquisitions, introducing new physical and digital technology infrastructure. They may do business in countries with specific data privacy or security requirements. They may also be leveraging data for real-time or near real-time business processes that require very low latency. While there are valid reasons data lives in multiple locations and systems, this reality can also impede the ability to derive value from that data. Imagine a manufacturer with a goal to improve its yield across sites, an energy company aiming to reduce its global carbon footprint - or a retailer that would like to improve the in-store experience by leveraging consumer data across brands. None of these scenarios can be accomplished with a fragmented enterprise data management strategy. In addition, many organizations are operating within complex ecosystems where the ability to share data and insights is paramount. Data sitting in multiple places and under the governance of multiple systems can also act as a barrier to valuable ecosystem participation. Leveraging an adaptive cloud approach to unify data and AI across a distributed estate As we spend time with customers around the globe with distributed data estates, it is clear a more holistic approach to data is required to achieve their business goals. There are several aspects of this holistic approach – the ability to collect data from any data source, the ability to process and analyze that data locally and globally for AI-enabled actionable insights anywhere, and the ability to securely share those insights with the people that need them to do their jobs, as well as key ecosystem partners. Microsoft’s adaptive cloud approach provides capabilities across each of these domains: Azure IoT Operations Today we’re announcing the General Availability of Azure IoT Operations, enabling IoT edge data collection, processing, and delivery of edge data into global systems. Azure IoT Operations provides the ability to connect and communicate with OPC UA servers, edge data normalization and contextualization, scalable, bidirectional edge to cloud communication through native integration with Microsoft Fabric, tooling to power event driven applications at the edge, and an intuitive user interface to manage assets and configure data pipelines. Microsoft Fabric Microsoft Fabric is a comprehensive, end-to-end analytics and data platform built to meet the needs of organizations seeking a unified solution. It brings together all aspects of data management- spanning data integration, movement, processing, ingestion, transformation, real-time event routing, and reporting - into a single platform. Since the launch of Fabric Real-Time Intelligence at Microsoft Build, there's been continued customer excitement about the ability to build real-time solutions quickly and easily to solve key business problems. And now, Fabric Real-Time Intelligence is Generally Available. Microsoft Fabric offers a complete set of capabilities for event and streaming data with Real-Time Intelligence, including ingestion, transformation, storage, analytics, visualization, tracking, AI, and real-time actions. These capabilities are critical to help derive insights and meaning from the vast amount of data available through Azure IoT Operations. Real-Time Intelligence complements Fabric’s extensive suite of capabilities, including Data Engineering, Data Factory, Data Science, Data Warehousing, Power BI, and Database management, all integrated as part of a cohesive data experience. Instead of relying on different databases or data warehouses, organizations can centralize data storage with OneLake. Analytical models can be informed by bringing together data from multiple sources in one place. AI capabilities are seamlessly embedded within Fabric, eliminating the need for manual integration. With Fabric, customers can also easily transition their raw data into actionable insights for business users. From a data governance standpoint, Microsoft Fabric provides a set of capabilities that help customers manage, protect, monitor, and improve the discoverability of an organization's sensitive information. Azure IoT Operations is Azure Arc-enabled so when deploying applications to the edge, common standards, access controls, and policies are applied to data across distributed environments as well. This consistent approach to data governance across the data estate ensures the right people and partners can access the data and insights they need to perform their business function. Scaling AI initiatives on a unified data foundation Once organizations have a consistent edge to cloud data foundation that includes proper governance and security mechanisms, they are well positioned to scale their AI initiatives across the enterprise. As organizations invest in solutions build on the Azure AI platform, they want to be able to deploy them to any location where insights are needed and manage them in a consistent way. The adaptive cloud approach extends Kubernetes to the edge with Arc-enabled Kubernetes, providing the ability to create and deploy container-based AI applications to the edge and manage those over their lifecycle. As mentioned earlier, Azure-Arc managed applications also inherit policies from the Azure Resource Manager, providing access control for AI generated insights as well. An example of such an application is Azure AI Video Indexer. Azure AI Video Indexer enabled by Arc is an Azure Arc extension enabled service that runs video and audio analysis, and generative AI on edge devices. The solution is designed to run on Azure Arc enabled Kubernetes and supports many audio and video formats, including MP4 and other common formats. At Ignite, a new multi-modal video description feature is being launched in preview. This functionality leverages the generative AI Phi3.5v to produce video summaries by examining text, audio, and visual components without the need to view the entire video. At Ignite, we are also announcing the private preview of anew Arc-enabled service focused on enabling generative AI applications at the edge. This service is deployable as a turnkey Arc for Kubernetes extension and allows customers to use industry-leading small and large language models to search their on-premises data using a technique called Retrieval Augmented Generation, or “RAG”. With it, organizations can use Azure-consistent tools and APIs to create Copilot-like experiences, while meeting data locality and compliance requirements. As the technology to derive value from data continues to become more intelligent and more accessible, the need for a modern data foundation to support those tools increases as well. This modern data foundation must support the ability to uncover and act on insights anywhere and do so in a secure manner. At Ignite this year, we’re looking forward to meeting with you to discuss how Azure’s adaptive cloud approach can help unify your distributed data estate and scale your business outcomes. Learn more: You can find a full listing of opportunities to learn more about our Adaptive cloud approach at Ignite here: Adaptive Cloud at Ignite – GitHub. Source: * Gartner, Hype Cycle™ for Edge Computing, 2024, Thomas Bittman, 15 July 2024 (Report accessible to Gartner subscribers only). GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally, and HYPE CYCLE is a registered trademark of Gartner, Inc. and/or its affiliates and are used herein with permission. All rights reserved.
Announcing the Azure Arc ISV Partner Program at Ignite
Empowering Partners and Enhancing Customer Experience We are thrilled to introduce the newly launched Azure Arc ISV Partner Program at Ignite! This innovative and growing ecosystem partner program allows them to publish offers on the Azure Marketplace that can be deployed to Arc-enabled Kubernetes clusters. Customers can now access validated, enterprise-grade applications and tools to enhance their Azure Arc development, while ISVs benefit from a deeper integration with Azure Arc services and access to the Arc enabled customer base. All marketplace images have been validated across the Azure Arc platform with the support of both Microsoft and partner teams. With the solutions each partner has made available on the Azure marketplace, the integration with Azure Arc offers central governance to build robust applications with consistent security and reliability for any hybrid deployments. What is Azure Arc? Azure Arc is a platform that extends Azure to datacenters, on-premises, edge, or even multi-cloud environments. It simplifies governance and management by delivering the consistency of the Azure platform. The ability to create offerings for Azure Arc in the marketplace is a significant benefit to our partners, allowing them to integrate with Azure services and tools and access a large and diverse customer base. Azure Arc also provides validated applications for customers to manage their Kubernetes clusters on our platform. Edge developers leverage the open-source community to build their enterprise applications, and we aim to provide them with a one-stop shop in Azure Marketplace, offering a choice of Kubernetes-based building blocks needed to develop their applications. Meet our partners With our Ignite launch, we have built the foundation of an ecosystem that is designed to bring the best capabilities and innovations to our marketplace, focused on leading building block categories: databases, big data/analytics, and messaging. We are excited to introduce our esteemed partners, (CloudCasa, MongoDB, Redis, MinIO, DataStax) who have Arc enabled their application and will now be available on the Azure Marketplace. Here’s a closer look at their offerings: CloudCasa CloudCasa is a leading provider of Kubernetes backup and recovery solutions. By Arc-enabling their application, CloudCasa offers robust, secure, and easy-to-use backup services for Kubernetes, ensuring the protection and availability of critical data. With CloudCasa, your Arc enabled Kubernetes deployments across hybrid environments are fully protected, ensure that your data is safe and recoverable, no matter the scenario. CloudCasa’s integration with Azure Arc offers three key components: handling persistent volume with or without CSI snapshots, unified management and monitoring across environments, and disaster recovery and migration for AKS hybrid. One-way CloudCasa manages persistent storage is that it natively integrates with Container Storage Interface snapshots, ensuring that all your persistent volumes can be captured and protected without interrupting your workloads. CloudCasa also provides a powerful disaster recovery and migration solution. For AKS on Azure Stack HCI, this means you can confidently deploy hybrid and edge clusters, knowing that you have a trusted solution to recover from any disaster, or even perform seamless migrations from edge to cloud or vice versa. To explore CloudCasa’s full capabilities for Azure Arc-enabled Kubernetes clusters, visit the CloudCasa Marketplace listing for Azure Arc or find out more at cloudcasa.io. For personalized assistance, feel free to contact casa@cloudcasa.io. DataStax DataStax is a leading provider of Gen AI solutions for AI developers. With DataStax HCD (Hyper-Converged Database), businesses can harness the power of Apache Cassandra, the highly scalable and resilient NoSQL database, to manage large volumes of structured and vector data with ease. By Arc-enabling their applications, DataStax HCD offers users a “single pane of glass” for streamlined deployment, monitoring, and lifecycle management of their entire infrastructure. Ensuring consistent operations across on-premises, Azure, and multi-cloud environments makes Azure with HCD an ideal choice for mission-critical applications. The combination of Azure Arc central governance and Mission Control, DataStax’s operations platform, on HCD will allow for provisioning of resources for on-premises and on the cloud. HCD brings to Azure Arc database management and the ability to support workloads and AI systems at scale with no single point of failure. DataStax HCD brings three key benefits to Azure Arc: data replication and distribution, node repair, and vector search capabilities to enhance your enterprise data workloads. To learn more about the full capabilities of DataStax HCD, please visit the DataStax HCD for Azure Arc or find out more on the HCD product page. MongoDB MongoDB Enterprise Advanced (EA) empowers customers to securely self-manage their MongoDB deployments on-premises or in hybrid environments, driving operational efficiency, performance, and control to meet specific infrastructure needs. Now with Arc-enablement, MongoDB EA allows developers to build, scale, and innovate faster by providing a robust and dynamic database solution across a multitude of environments. MongoDB’s document data model is intuitive and powerful, and it can easily handle a variety of data types and use cases efficiently. MongoDB EA includes advanced automation, reliable backups, monitoring capabilities, updating deployments, and integrating with various Kubernetes services. The MongoDB integration with Azure Arc provides three key benefits: support for multi-Kubernetes cluster deployments, centralized provisioning through the Azure portal, and leveraging the resilience of Kubernetes deployments. As Azure Arc provides centralized management of Kubernetes environments across a multitude of environments, MongoDB EA adds value with databases that can run across multiple Kubernetes clusters. To explore MongoDB EA on Azure Marketplace for Azure Arc - and to learn more about the full potential of this offering - please visit MongoDB Enterprise Advanced for Azure Arc. For licensing inquiries and to learn more about MongoDB Enterprise Advanced, please visit MongoDB's website. Redis Redis Software, an enterprise-grade, real-time data platform, offers an in-memory data structure store used as a cache, vector database, document database, streaming engine, and message broker. With its Arc-enabled application, Redis Software provides ultra-fast data access, real-time analytics, and seamless scalability. This makes Redis Software ideal for applications requiring high performance and low latency. Integrating with Azure Arc allows users to deploy Redis workloads across on-premises, Cloud and hybrid infrastructure. The benefits Redis Software brings to Azure Arc are support multi-core deployments, Active-Active geo-distribution, data tiering, high-availability with seamless failover and multiple level of on-disk persistence. As it is integrated with Arc, these Redis instances are located on-premises or on the cloud and can be managed centrally on the Azure portal. To explore Redis Software on the Azure Marketplace for Azure Arc, please visit Redis Software for Kubernetes for Azure Arc. You can learn more about licensing inquiries at Redis Software. MinIO MinIO AIStor is the standard for building large scale AI data infrastructure. It is a software-defined, S3 compatible object store that is optimized for the private cloud but will run anywhere - from the public cloud to the edge. Enterprises use AIStor to deliver against artificial intelligence, machine learning, analytics, application, backup and archival workloads - all from a single platform. It was built for the cloud operating model, so it is native to the technologies and architectures that define the cloud, such as: containerization, orchestration with Kubernetes, microservices and multi-tenancy. By Arc-enabling their application, MinIO ensures that Azure users can experience the unmatched scalability, robust security, and lightning-fast storage performance that has made MinIO the most widely integrated object store in the market today. Users can now run these hybrid or multi-cloud deployments on Azure Arc and manage them in a single pane of glass on the Azure portal. To deploy and learn more about MinIO AIStor on Azure Arc, please visit MinIO AIStor for Azure Arc here. For further information on MinIO AIStor for Azure Arc, please visit MinIO | AI Storage is Object Storage. Become an Arc Enabled Partner These partners have collaborated with Microsoft to join our ISV ecosystem, providing resilient and scalable applications readily accessible for our Azure Arc customers via the Azure Marketplace. Joining forces with Microsoft enables partners to stay ahead of the technological curve, strengthen customer relationships, and contribute to transformative digital changes across industries. We look forward to expanding this program to include more ISVs, enhancing the experience for customers using Arc enabled Kubernetes clusters. As we continue to expand our Azure Arc ISV Partner Program, stay tuned for more blogs on the new partners being published to the Azure Marketplace. To reach out and learn more about the Azure Arc ISV Partner Program, please feel free to reach out to us at https://aka.ms/AzureArcISV.Using Azure Policy to onboard multiple SQL Servers at scale to Azure Arc-enabled SQL Server - Part 1
If you're planning to onboard hundreds of instances, it's recommended to scale the Arc extension installation either through Azure policy or with a PowerShell script remotely. However, using Azure policy would be a much easier approach for managing a large number of instances, compared to manually executing a PowerShell script. In this particular blog post, I covered on how to simplify onboarding multiple SQL Servers to Azure Arc-enabled SQL Server at scale using Azure policy.Enabling Microsoft Defender for Cloud for Arc Enabled SQL Server Machines
Microsoft Defender offers robust protection and detection capabilities for SQL servers, safeguarding your databases from potential threats and vulnerabilities. In this blog post, we will explore the key features and steps to enable Microsoft Defender for SQL servers, both in Azure-native and hybrid environments. By implementing these security measures, you can ensure the integrity and safety of your SQL server deployments.Announcing the General Availability of Jumpstart HCIBox
Almost one year ago the Jumpstart team released the public preview of HCIBox, our self-contained sandbox for exploring Azure Stack HCI capabilities without the need for physical hardware. Feedback from the community has been fantastic, with dozens of feature requests and issues submitted and resolved through our open-source community.Today, the Jumpstart team is excited to announce the general availability of HCIBox!Announcing Preview of Azure AI Video Indexer Cloud-to-Edge Version
Introducing the Public Preview of Azure Video Indexer enabled by Arc: Harness the full capabilities of Video and Audio Content Anywhere. We are excited to unveil that Azure Video Indexer cloud offering is expanding, as part of Microsoft’s adaptive cloud approach. The newly released Video Indexer extension can run within Azure Arc-enabled Kubernetes cluster. This tool allows you to harness the full capabilities of video and audio content from any location. Register for the public preview by submitting this form.Evaluate SQL Server configuration using Best practices assessment for Azure Arc Enabled SQL Server
Azure Arc-enabled SQL Server extends Azure services to SQL Server instances hosted outside of Azure; in your datacenter, on the edge, or in a multi-cloud environment..Azure ARC-enabled SQL Server provides a single pane of glass for all SQL deployments irrespective of their location.This article provides instructions for using best practices assessment on an instance of Azure Arc-enabled SQL Server.
