From GlusterFS to Azure Files: A Real-World Migration Story
A few weeks ago, we received a call familiar to many cloud architects—a customer with a massive GlusterFS deployment impacted by Red Hat's end-of-support deadline (December 2024) wondering: "What now?". With hundreds of terabytes across their infrastructure serving both internal teams and external customers, moving away from GlusterFS became a business continuity imperative. Having worked with numerous storage migrations over the years, I could already see the late nights ahead for their team if they simply tried to recreate their existing architecture in the cloud. So, we rolled up our sleeves and dug into their environment to find a better way forward. The GlusterFS challenge GlusterFS emerged in 2005 as a groundbreaking open-source distributed file system that solved horizontal scaling problems when enterprise storage had to work around mechanical device limitations. Storage administrators traditionally created pools of drives limited to single systems and difficult to expand without major downtime. GlusterFS addressed this by allowing distributed storage across physical servers, each maintaining its own redundant storage. Red Hat's acquisition of GlusterFS (Red Hat to Acquire Gluster) in 2011 brought enterprise legitimacy, but its architecture reflected a pre-cloud world with significant limitations: Costly local/geo replication due to limited site/WAN bandwidth Upgrades requiring outages and extensive planning Overhead from OS patching and maintaining compliance standards Constant "backup babysitting" for offsite tape rotation 24/7 on-call staffing for potential "brick" failures Indeed, during our initial discussions, customer’s storage team lead half-jokingly mentioned having a special ringtone for middle-of-the-night "brick" failure alerts. We also noticed that they were running the share exports on SMB 3.0 and NFS 3.0, something which is considered “slightly” deprecated today. Note: In GlusterFS, a "brick" is the basic storage unit—a directory on a disk contributing to the overall volume that enables scalable, distributed storage. Why Azure Files made perfect sense With the challenges our customer faced with maintaining redundancies & administration efforts, they required a turnkey solution to manage their data. Azure Files provided them a fully managed file share service in the Cloud, offering SMB, NFS, and REST-based shares, with on-demand scaling, integrated backups & automated failover. GlusterFS was designed for large scale distributed storage systems. With Azure Files, GlusterFS customers can take advantage of up to 100TiB of Premium file or 256TiB of Provisioned V2 HDD, 10 GBPs of throughput and up to 10K IOPS for demanding workloads. The advantages of Azure Files don’t just end at performance. As customers migrate from GlusterFS to Azure files, these are the additional benefits out of the box: Azure Backup integration One-click redundancy configuration upgrades Built-in monitoring via Azure Monitor HIPAA, PCI DSS, and GDPR compliance Enterprise security through granular access control and encryption (in transit and at Rest) The financial reality At a high level, we found that migrating to Azure files was 3X cheaper than migrating to an equivalent VM based setup running GlusterFS. We compared a self-managed 3-node GlusterFS cluster (running SMB 3.0) on Azure VMs via Provisioned v2 disks with Azure Files - Premium tier (SMB 3.11). Note: All disks on VM are using Provisioned V2 for best cost saving. Region - East US2. Component GlusterFS on Azure VMs with Premium SSD v2 Disk Azure Files Premium Compute 3 x D16ads v5 VMs (16 vCPUs, 64 GiB RAM) $685.75 N/A VM OS Disks (P10) $15.42 N/A Storage 100TB Storage $11,398.18 $10,485.75 Provisioned Throughput (storage only) 2400MBps 10,340MBps Provisioned IOPS (storage only) 160000 102400 Additional Storage for Replication (~200%) $22,796.37 N/A Backup & DR Backup Solution (30 days, ZRS redundancy) $16,343.04 $4,608.00 Monthly Total $51,238.76 $15,094.75 As the table illustrates, even before we factor in the administration cost, Azure Files already has a compelling financial advantage. We also recently released “Provisioned v2” billing model for Azure files – HDD tier which provides fine grained cost management and can scale up to 256TiB!! With GlusterFS running on-premises, customers must take in account the various administrative overheads, which will be taken away with Azure Files. Factors Current (GlusterFS) Azure Files Management & Maintenance Significant None Storage Administration Personnel 15-20 hours/week Minimal Rebalancing Operations Required Automatic Failover effort Required Automatic Capacity Planning Required Automatic Scaling Complexity High None Implementation of Security Controls Required Included The migration journey We developed a phased approach tailored to the customer's risk tolerance, starting with lower-priority workloads as a pilot: Phase 1: Assessment (2-3 weeks) Inventory GlusterFS environments and analyse workloads Define requirements and select appropriate Azure Files tier Develop migration strategy Phase 2: Pilot Migration (1-2 weeks) Set up Azure Files and test connectivity Migrate test workloads and refine process Phase 3: Production Migration (variable) Execute transfers using appropriate tools (AzCopy, Robocopy, rsync // fpsync) Implement incremental sync and validate data integrity Phase 4: Optimization (1-2 weeks) Fine-tune performance and implement monitoring Decommission legacy infrastructure Results that matter Working with Tata Consultancy Services (TCS) as our migration partner, the customer did a POC migrating from a three-node RHEL 8 environment with a 1TB SMB (GlusterFS) share, to Azure Storage Account- Premium files. The source share was limited to ~1500 IOPS, and had 20+ subfolders, each being reserved for application access which made administrative tasks challenging. The application sub-folder structure was modified to individual Azure Files shares as part of the migration planning process. In addition, each share was secured using on-premises Active directory – Domain controller-based share authentication. Migration was done using Robocopy with SMB shares mounted on Windows clients and data copy being done in a mirror mode. The migration delivered significant benefits: Dramatically improved general-purpose performance due to migration of HDD based shares to SSD (1500 IOPS shared at source vs 3000 IOPS // 200MBPS base performance per share) Meeting and exceeding current RTO and RPO requirements (15 min) set by customer Customer mentioned noticeable performance gains for SQL Server workloads Flexibility to resize each share to Azure files maximum limit, independent of noise neighbours as previously configured Significant reduced TCO (at 33% of cost compared to equivalent VM based deployment) with higher base performance What this means for your GlusterFS environment If you're facing the GlusterFS support deadline, this is an opportunity to modernize your file storage approach. Azure Files offers a chance to eliminate infrastructure headaches through simplified management, robust security, seamless scalability, and compelling economics. Looking to begin your own migration? Reach out to us at azurefiles@microsoft.com, contact your Microsoft representatives, or explore our Azure Files documentation to learn more about capabilities and migration paths.Enhance Your Linux Workloads with Azure Files NFS v4.1: Secure, Scalable, and Flexible
Enhance your Linux workloads with Azure Files NFS v4.1, enterprise-grade solution. With new support for in-transit encryption and RESTful access, it delivers robust security and flexible data access for mission-critical and data-intensive applications.
Supercharge Azure Files performance for metadata-intensive workloads
Handling millions—or billions—of small files is business as usual for many cloud workloads. But behind the scenes, it’s not just about reading and writing data—it's the constant file opens, closes, directory listings, and existence checks that really impact performance. These metadata operations may seem small, but they’re critical—and can become a major bottleneck if they’re not fast. From AI/ML workloads on Azure Kubernetes Service, to web apps like Moodle, to CI/CD pipelines and virtual desktops—many applications are metadata intensive. And when every millisecond counts, latency in these operations can drag down the entire experience. That’s why we’re excited to introduce a major boost in metadata performance. Applications experience up to 55% lower latency and 2–3x more consistent response times, ensuring greater reliability. Workloads with high metadata interaction, such as AI/ML pipelines, see the biggest gains with 3x higher parallel metadata IOPS for improved efficiency and scalability. Removing metadata bottlenecks allows more data operations too. We've seen workloads increase data IOPS and throughput up to 60%. In Azure Files SSD (premium), this enhancement accelerates metadata operations for both SMB and REST protocols, benefiting new and existing file shares at no extra cost. Whether you're running critical business applications, scaling DevOps workflows, or supporting thousands of virtual desktop users, Azure Files is now faster, more scalable, and optimized for your most demanding workloads. Metadata Caching accelerates real-world solutions. GIS on Azure Virtual Desktop GIS (Geographic Information System) workloads are crucial for analyzing and managing spatial data, supporting industries like urban planning, agriculture, and disaster management. By visualizing spatial relationships, GIS helps organizations make better decisions about infrastructure and resource management. Azure Virtual Desktop (AVD) is a popular choice for hosting GIS workloads in the cloud. These workloads often experience performance bottlenecks due to frequent interactions with large volumes of smaller files on shared file storage. Metadata caching reduces latency and accelerates file interactions—such as opening and closing these files—enabling faster data access and improving GIS job execution in virtual desktop environments. Customers, like Suncor Energy, are already experiencing the impact of Metadata Caching in GIS workloads. “Enabling Metadata Cache in Azure Files SSD (premium) significantly improved geospatial (GIS) workload performance, reducing execution time by 43.18%. This enhancement boosts throughput and IOPS, increasing the value of Azure Files.” — Colin Stuckless, Suncor Energy Inc. Moodle Web Services Moodle is a comprehensive learning management system (LMS) that combines server hosting, databases (such as MySQL or PostgreSQL), file storage (using Azure Files SSD), and PHP-based web servers. It’s designed to facilitate course management, allowing instructors to upload materials, assignments, and quizzes. Moodle requires frequent read/write requests for course materials, assignments, and user interactions generating a high volume of metadata lookups, particularly when accessing shared content or navigating large course repositories. With Metadata Caching, Moodle operates faster and more efficiently. Response times have improved by 33%, reducing wait times for students and instructors when accessing materials or submitting work. These enhancements also boost Moodle’s scalability, enabling it to support 3x more students and user sessions without compromising performance. Even during peak usage, when many users are active simultaneously, Moodle remains stable and responsive. As a result, students can access resources and submit work more quickly, while instructors can manage larger courses and assignments more effectively. GitHub Actions on Azure Kubernetes Service (AKS) GitHub Actions is a powerful automation tool seamlessly integrated with GitHub, enabling developers to build, test, and deploy code directly from their repositories. By leveraging Azure Kubernetes Service (AKS), GitHub Actions automates tasks through workflows defined in AKS YAML files, facilitating efficient container orchestration, scaling, and deployment of applications within a Kubernetes environment. These workflows can be triggered by various events, such as code pushes, pull requests, or even scheduled times, streamlining the development process and enhancing efficiency. These operations generate a high volume of metadata lookups, as each workflow execution involves checking for updated dependencies, accessing cached libraries, and continuously writing execution logs. Metadata caching significantly reduces the time required to retrieve and process metadata, resulting in quicker build artifact handling and smoother, more efficient deployment cycles. As a result, pipeline execution is 57% faster, allowing developers to build and deploy in half the time! How to get started You can now supercharge your Azure Files performance by enabling metadata caching for your applications today, at no extra cost! So don’t wait! To get started, register your subscription with the Metadata Cache feature using Azure portal or PowerShell to enable all new and existing accounts with Metadata Caching. Metadata Cache is now generally available in multiple regions, with more being added as we expand coverage. For Regional Availability please visit the following Link
Azure Backup for AKS: Cloud native, Enterprise ready, Kubernetes aware backup
Kubernetes has emerged as the leading platform for deploying containerized applications, and many organizations are turning to Azure Kubernetes Service (AKS) to enjoy a managed and efficient container orchestration experience. Thanks to the continuous innovations driven by the Kubernetes community, it has become easier to store state within Kubernetes clusters and even run mission-critical databases within these clusters. As more businesses adopt stateful AKS clusters, it is imperative for IT administrators to implement robust data protection measures to safeguard their applications against accidental deletion, malicious actors, and ransomware attacks. In response to this need, we are excited to announce the General Availability of Azure Backup for AKS, a simple, cloud-native solution that enables you to protect your AKS clusters via backing up Kubernetes workloads deployed along with the application data. How Azure Backup for AKS Works Customers running stateful AKS clusters previously relied on the native Azure Disk Backup service to protect their applications stored in Persistent Volumes. While this service offered a convenient way to back up data, restoring these snapshots to the cluster required substantial effort from the customers. Furthermore, customers were not only interested in protecting volumes but also sought an application-centric approach. They wanted to back up the entire application, including both the cluster state and volumes, in a Kubernetes-aware manner. With a "Linux first" mindset, we developed a solution leveraging Velero, an open-source software that provides backup and restore capabilities for Kubernetes cluster resources and persistent volumes. Azure Backup complements this by offering orchestration and management features to our customers. This combination of open-source innovation and enterprise-grade data protection capabilities within an easy-to-use Azure-native experience significantly reduces the total cost of ownership and provides peace of mind to our customers. Getting started with Azure Backup for AKS is a breeze. You only need to install the Azure Backup extension in your cluster, and it's ready to be backed up. After installation, you can configure backups from the AKS Portal or enjoy the convenience of a single pane of glass view from the Backup Center to manage your AKS backups alongside other Azure workloads such as virtual machines and SQL Server. Key Features of Azure Backup for AKS With Azure Backup for AKS, you can configure scheduled backups for both your cluster state and application data, including persistent volumes. The cluster state is stored in a blob container, while the persistent volumes (based on Azure Disk) are backed up as snapshots in a Kubernetes-aware manner using the Container Storage Interface (CSI) Driver. This flexibility allows you to leverage backups for operational recovery or to recover from accidental data deletion at any time. One of the standout features of this solution is the fine-grained control it provides. You can choose to back up a specific namespace or an entire cluster, making it ideal for organizations with shared cluster architectures used by multiple application teams or departments. With granular namespace-level control, you can configure backups for each application based on your required Recovery Point Objectives (RPO) and Recovery Time Objectives (RTO) using backup policies. In terms of restores, the backups taken from your AKS cluster can be restored to the original cluster or to an alternate cluster, even in a different Azure subscription. This unlocks various scenarios where you can use backups to clone production environments for testing or migrate applications across clusters as part of a blue-green deployment strategy. For customers running database workloads as containers, such as MySQL, Postgres, or MongoDB, Azure Backup for AKS provides the confidence to define specific customizations for these databases during backup, ensuring application consistency. With Azure Backup for AKS, customers can now confidently run stateful applications within their AKS clusters, benefiting from the advanced orchestration capabilities of Kubernetes managed by Azure while also having their data backed up using Azure Backup. As more organizations start operating mission-critical applications in a stateful manner on AKS, Azure Backup empowers IT administrators to have a peaceful night's sleep, knowing that their applications are well-protected. This solution represents another step forward in Azure's commitment to providing robust, integrated tools for the Azure Kubernetes Service, and it ensures that your data is safe, secure, and always within your reach. Learn more Check out this article to learn more about Azure Backup for AKS and start protecting your applications running on AKS today.
