Introducing New Performance Tiers for Azure Managed Lustre: Enhancing HPC Workloads
Building upon the success of its General Availability (GA) launch last month, we’re excited to unveil two new performance tiers for Azure Managed Lustre (AMLFS): 40MB/s per TiB and 500MB/s per TiB. This blog post explores the specifics of these new tiers and how they embody a customer-centric approach to innovation.Azure Managed Lustre: not your grandparents' parallel file system
Although our new Azure Managed Lustre File System is built on tried-and-true Lustre, thinking about it as the same monolithic parallel file system you'd see on-prem leaves most of its real value on the table. Let's talk about how we've made Lustre easier to use and manage in Azure, and what new ways of approaching HPC storage open up as a result.
Moving HPC applications to object storage: a first exercise
With this blog / paper we demonstrate using object storage as the direct back end for running HPC applications on Azure. We expect this to be usable for read-compute-write application flows, where the read / write can be either the full or partial dataset.Scaling Up in the Cloud: The WEKA Data Platform and Azure HPC Windows Grid Integration
Unlocking the potential of High Performance Compute (HPC) grids in the Financial Services Industry demands a storage solution that goes beyond the limitations of traditional systems. Windows-based workloads, often overlooked by existing HPC Storage, demand a specialized approach to affordably deliver great performance and massive scale. Enter the WEKA Data Platform—a new contender in the SMB Shares arena in Azure. In our quest for high-performance, scalable storage solutions for massive Windows grid environments, we delve deeper into WEKA's capabilities and its promise to revolutionize the landscape.
HPC/AI Storage options for NDm_v4 (A100) Azure kubernetes service (AKS) cluster
We will show how to set-up and use popular azure HPC/AI storage options (such as local NVMe SSDs, Azure managed lustre Filesystem (AMLFS) and Azure files+NFSv4) in an NDm_v4 AKS cluster and provide I/O benchmark yaml files that you can execute directly on an NDm_v4 AKS cluster.Breaking the Speed Limit with WEKA File System on top of Azure Hot Blob
WEKA delivers unbeatable performance for your most demanding applications running in Microsoft Azure, supporting high I/O, low latency, small files, and mixed workloads with zero tuning and automatic storage rebalancing. Examine how WEKA’s patented filesystem, WekaFS™, and its parallel processing algorithms accelerate Blob storage performance. The WEKA® Data Platform is purpose-built to deliver speed, simplicity, and scale that meets the needs of modern enterprises and research organizations without compromise. At the heart of the WEKA® Data Platform is a modern fully distributed parallel filesystem, WekaFS™, which can span across 1,000’s of NVMe SSD spread across multiple hosts and seamlessly extend itself over compatible object storage.
Azure Managed Lustre with Automatic Synchronisation to Azure BLOB Storage
This blog post walks through how to setup an Azure Managed Lustre Filesystem (AMLFS) that will automatically synchronise to an Azure BLOB Storage container. The synchronisation is achieved using the Lustre HSM (Hierarchical Storage Management) interface combined with the Robinhood policy engine and a tool that reads the Lustre changelog and synchronises metadata with the archived storage. The lfsazsync repository on GitHub contains a Bicep template to deploy and setup a virtual machine for this purpose.
