Blog Post

Azure High Performance Computing (HPC) Blog
5 MIN READ

Scale Azure Managed Lustre as your AI and HPC workloads grow

sachincsheth's avatar
sachincsheth
Icon for Microsoft rankMicrosoft
Nov 19, 2026

Azure Managed Lustre file systems can now expand capacity and nominal maximum throughput in place while remaining online and mounted. Organizations can add storage as workloads grow—without migrating data or interrupting running jobs

AI and high-performance computing workloads rarely remain at their originally planned size. New and curated data expands training datasets, larger models generate more checkpoints, and simulation campaigns produce results faster than expected. Adding GPU and CPU resources compounds that growth by increasing demand on shared storage.

Previously, an Azure Managed Lustre file system’s capacity was fixed at creation. Organizations had to provision for anticipated growth or create a larger file system and migrate their data when they needed more capacity. In-place expansion removes that tradeoff.

During an in-place expansion, the Azure Managed Lustre file system stays online and available to clients. Existing mounts remain valid, and applications can continue reading and writing while additional capacity comes online. Because each performance tier provides fixed bandwidth per TiB, the expansion increases nominal maximum throughput in proportion to capacity. This proportional scaling helps storage keep pace with both large HPC compute fleets and AI platforms that depend on sustained data delivery and fast checkpoint I/O—all without replacing the file system or interrupting active mounts.

Keep AI and HPC jobs running

You do not need to schedule downtime, remount clients, or restart pipelines solely to add capacity. The file system remains online and mounted during expansion, so clients can continue accessing data. For shared file systems serving large compute environments, planned downtime or maintenance can mean draining job queues, leaving accelerators idle, and disrupting long-running training or simulation workloads. In-place expansion lets storage grow without those interruptions.

Grow storage and throughput with the workload

In-place expansion lets organizations start with the storage required for today’s workload and add capacity as datasets, checkpoints, output, and compute environments grow. This reduces the need to overprovision at deployment simply to accommodate an uncertain future peak.

Azure Managed Lustre offers performance tiers of 20, 40, 125, 250, and 500 MB/s per TiB. Each tier has its own capacity increment and maximum supported capacity; see Azure Managed Lustre scale targets and throughput configurations for the current values. For example, a file system in the 250 MB/s per TiB tier expands in 8 TiB increments, and each increment adds 2 GB/s of nominal maximum throughput.

Requested capacity values are rounded up to the next supported increment. Before starting an expansion, review the resulting capacity, nominal maximum throughput, and expected cost. Although nominal maximum throughput increases with capacity according to the selected tier, realized performance depends on I/O characteristics, client networking, network topology, and how file data and metadata are distributed across the file system.

Expand first, then rebalance when needed

Expansion and rebalancing address related but distinct needs. Expansion adds storage capacity and nominal maximum throughput. Afterward, optional rebalancing can help existing datasets and directory trees use the expanded resources more evenly.

Lustre stores file data on Object Storage Targets (OSTs) and file and directory metadata on Metadata Targets (MDTs). Expansion adds OSTs and, when applicable, MDTs. Newly added targets begin empty. New content can use them, but existing data and established directory trees remain on their original targets.

Without rebalancing, established datasets can continue concentrating capacity consumption and I/O activity on the original targets while new targets remain underused, creating potential capacity and performance hotspots. Rebalancing redistributes existing file data across OSTs and directory metadata across available MDTs, helping existing workloads make fuller use of the added space, data bandwidth, and metadata capacity.

Rebalancing with expansion is available in public preview and is optional; the added capacity is available without it. Customers choose whether to rebalance when they initiate the expansion; rebalancing cannot currently be started as a separate operation after the expansion completes. In the Azure portal, rebalancing is off by default and requires opt-in. For expansions initiated through Azure CLI, the Azure StorageCache Management SDK, Bicep, Terraform, or ARM templates, rebalancing is on by default unless disabled. Before starting the expansion, compare utilization across OSTs and MDTs and decide whether established datasets and directory trees should be redistributed. Because rebalancing moves existing data and metadata and consumes file-system resources, schedule the expansion and rebalancing around workload priorities, monitor the operation, and allow it to finish before starting another expansion.

 

Figure 1. Expansion adds capacity and nominal maximum throughput while the file system remains online. Optional rebalancing can then redistribute existing data and metadata across the expanded resources.

Expansion options: Use the Azure portal or automation guidance, including Azure CLI, the Azure StorageCache Management SDK, Bicep, Terraform, and ARM templates. For Azure CLI syntax, see az amlfs expansion.

Expand from the Azure portal

You can initiate an expansion from the Azure Managed Lustre resource in the Azure portal. The workflow lets you specify the new total capacity, review the resulting nominal maximum throughput, and monitor progress. During the operation, the file system shows an Expanding state; it returns to Available when the additional capacity is online.

 

Figure 2. Start an in-place expansion from the Azure Managed Lustre Overview page in the Azure portal.

  1. Open the file system. In the Azure portal, open the Azure Managed Lustre file system to expand.
  2. Choose rebalancing before you start. Before submitting the expansion, evaluate OST and MDT utilization and decide whether to include rebalancing so established datasets can use the added resources more evenly.
  3. Start the expansion. On the Overview page, select Expand.
  4. Enter the new capacity. Specify the desired total storage capacity. Values that do not align with the tier’s increment are rounded up.
  5. Review the result. Confirm the final capacity, nominal maximum throughput, and expected cost.
  6. Monitor the expansion. Track progress as the file system moves from Expanding back to Available.

Before you start

  • Confirm that the file system is in the Available state.
  • Choose a capacity larger than the current size. Before proceeding, the service validates the request against the selected tier’s supported increments and maximum capacity.
  • Plan subnet capacity before deployment. Size the file-system subnet for the largest Azure Managed Lustre configuration you may need. The subnet cannot be resized later, and an existing Azure Managed Lustre file system cannot be moved to another subnet. Use az amlfs get-subnets-size to estimate the subnet size required for the target configuration.
  • Before starting another expansion, confirm that any rebalancing job from a previous expansion is complete.
  • Remember that expansion is one-way: after the file system grows, its capacity and throughput cannot be reduced.
  • Plan for regional capacity and SKU limits. Expansion is subject to available regional capacity and maximum SKU-size restrictions. If the required size is unavailable or the expansion request returns a capacity-related error, open an Azure support request.
  • Keep the expanded file system within the maximum capacity supported by its selected throughput tier.

Get started

Use in-place expansion when a file system needs more capacity and migrating data solely to accommodate growth would add unnecessary operational work. Consult the Azure Managed Lustre expansion documentation for current availability, supported configurations, automation options, tier-specific increments and limits, and troubleshooting guidance. Start with what your workloads need today. Expand as your data grows. Keep AI and HPC running.

Learn more

 

 

 

 

 

 

 

 

 

 

 

 

Updated Nov 19, 2026
Version 1.0