Bring compute and data together to expand your HPC possibilities. Azure CycleCloud Multi-region plugin for AMLFS combines compute autoscaling and automated job data staging in one coordinated workflow, helping teams put additional Azure regions to work for their simulations.
As HPC workloads grow, organizations need the flexibility to access more compute capacity without adding complexity to how they manage data. Expanding across regions can unlock new resources, but moving data between environments often introduces additional operational overhead. The Azure CycleCloud Multi-region plugin for Azure Managed Lustre (AMLFS) helps address this challenge by bringing compute, storage and automated data staging together in a coordinated workflow.
Azure CycleCloud Multi-region plugin for Azure Managed Lustre (AMLFS) makes regional data staging part of the familiar Slurm job lifecycle. Azure CycleCloud manages compute provisioning and autoscaling, while the integrated extension uses Slurm Burst Buffer hooks to automate declared input staging and output return to the home region. Once configured, the workflow reduces manual data handling and keeps archive and import operations outside the job's compute allocation, helping teams focus allocated cores on simulation rather than bulk transfers.
This architecture keeps ongoing simulation I/O close to the nodes, helping limit runtime and core hours spent waiting on remote I/O. Teams gain these execution options while retaining familiar Slurm submissions and application file paths.
Stage inputs in the selected spoke region, run the simulation against local storage, and return results to the home region. Azure CycleCloud and its integrated Slurm workflow connect these steps, giving teams broader execution options while preserving familiar job submissions and application file paths. Regional routing and scaling policies remain managed through the Slurm scheduler configuration.
More Regional Capacity, One Connected HPC Workflow
Expanding HPC workloads across regions gives organizations greater flexibility to access compute resources where they’re available. By keeping data close to the workloads that need it, teams can reduce unnecessary data movement and simplify how simulations are managed across regions. The result is a more flexible approach to scaling HPC environments without having to manage each region as a separate workflow.
Azure CycleCloud Multi-region plugin for AMLFS brings these pieces into one coordinated workflow. The home region hosts the Slurm scheduler node and the Azure Managed Lustre file system holding inputs and receiving results. Each spoke region pairs compute with its own Azure Managed Lustre file system for local simulation I/O. Azure CycleCloud manages compute provisioning and autoscaling, while configured Slurm partition mappings associate jobs with their execution region.
At a high level, the solution automates how data moves between the home region and the regions where compute jobs run. Input data is staged before execution, jobs access it through regional Azure Managed Lustre file systems, and outputs are returned to the home region when processing is complete. This reduces the need for teams to coordinate each data transfer manually.
Slurm Burst Buffer hooks on the home scheduler automate declared input staging through a shared Azure Blob Storage container, archiving from the home Azure Managed Lustre file system and importing into the selected spoke file system through hierarchical storage management (HSM). The job remains pending until stage in completes. Import establishes the namespace and metadata; file contents are restored on first access. Managed identity authorizes storage operations, and the Azure Managed Lustre and Blob Storage integration transfers the data. After execution, the same workflow returns declared outputs to the home file system.
Archive and import operations run outside the job's compute allocation. In an autoscaled pool starting from powered down nodes, Azure CycleCloud starts the required compute after stage in and scheduling. Once execution and normal node completion activities finish, outputs remain on the spoke Azure Managed Lustre file system while released compute can serve other jobs or scale down under configured policies. Jobs running in the home region use the home file system directly.
For HPC teams, this means more options for where workloads run, while maintaining a consistent way to submit jobs and manage data. Automated staging helps reduce manual data handling, allowing teams to focus more on running simulations and less on coordinating transfers between regions. Performance and cost can also be assessed across different regional configurations to help inform workload placement decisions.
Users add #BB_LUA stage_in and stage_out directives to their Slurm scripts, retaining uniform application and data paths through the Azure Managed Lustre file system in the selected region. Administrators configure regional mappings at the cluster level. See the job usage guide for the complete example.
Getting Started with the Multi-region plugin for AMLFS
Follow the project's quick start and architecture guide to configure networking, an Azure Managed Lustre file system in the home region and each spoke region, compatible clients, and managed identity permissions for the scheduler node in the home region. Enable the project extension through the Azure CycleCloud cluster-init configuration to integrate staging with the Slurm job lifecycle.
Start with a representative simulation and follow its complete journey: input staging from the home Azure Managed Lustre file system, data restoration on the spoke Azure Managed Lustre file system, spoke execution, compute release, and output return to the home region. Configure workload paths, permissions, regional placement, and scaling policies to match your operational requirements, then measure the workflow against your performance and cost goals.
Put more of Azure's regional HPC possibilities to work. Azure CycleCloud Multi-region plugin for AMLFS brings compute management, Azure Managed Lustre file systems in the home and spoke regions, and automated data staging together, helping teams expand execution options, simplify operations, and keep simulations at the center of the workflow.
Conclusion
Scaling HPC is about more than adding compute capacity. It’s about ensuring the right data is available wherever workloads need to run. By combining regional compute flexibility with automated data staging, the Azure CycleCloud Multi-region plugin for AMLFS offers HPC teams a more coordinated way to expand their environments while retaining familiar tools and workflows.
See it at SC26
See the “Multi-region HPC with AMLFS on Azure CycleCloud” demo at the Microsoft booth (#3411), Exhibit Hall, McCormick Place, Chicago, 16-19 November.