The Azure Batch team is happy to announce a new version of Batch Shipyard with major improvements to support Singularity and a new seamless MPI interface with transparent HC/HB instance support across both Docker and Singularity containers.
The improvements to support Singularity in Batch Shipyard include the ability to use SIF™ container images and integrated support to pull SIF™ container images from Azure Container Registry via ORAS. Building on our collaboration with Sylabs, these combined improvements ease the image management and distribution of Singularity container images across Azure Batch pools while providing the ability to validate SIF™ container image provenance.
The latest release also includes a new MPI interface for multi-node job execution without having to deal with potential configuration complications between Batch multi-instance tasks, the selected MPI runtime, and the VM instance size selected. Batch Shipyard now provides an easy-to-use schema for executing your MPI jobs with support for popular MPI frameworks such as Open MPI, MPICH, MVAPICH, and Intel MPI. This new interface works seamlessly between both Singularity and Docker containers, which when combined with Azure's HB/HC instances with 100Gbit/s EDR InfiniBand, provides unparalleled performance for your distributed HPC container applications in the cloud.
HPC applications and artificial intelligence training are routinely demanding in their appetite for both compute capability and compute capacity. From support for purpose-built VM instances and popular MPI frameworks, to InfiniBand interconnect fabrics, Azure is well placed to make impressive compute capability available. That this capability can be simultaneously harnessed at cloudscale, by applications containerized via Singularity and managed with Batch Shipyard, serves as a compelling demonstration of compute capacity. This latest milestone in the ongoing collaboration between the Azure Batch team and Sylabs has resulted in a significant usability enhancement, as Batch Shipyard now permits direct use of SIF™ files available from the Azure Container Registry.- Sylabs CEO and founder Gregory Kurtzer
Please see the Batch Shipyard Change Log for a description of all recent improvements, and we welcome your feedback, suggestions and PRs!
Microsoft
