Dr. Wolfgang De Salvador - EMEA GBB HPC/AI Infrastructure Senior Specialist
Dr. Darko Mocelj - EMEA GBB HPC/AI Infrastructure Senior Specialist
Resources and references used in this article:
- Repository of the Azure CycleCloud OnDemand project
- Open OnDemand Portal
- Open OnDemand Portal Documentation
- OnDemand Job template for OpenFOAM
Visit us at ISC24 - Microsoft Booth #F30 for a live demo of this integration on Wednesday, the 15th | 9 AM - 12.30 PM
Introduction
As of today, several tools, frameworks and automations allow the deployment of HPC infrastructures in a cloud environment.
Azure CycleCloud enables users and IT administrators to run cloud-only or hybrid (bursting) clusters leveraging traditional HPC schedulers like OpenPBS, SGE, Altair PBS Professional and Slurm.
Azure CycleCloud provides the possibility to preserve standard submission interfaces from on-premises HPC systems, without the need to re-architect or alter by any means the standard simulation workflows. In this way, end users can keep running and using their standard applications without any disruption.
Azure CycleCloud provides out of the box the possibility for interaction and cluster operations only using a standard SSH connection for the end users or scheduler default APIs (e.g. Slurm APIs).
This blog post presents an Azure CycleCloud project allowing to deploy an Open OnDemand portal, an efficient open-source web portal for job submission, job monitoring, file management and remote desktop/application sessions.
This project allows to deploy an Open OnDemand Portal like az-hop, but allowing the user to just deploy a single VM with a portal to be attached to an already existing and configured Azure CycleCloud cluster.
What is required as prerequisites
- The project requires a working installation of Azure CycleCloud.
- At the same time, Open OnDemand portal is meant to be attached to an existing Slurm or OpenPBS cluster deployed in Azure CycleCloud. Support for additional schedulers is planned to be available in future releases.
- The deployment relies on an Azure Key Vault and a Managed Identity for secret access during the deployment. The following elements need to be uploaded depending on the selected SSL and authentication mechanism need in the Azure Key Vault:
- SSL certificate
Required to avoid using self-signed certificate. - Secret for the OIDC Client
This is required in case the authentication mechanism is OIDC. This will require an Entra ID app registration - LDAPS CA Certificate
This is required only in case of OIDC Dex LDAP if the LDAPS server requires a dedicated certification authority for SSL connections - LDAP Service Account BIND password
This is required only in case of OIDC Dex LDAP among with a read only account
- SSL certificate
Basic authentication option and self-signed SSL certificates should be considered only for test/development purpose, away from production systems because of the security concerns
How to deploy
The project can be deployed following the step-by-step guide provided in the README of the GitHub repository.
The steps involved in getting the project accessible inside Azure CycleCloud are:
- Definition of a custom cluster-init project source in Azure CycleCloud
- Import of the OnDemand Portal template in Azure CycleCloud
What the project will deliver
The project will deploy a single sever hosting an Open OnDemand portal allowing the users to specify:
- The general server configuration in terms of Azure VM Size, Virtual Network, IP Address and server name.
- The cluster to which the OnDemand Portal should be attached in terms of scheduler type (OpenPBS/Slurm) and scheduler version
- The shared NFS file system to be attached in /shared and /sched as common shared file systems for users and file management
- The OnDemand Portal configuration in terms of:
- Authentication
- OIDC
- OIDC Dex LDAP
- Basic PAM (insecure)
- SSL termination
- Bring-your-own certificate
- Self-signed (insecure)
- Authentication
All the secrets and certificates involved in the configuration are safely stored inside an Azure Key Vault which is accessed by the Azure CycleCloud nodes through a Managed Identity.
After the cluster is successfully deployed, the user will be able to have a basic interface to access the main OnDemand functionality:
- In-browser SSH connection
- Ability to upload/download files
- Possibility to customize the OnDemand portal configuration with dedicated Interactive Apps templates.
Additional considerations
Open OnDemand portal must be able to map the username provided by an external authentication mechanism like OIDC or OIDC Dex LDAP to a local Linux user account. This will be the Linux account that will be impersonating the user and interacting with the cluster through Open OnDemand.
This is something that remains responsibility of the user following the Open OnDemand documentation.
An easy way to realize this is to enable Azure CycleCloud EntraID and using the following additional configuration in Open OnDemand Portal:
user_map_match: '^([^@]+)@example.com$'
oidc_remote_user_claim: "email"
This will map the users authenticated from EntraID directly to a local user in the system. The configuration above can be inputted directly from Azure CycleCloud UI and respectively:
- the user map match in the additional configuration
- the OIDC Remote user claim in the authentication section
Creation of Interactive Desktop Sessions with auto-scaling in Azure CycleCloud
As already extensively implemented and developed in az-hop, Open OnDemand allows to create on-demand interactive Desktop Session or Interactive App session with nodes dynamically allocated by Azure CycleCloud.
The underlying concept is that a Desktop session will be submitted as a job to the scheduler and Azure CycleCloud will allocate the required nodes for the session duration.
In order to get this up in OnDemand, the steps are:
- Defining a dedicated nodearray in Azure CycleCloud for the purpose. There is an example of a Slurm cluster template based on 3.0.6 version of the CycleCloud project inside the repository.
- Building a dedicated OS image on Azure with the required Desktop environment. An example script for non-accelerated GPU environments is available in the repository. For GPU accelerated environment it is possible to install also the dedicated driver and VirtualGL.
- Configuring Open OnDemand portal Desktop form attributes and enabling reverse proxy.
There is the plan in a future project release to integrate this configuration also in an automation.
Creation of a submission batch application
In a similar way of interactive session, Open OnDemand allows to define submission forms for specific batch submission logics.
For example, here an example of integrating OpenFOAM submission in the OpenOnDemand portal:
Once the job is finished, it can be visualized using a Desktop session GPU accelerated:
Visit us at ISC24 - Microsoft Booth #F30 for a live demo of this integration on Wednesday, the 15th | 9 AM - 12.30 PM
