Introduction

Open OnDemand, developed by the Ohio Supercomputer Center (OSC), is an open-source, web-based portal designed to offer seamless access to high-performance computing (HPC) resources. The integration with Azure CycleCloud Workspace for Slurm facilitates the deployment and configuration of an Open OnDemand virtual machine using a dedicated CycleCloud cluster template and project scripts. Upon the completion of setup, a local daemon will query CycleCloud to automatically register Slurm clusters.

User authentication will be managed through Microsoft Entra ID, requiring the registration and configuration of a Microsoft Entra ID Application to support the OpenID Connect mechanism. CycleCloud will manage local user administration for clusters.

Additionally, a Visual Studio Code application will be pre-configured on Open OnDemand, enabling users to run VSCode in web mode on login nodes or dynamically provisioned compute nodes.

Pre-requisites

You should have some knowledge on how to deploy and use Azure CycleCloud Workspace for Slurm for which the public documentation is available here.

Networking

Direct connectivity to the Open OnDemand virtual machine is required due to the necessity of an SSL certificate and authentication using Microsoft Entra ID. Since this solution does not enable public IP by default, it is essential to establish a VPN connection from the machine used to access the Open OnDemand web portal. Please note that Azure Bastion tunneling is not supported in this context.

File-system

The home directory must be shared between the Slurm cluster and the Open OnDemand virtual machine because the latter needs to map authenticated users to local Linux users. As such, it is recommended to use Azure NetApp Files for the /shared and /home directory settings or to mount an existing network file share. Furthermore, it is advised to strictly use a built-in network file share for testing purposes, as termination of an associated Slurm cluster will cause Open OnDemand to stop working.

Microsoft Entra ID Application

It is necessary to register and configure Microsoft Entra ID application to enable authentication using OpenID Connect with Open OnDemand. Registration of an application during deployment using the marketplace UI is not possible due to technical limitations, so the user must take a manual step of either configuring an existing application or registering a new one.

 

 

The application uses federated credentials, a secure way that eliminates the need to use secrets when requesting authentication tokens. A user-assigned managed identity acts as the trust source and is assigned to the Open OnDemand virtual machine from which the authentication request originates.

Building the infrastructure

Deployment using the Marketplace UI

Search for Slurm the Azure Marketplace

Select the Azure CycleCloud Workspace for Slurm.

Click the Create button and follow the setup process documented here. The steps below are for enabling Open OnDemand.

In the File-system section, select Azure NetApp Files for /shared and /home as explained in the pre-requisites

 

 

In the Networking section, ensure that your network is accessible through its private IP by either implementing a hub-and-spoke model and peering the new network to an existing hub containing the VPN or using an existing VNET with these capabilities.

In the Slurm Settings section, ensure that there is at least one initial login node set to be used.

 

 

In the Open OnDemand settings, specify the user domain for user mapping. For Microsoft Entra ID users with email addresses following the pattern foo@contoso.com, enter contoso.com as the domain. Leave the FQDN field empty to generate a self-signed certificate using the private IP address of the Open OnDemand VM.

Finally, choose between using an existing Microsoft Entra ID application or registering a new one.

 

 

Specify the user-assigned managed identity associated with the federated identity credentials and Application (client) ID when using an existing Entra ID application.

 

 

Finally, add resource tags if needed then create.

After completing the deployment, access the CycleCloud VM and verify the cloud-init output to ensure successful completion.

tail -f /var/log/cloud-init-output.log

 

 

Navigate to the CycleCloud user interface.

If using an existing Microsoft Entra ID application

Verify that both the ccw and OpenOnDemand clusters are initiating. Although there may be some temporary errors, both clusters should be ready within a few minutes.

Retrieve the IP address of the Open OnDemand VM from the CycleCloud UI to complete the configuration of the registered application. Then add a new redirect URI, 'https://<ip>/oidc', in the Authentication settings of the application as illustrated below.

 

 

If manually registering a Microsoft Entra ID application

After the deployment is finished, it is necessary to execute the following commands from a Linux shell with Azure CLI installed to register and configure the Microsoft Entra ID application for authentication. The account logged into the CLI must have the appropriate permissions to register an application and its active subscription must be the one used for the deployment if it is not already set.

resource_group=<resource_group_name> az deployment group create -g $resource_group --template-uri https://raw.githubusercontent.com/Azure/cyclecloud-slurm-workspace/refs/heads/main/bicep/ood/oodEntraApp.json --parameters "$(az deployment group show -g $resource_group -n pid-d5d2708b-a4ef-42c0-a89b-b8bd6dd6d29b-partnercenter --query properties.outputs | jq '.oodManualRegistration.value | with_entries(.value |= {value: .})')"

 

Once executed, check that the application is well registered and copy its client ID. Ensure the redirect URI in Authentication is correct, federated credentials are set, upn is an optional claim in Token configuration, and API permissions are present.

Lastly, finalize the settings for Open OnDemand. Browse to the CycleCloud web portal, select the OpenOnDemand cluster, and click on the Edit button. This will open the cluster template definition.

• Leave FQDN empty,
• Set the Client ID to that of the registered application ID created in previous steps,
• Set the user domain to the enterprise domain,
• Tenant ID should be set to that of the tenant in which the application registration exists (blank here for confidentiality),

The managed identity should be manually selected to the one named <your_resource_group>/ccwOpenOnDemandManagedIdentity. Please note: this value will initially fail to appear due to a bug, so this will need to be set again when editing the template.

 

 

Press Save and then Start cluster and wait for the Open OnDemand virtual machine to be ready. There could be a transient error which will be retried after few minutes.

Deployment using the az bicep CLI

Deploy a full environment

 

1. Clone the preview branch of the Azure CycleCloud Workspace for Slurm repository:

git clone --depth 1 https://github.com/azure/cyclecloud-slurm-workspace.git

 

2. Enter to the cyclecloud-slurm-workspace directory and copy the content of the UI definition file found at: uidefinitions/createUiDefinition.json
3. Navigate to the UI Definition Sandbox:

• For Azure Public Cloud, visit the Azure Public version
• For Azure US Government Cloud, visit the Azure US Gov version

4. Paste the content of the UI definition file into the multiline text box on the right-hand side of the portal, ensuring to entirely replace any content already present
5. Click “Preview” in the bottom-left corner to bring up a UI experience
6. Proceed through each page of the UI flow to ensure that the required values populate, but:

• Do not click the “Create” button on the bottom of the page as this will reset the UI Definition Sandbox environment (this is a known bug)

7. In the Open OnDemand settings, a new option will be available to automatically register and configure the Microsoft Entra ID application used for authentication

• Choose this option only if the user account used for deployment can register a Microsoft Entra ID applications
• See the instructions available on subsection after this list to learn how to register a Microsoft Entra ID application prior to deployment

8. Once on the page labeled “Review + create,” click on the link labeled “View outputs payload” adjacent to the “Create” button on the bottom of the page

This will generate a pane with JSON-formatted text in its body on the right-hand side of the browser window

9. Copy the JSON-formatted text into a local JSON file at the top level of the local cyclecloud-slurm-workspace directory and save it as parameters.json

This is what is hereafter referred to as the “Parameters File”

10. Open the shell of choice and navigate to the folder/directory that contains the cyclecloud-slurm-workspace repository,
11. Accept the terms of the CycleCloud VM image plan:

az vm image terms accept --urn azurecyclecloud:azure-cyclecloud:cyclecloud8-gen2:latest

 

12. Run the following deployment command in shell after making substitutions for fields with square brackets (and be sure to delete brackets):

az deployment sub create --template-file ./bicep/mainTemplate.bicep --parameters parameters.json --location [ANY AZURE LOCATION E.G. eastus]

 

13. Wait until the shell indicates that the deployment was successful

Alternatively, track deployment process in the Azure Portal by navigating to the resource group indicated int he UI, selecting “Deployments” form the Settings dropdown menu on the left-hand side menu, and checking the status of the deployment name that begins with “pid-” at the bottom of the displayed list

How do I register a Microsoft Entra ID application before deployment?

It is possible to register a Microsoft Entra ID application before the deployment of a full environment and configure it afterwards

Create an  app.json parameter file containing:

• appName : the name of the application to be registered,
• fqdn: the IP address or the Fully Qualified Domain Name (FQDN) of the Open OnDemand virtual machine (may be temporarily set as it can be modified later),
• umiName: the name of the user-assigned managed identity created for the federated identity credentials assigned to the Open OnDemand virtual machine.

{ "appName": { "value": "" }, "fqdn": { "value": "" }, "umiName": { "value": "" } }

Execute the below commands to create a resource group and user-assigned managed identity and to register the Entra ID application.

resource_group=<the_resource_group_you_deployed_in> location=<location> az group create -l $location -n $resource_group az identity create --name $(jq -r '.umiName.value' app.json) --resource-group $resource_group --location $location az deployment group create -g $resource_group --template-uri https://raw.githubusercontent.com/Azure/cyclecloud-slurm-workspace/refs/heads/main/bicep/ood/oodEntraApp.json --parameters @app.json

 

Add Users

As explained in previous sections, it is required to add local Linux users that map to the Microsoft Entra ID of users authenticated on the Open OnDemand portal. In the CycleCloud web UI, select the Users menu from the top right gear dropdown.

 

 

Then use the Create menu

 

 

To add a new user, use the email address as the username without including the domain. Assign the user, minimum, the Global Node User role for regular access. Assign the Global Node Admin role if administrative privileges are required.

 

 

Save, add others if necessary, and browse back to Clusters.

Make note of a green check mark next to Users to confirm synchronization.

 

 

Use Open OnDemand

Browse to the private IP of the Open OnDemand Virtual Machine

The browser may display a warning because the website uses a self-signed certificate. There may be a consent message similar to the below displayed after completing Microsoft Entra ID authentication.

 

 

The Open OnDemand dashboard will then appear.

 

 

To enjoy using Open OnDemand, you can try these actions:

• Start a shell session from the Slurm ccw Shell Access icon,
• Request a VSCode on Login Node and connect to it almost instantly,
• Request a VSCode on Compute node, see in CycleCloud how nodes are dynamically allocated, and connect to it once it is ready,
• Use the Files menu to browse, copy, and edit your files,
• Use the Jobs/Job Composer to submit jobs

Read the Open OnDemand documentation to understand how to enable application development.

Use the Open OnDemand discourse to communicate with the community.

Provide feedback regarding Open OnDemand integration using GitHub issues here.

Known Limitations

• Only Slurm clusters are automatically registered in Open OnDemand
• Clusters register automatically every 5 minutes. Therefore, in the event of an error labeled "No Cluster defined" when starting a shell session, please wait 5 minutes before trying again. Alternatively, restart the web server from the Help menu to clean up the cache.