Research IT

Nova

Overview

The Nova cluster is the primary high performance computing (HPC) solution for research and education at ISU.   It is available to users from every academic discipline at the University.  

Contents

Equipment

The cluster currently consists of:

  • 330+ compute nodes with over 38,000 CPU cores
  • 200+ GPUs
  • 200 TB of memory
  • 2PB of high speed data storage
  • High-speed networking 

New components are added on a regular basis to ensure access to the latest technology.  For in-depth hardware details, see the Nova Hardware page.

Getting Access to Nova

There are three access tiers on the cluster – sponsored, community, and instructional. 

  • The Sponsored tier consists of research groups that have purchased hardware for the cluster.   The purchased hardware is added to the pool of resources available to all researchers.  Buying hardware is optional, but users with sponsored access will have higher job priority and more generous resource limits than community users.  Research groups can purchase compute nodes or storage using the  Nova Cluster Node and Storage Purchase Form.  
     
  • The Community tier users are researchers that have not purchased hardware for the cluster.   While community users have lower priority access to compute nodes and less data storage than sponsored users, they are able to use any of the compute nodes, which gives them sufficient resources for testing and proof-of-concept work.
    • The groups in the community tier receive 1TB of group working space
    • A free group on Nova can be created for student clubs and organizations in the Community Tier of Nova, but it will need to be sponsored by a faculty member, usually the advisor for the group.
    • Community users can request access to the cluster using the HPC Cluster Access Request form.
       
  • The Instructional tier users is students and instructors associated with a class that is using the cluster for teaching.  Instructional users get access to the instruction queue hardware that is reserved for instructional use.
    Instructors can request access for their class using the HPC Cluster Access Request form.

HPC Class

HPC Class is a partition on the Nova cluster set aside for classroom and instructional users.   See this page for more information.

Logging In

Once your access request has been approved, you can log in to the cluster.  Note that you must be first connected to the university network either by being on-campus, or logged in remotely over Eduroam or the ISU VPN.  ]  

There are three commonly used methods for logging in:

  • Nova OnDemand

    The easiest way to access Nova is by using the Nova OnDemand web interface.   Point your browser to:

    https://nova-ondemand.its.iastate.edu 

    From the main Nova OnDemand menu, you can select Clusters->_Nova Shell to get a simple shell window on a head node.

    To get a full graphical desktop, select  Interactive Apps -> Nova Desktop from the main menu.   You will need to fill out the form for the amount of resources you need such as number of processors, amount of memory, length of time, or GPU specification.   After you click Launch, a full graphical desktop session will be launched for you on a Nova compute node.

  • SSH

    Many users prefer to use an SSH client to connect to the cluster.  Instead of a web browser, you connect using an SSH client such as OpenSSH.  Once logged in, you use the command line to interact with the cluster.  Note that there are two methods for authentication available when using SSH:  Multi-factor authentication, or SSH Public Keys. The current ssh host keys for nova login and dtn nodes can be found here.

    Logging in with Multi-factor Authentification (MFA)

    Multi-factor authentication (MFA) is the default method for authenticating over SSH.   With multi-factor authentication, you will be asked to provide a time-dependent PIN in addition to your regular ISU password.   The key for generating the PIN is created for your account automatically the first time you login to Nova OnDemand.   Then you will need to add the key to a MFA application such as Microsoft Authenticator or Google Authenticator.   You can obtain the QR code for adding your key to MFA application by logging into Nova OnDemand and selecting the MFA Recovery option under Clusters menu option or by following this link.  Use the QR code when adding your Nova MFA key to your MFA app.

    SSH with Public Keys

    SSH logins can also use Public Key Authentication.  Public key authentication is useful when you want to access the cluster securely but MFA authentication is not possible or inconvenient.  To use this you must be first connected to the ISU VPN or Eduroam, regardless of whether you are on or off campus.

Slurm Job Scheduler

To use the cluster for running a job, a user submits job requests to the Slurm workload manager which places the job into the job queue where it will be pending until the necessary hardware becomes available.  Whether the job is run as a batch job or interactively, Slurm controls what hardware is assigned to it. 

Batch jobs are submitted to Slurm using the command sbatch, which reads a batch script containing the commands a user wants to run.  The job's Slurm parameters can be specified in the script using lines that begin with #SBATCH followed by the desired parameters.    Users can use the Slurm Script Generator to create batch job scripts.

Users can also request interactive command line sessions if they want to run ad hoc commands in a Linux shell on a compute node.   This is very useful for testing and development of an application.  To launch a command line session on a compute node, use the  salloc command on the login node along with the desired job parameters.  Never run programs on the login nodes.

Another way to get an interactive sessions is with Nova OnDemand which can provide access to a simple command line session, or a full graphical Linux desktop, all using your web browser.

Software

The cluster runs on Linux.  There are thousands of research software packages available, including both open source and proprietary commercial applications.  Research IT staff are available to help build and integrate packages as needed.   Users can access any of the packages in the software repository using the environment module commands.

File Storage

There are several options for storing files on Nova.  See the HPC File Storage Guide for more details.

VS-Code on the Cluster

Microsoft VS-Code is a very popular Interactive Development Environment (IDE) used by software developers.  This guide contains information about how to set this up.