This introduction will take you through a typical command-line session on the Nova cluster, showing the commands you would use to perform some simple tasks.
All of this assumes that you've logged in to the head node using SSH so that you are working in a command shell on the cluster.
Use salloc to create a Session on a Compute Node
In general, you don't want to work on the Nova head node. Instead, you should use the salloc command to open a session on a compute node. This will ensure that you are logged in to a computer with enough memory and processors to do what you need. The head node does not allow you to run intensive programs. The following command will request a session on one compute node (-N 1), with 8 processors (-n 8), for 4 hours (-t 4:00:00):
$ salloc -N 1 -n 8 -t 4:00:00
This will create a new session for you on one of the available compute nodes. You can run any Linux commands you need in this new session. Once the time limit of the session is reached, the session will be ended automatically. You can also exit the session at any time by entering the exit command.
Find the Path to Your Group's Work Folder
When you first log in, your working directory will be your home directory. You don't really want to use the home directory for storing a lot of data because you can only store a maximum 10GB of data there. So, we want to create a work folder to store your data. The first step is finding out where your work folder is. Run the command:
$ my_workdir
This will return the path that has been created for you and other members of your research group to store data. This path is usually a path under /work with your group's name,. To find the name of your primary research group, type the command:
$ my_primary_group
The primary research group is often the same as the username of a research group's Primary Investigator (PI). So, if your primary group is ajohnson, the work directory is /work/ajohnson.
Make a Folder Under Your Group's Work Folder For Yourself
You have permissions to create a folder for yourself under your group work directory. Let's say your username is rockyb and your primary group's name is ajohnson, you can use the mkdir command to create a work folder:
$ mkdir /work/ajohnson/rockyb
Now you can use this as your primary working directory:
$ cd /work/ajohnson/rockyb
Moving a Python Environment to Your Work Folder
A common problem users encounter is exceeding their home directory storage (10GB limit) when an application such as Conda or VS-Code puts all of the Python components it uses into folder in their home directory. A good way to solve this is to move the directory to a path under your work directory and then create a symbolic link so the old path still exists but all the files are actually in the new location.
Let's say you're using Conda. The default directory Conda uses to store its data is ~/.conda. Note: In the Bash shell, the tilde (~) is a shorthand for your home directory. What we'd like to do, then, is move this directory to a location in your work directory, but also create up a symbolic link that connects the new location to the old location. Let's assume you have a work directory called /work/ajohnson/rockyb.
1. List the contents of your ~/.conda directory. Then move it to /work/ajohson/rockyb/conda :
$ ls ~/.conda
environments.txt pkgs
The pkgs folder can be fairly large because that's where Conda installs any Python software you installed.
$ mv ~/.conda /work/ajohnson/rockyb/conda
(depending on how many files are there, this may take a couple of minutes).
2. Check to make sure that the ~/.conda folder is actually moved:
$ ls ~/.conda
ls: cannot access '/home/rockyb/.conda': No such file or directory
This tells us that the .conda folder in rockyb's home directory does not exist (As expected. It has been moved to /work/ajohnson/conda).
3. What does the /work/ajohnson/rockyb/conda directory contain?
$ ls /work/ajohnson/rockyb/conda
environments.txt pkgs
That looks right. We should see the same contents as showed in step 1 above.
4. Create a symbolic link that connects the new path to the old location:
$ ln -s /work/ajohnson/rockyb/conda ~/.conda
5. Now when we list the ~/.conda path we see:
$ ls ~/.conda
environments.txt pkgs
When we use the long listing option:
$ ls -l ~/.conda lrwxrwxrwx. 1 rockyb domain users 31 Mar 22 12:30 /home/rockyb/.conda -> /work/ajohnson/rockyb/conda
we can see that the ~/.conda folder is now linked to /work/ajohnson/rockyb/conda. Any files created in /home/rockyb/.conda will actually be created in /work/ajohnson/rockyb/conda.