How to create an empty virtual environment¶

If you need to install a really old version of some package, or make modifications to a package, you can create a completely empty virtual environment in the container. This virtual environment will be isolated from the pre-installed packages and you will need to install everything yourself.

First, get an interactive session on a GPU node.

srun --time=01:00:00 --nodes=1 --cpus-per-task=4 --partition=gpu --gres=gpu:1 --pty /usr/bin/bash

Load the version of the ml-gpu module you want to use. For example:

module load ml-gpu/20230427

Next, create a directory to install the packages to. This should be within your group's /work directory, and specific to the version of the ml-gpu container that you're using.

ml-gpu python -m venv /work/LAS/your-lab/emptymlgpuvenv-20230427

The difference between this command and the one given in previous sections is the removal of the --system-site-packages flag, which isolates the environment.

Now install any packages you need with pip.

ml-gpu /work/LAS/your-lab/emptymlgpuvenv-20230427/bin/pip3 install somepackage

To confirm the packages are installed:

ml-gpu /work/LAS/your-lab/emptymlgpuvenv-20230427/bin/pip3 freeze | grep somepackage

Your package is now installed.

To use this virtual environment in your batch scripts, load the ml-gpu module, then invoke python like this:

ml-gpu /work/LAS/your-lab/emptymlgpuvenv-20230427/bin/python your_script.py

Be sure to replace the path with the actual location you installed the packages.