In TODO, the CS department has made available Apptainer software (formally named Singularity) for containers in the environment. This is available as a software module and can be used throughout the environment (including on SLURM nodes).

Apptainer is available as a module. To load and use it simply execute the following

~$ module load apptainer

By default for Apptainer containers, mountpoints such as home, project, and bigtemp directories are available along with certain system directories. This means that changes to files you make when using an image will persist outside of the image as well.

System directories examples (local to the server):

• /tmp
• /proc

Mounted Directories (remote storage):

• /u
• /l
• /s
• /p
• /bigtemp

To disable such directories from becoming available within a container, add the flag -c Apptainer commands such as exec or run.

Apptainer image files are named with the suffix Singularity Image Format (.sif). To create your own, a definition file or def file must be written first. A full guide on def files can be found on the official Apptainer website.

Once a def file has been created, the following can be done (replace <imagename> with the name of the resulting SIF image, and <defname> with the name of your def file)

~$ apptainer build <imagename>.sif <defname>.def

Prebuilt images from an Open Container Initative (OCI) such as docker can be built and are automatically converted to an SIF image when using the build command

~$ apptainer build mylolcow.sif docker://ghcr.io/apptainer/lolcow

Note, when building images, be sure to check the file size. By default, Apptainer will create the SIF file in the current directory where the command was executed. For large images, it's recommended to use project directories for persistent storage or bigtemp for temporary image storage.

To open an interactive shell within a container

~$ apptainer shell <imagename>.sif
Apptainer>

Note, after opening a shell, commands such as ls are available. As mentioned previously, home directories are available by default within an image

Apptainer> ls -al

To run commands within a given container, for example when a container is made for running a python program

~$ apptainer exec <imagename>.sif python3 mypythonprogram.py 

When an image is created from a def file, a runscript can also be defined to turn the image into more of an executable program. Read more about runscripts here.

~$ apptainer run <imagename>.sif
or
~$ ./<imagename>.sif

When using a GPU image, the host system's GPU drivers can be loaded into an image by including the flag --nv when running an Apptainer command

For example,

~$ apptainer exec --nv <imagename>.sif python3 mygpuprogam.py 

Be sure to reference the CS wiki page about slurm for additional information here.

After initializing an interactive job in slurm

~$ module purge
~$ module load apptainer
~$ apptainer shell /bigtemp/example/myimage.sif
Apptainer>

Below is a simple example of a possible sbatch script for running an image on a GPU system

#!/bin/bash

#SBATCH --cpus-per-task=1
#SBATCH --gpus=1
#SBATCH -t 02:00:00
#SBATCH -p gpu

module purge
module load apptainer
apptainer exec --nv myimage.sif python3 mygpuprogam.py