This is an old revision of the document!
Environment Modules
Environment Modules are designed to manage custom package installations. Software in our department is all installed to a network filesystem mounted over NFS where each package available is installed to it's own directory. This allows us to manage a wide range of software across different Linux distributions without users having to deal with any configuration. We can include different versions of software, letting users choose which they want to use.
Available Modules
Please see Available Modules for a listing of available software.
Using Modules
On any Linux system in our department, you can run module avail
from your shell to get a list of available software.
[pgh5a@portal04 ~]$ module avail ------------------------------- /sw/centos/Modules/systemModfiles ------------------------------------------------------------- slurm-client slurm-client-testing --------------------------------- /sw/centos/Modules/modulefiles -------------------------------------------------------------- afl cmake-3.10 cudnn-7.3.1 gdb-8.1 libav netdata php7 ruby altera_pro cmake-3.15 cudnn-7.5.0 ghex libtorch nios2eds php7.1.10 ruby2.5.1 anaconda3 cs6620 dmd git libtorch-1.3.1 nvtop python sbt anaconda3-2019.10 cuda-toolkit dmd-2.088.0 gradle lua openmpi python2 scala antlr cuda-toolkit-10 doxygen hashcat lua-5.3.4 openmpi2.1.5 python2.7.15 sqlite3 atom cuda-toolkit-10.1 eclipse java lxd openmpi4.0.0 python3 storm boost cuda-toolkit-8.0 emacs java11 modelsim_ae openmpi4.0.2 python3.6.2 svn boost-1.67.0 cuda-toolkit-9.0 fio java8 modelsim_ase parallel python3.8.0 torch clang-llvm cuda-toolkit-9.2 gcc java8u161 nano perl qsys verilator clang-llvm-6.0.0 cudnn gcc-5.5.0 java9 nccl perl-5.26 quartus clang-llvm-7.1.0 cudnn-7.0.5 gcc-6.3.0 java9.0.4 nccl-10.0 perl-6 R cmake cudnn-7.1.1 gcc-7.1.0 lammps nccl-9.2 php rstudio ------------------------------- /sw/linux-any/Modules/modulefiles ------------------------------------------------------------- ant cplex cplex-opl dmd-2.088.0 ripgrep scons-3.0.1 vscode antlr cplex-concert cplex-studio-all go sbt tmux apktool cplex-cpoptimizer dmd matlab scons tmux-2.7
After deciding what software you want to use, load the package usingmodule load
. Now the software is ready to use.
pgh5a@portal04 ~ $ module load python3 # Load the module for use in your current session pgh5a@portal04 ~ $ which python /sw/centos/python/3.6.2/bin/python pgh5a@portal04 ~ $ python # When you run `python` you get the version that has been loaded Python 3.6.2 (default, Oct 6 2017, 13:03:05) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux Type "help", "copyright", "credits" or "license" for more information. >>>
After loading several modules in your session, module list
will show what modules have been loaded.
pgh5a@portal ~ $ module load matlab pgh5a@portal ~ $ module load php pgh5a@portal ~ $ module list Currently Loaded Modulefiles: 1) matlab 2) python3 3) php
module display
will show specific information about the environment variables that will be modified by a module.
pgh5a@portal ~ $ module display python ------------------------------------------------------------------- /sw/centos/Modules/modulefiles/python: module-whatis Python current prepend-path PATH /sw/centos/python/current/bin prepend-path LD_LIBRARY_PATH /sw/centos/python/current/lib prepend-path LD_INCLUDE_PATH /sw/centos/python/current/include prepend-path MANPATH /sw/centos/python/current/share/man ------------------------------------------------------------------- pgh5a@portal ~ $ echo $LD_LIBRARY_PATH /sw/centos/python/3.6.2/lib
You can switch
, changing one version of software for another, or unload
modules when you are done or no longer want this software.
pgh5a@portal ~ $ module switch python3 python2 pgh5a@portal ~ $ module unload php
Things to keep in mind
Shell Scripting/Slurm Batch
If you write shell scripts or slurm batch files that make use of these software packages, remember that you must still load modules from inside of your script. For example, your script must contain the line module load python3
before you can use python from inside of your script.
Initialize Modules System
To make modules work from inside SLURM or scripts, you must including the following line in your script before loading any modules:
source /etc/profile.d/modules.sh