Environment Modules

Environment Modules are designed to manage custom package installations. Software in our department is all installed to a network filesystem mounted over NFS where each package available is installed to it's own directory. This allows us to manage a wide range of software across different Linux distributions without users having to deal with any configuration. We can include different versions of software, letting users choose which they want to use.

Please see Available Modules for a listing of available software.

On any Linux system, you can run module avail to get a list of available software.

[pgh5a@portal04 ~]$ module avail
------------------------------- /sw/centos/Modules/systemModfiles -------------------------------------------------------------
slurm-client         slurm-client-testing

--------------------------------- /sw/centos/Modules/modulefiles --------------------------------------------------------------
afl               cmake-3.10        cudnn-7.3.1       gdb-8.1           libav             netdata           php7              ruby
altera_pro        cmake-3.15        cudnn-7.5.0       ghex              libtorch          nios2eds          php7.1.10         ruby2.5.1
anaconda3         cs6620            dmd               git               libtorch-1.3.1    nvtop             python            sbt
anaconda3-2019.10 cuda-toolkit      dmd-2.088.0       gradle            lua               openmpi           python2           scala
antlr             cuda-toolkit-10   doxygen           hashcat           lua-5.3.4         openmpi2.1.5      python2.7.15      sqlite3
atom              cuda-toolkit-10.1 eclipse           java              lxd               openmpi4.0.0      python3           storm
boost             cuda-toolkit-8.0  emacs             java11            modelsim_ae       openmpi4.0.2      python3.6.2       svn
boost-1.67.0      cuda-toolkit-9.0  fio               java8             modelsim_ase      parallel          python3.8.0       torch
clang-llvm        cuda-toolkit-9.2  gcc               java8u161         nano              perl              qsys              verilator
clang-llvm-6.0.0  cudnn             gcc-5.5.0         java9             nccl              perl-5.26         quartus
clang-llvm-7.1.0  cudnn-7.0.5       gcc-6.3.0         java9.0.4         nccl-10.0         perl-6            R
cmake             cudnn-7.1.1       gcc-7.1.0         lammps            nccl-9.2          php               rstudio

------------------------------- /sw/linux-any/Modules/modulefiles -------------------------------------------------------------
ant               cplex             cplex-opl         dmd-2.088.0       ripgrep           scons-3.0.1       vscode
antlr             cplex-concert     cplex-studio-all  go                sbt               tmux
apktool           cplex-cpoptimizer dmd               matlab            scons             tmux-2.7

Load the software module usingmodule load. Now the software is ready to use.

pgh5a@portal04 ~ $ module load python3              # Load the module for use in your current session
pgh5a@portal04 ~ $ which python
/sw/centos/python/3.6.2/bin/python
pgh5a@portal04 ~ $ python                           # When you run `python` you get the version that has been loaded
Python 3.6.2 (default, Oct  6 2017, 13:03:05) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

module list will show what modules you have loaded.

pgh5a@portal ~ $ module load matlab
pgh5a@portal ~ $ module load php
pgh5a@portal ~ $ module list
Currently Loaded Modulefiles:
  1) matlab    2) python3   3) php

module display will show specific information about the environment variables setup by a module.

pgh5a@portal ~ $ module display python
-------------------------------------------------------------------
/sw/centos/Modules/modulefiles/python:
 
module-whatis	 Python current 
prepend-path	 PATH /sw/centos/python/current/bin 
prepend-path	 LD_LIBRARY_PATH /sw/centos/python/current/lib 
prepend-path	 LD_INCLUDE_PATH /sw/centos/python/current/include 
prepend-path	 MANPATH /sw/centos/python/current/share/man 
-------------------------------------------------------------------
pgh5a@portal ~ $ echo $LD_LIBRARY_PATH 
/sw/centos/python/3.6.2/lib

Use module switch to change one version of software for another and module unload to unload modules when you are done or no longer want this software.

pgh5a@portal ~ $ module switch python3 python2
pgh5a@portal ~ $ module unload php

If you write shell scripts or slurm batch files that make use of these software packages, remember that you must still load modules from inside of your script. For example, your script must contain the line module load python3 before you can use python from inside of your script.

To make modules work from inside SLURM or scripts, you must including the following line in your script before loading any modules:

source /etc/profile.d/modules.sh