Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
linux_environment_modules [2018/10/19 16:55]
ktm5j
linux_environment_modules [2021/12/02 14:08] (current)
Line 1: Line 1:
-====== Environment ​Modules ​======+==== Software ​Modules ====
  
-[[http://​modules.sourceforge.net|Environment ​Modules]] ​are designed ​to manage custom package installations. ​ Software in our department is all installed to a network filesystem mounted over NFS where each package available is installed to it's own directory This allows us to manage ​a wide range of software across ​different Linux distributions without users having ​to deal with any configuration.  We can include ​different versions of software, letting users choose which they want to use.+[[http://​modules.sourceforge.net|Software ​Modules]] ​make it easy for you to use software installed on our Linux systemsWe provide ​a wide range of software across ​all our servers so that you don't have to install software on your own.  We also offer different versions of software, letting users choose which they want to use. 
  
-===== Available Modules =====+\\
  
-Please see [[linux_modules:​start|Available ​Modules]] for a listing of available software.+=== Using Modules ​===
  
-===== Using Modules ===== +Type ''​%%module avail%%''​ to get a list of available software.
- +
-On any Linux system in our department, you can run ''​%%module avail%%'' ​from your shell to get a list of available software.+
  
 <​code>​ <​code>​
-ktm5j@applecake ​~ $ module avail+[abc1de@portal04 ​~]$ module avail 
 +------------------------------- /​sw/​centos/​Modules/​systemModfiles ------------------------------------------------------------- 
 +slurm-client ​        ​slurm-client-testing 
 + 
 +--------------------------------- /​sw/​centos/​Modules/​modulefiles -------------------------------------------------------------- 
 +afl               ​cmake-3.10 ​       cudnn-7.3.1 ​      ​gdb-8.1 ​          ​libav ​            ​netdata ​          ​php7 ​             ruby 
 +altera_pro ​       cmake-3.15 ​       cudnn-7.5.0 ​      ​ghex ​             libtorch ​         nios2eds ​         php7.1.10 ​        ​ruby2.5.1 
 +anaconda3 ​        ​cs6620 ​           dmd               ​git ​              ​libtorch-1.3.1 ​   nvtop             ​python ​           sbt 
 +anaconda3-2019.10 cuda-toolkit ​     dmd-2.088.0 ​      ​gradle ​           lua               ​openmpi ​          ​python2 ​          ​scala 
 +antlr             ​cuda-toolkit-10 ​  ​doxygen ​          ​hashcat ​          ​lua-5.3.4 ​        ​openmpi2.1.5 ​     python2.7.15 ​     sqlite3 
 +atom              cuda-toolkit-10.1 eclipse ​          ​java ​             lxd               ​openmpi4.0.0 ​     python3 ​          ​storm 
 +boost             ​cuda-toolkit-8.0 ​ emacs             ​java11 ​           modelsim_ae ​      ​openmpi4.0.2 ​     python3.6.2 ​      svn 
 +boost-1.67.0 ​     cuda-toolkit-9.0 ​ fio               ​java8 ​            ​modelsim_ase ​     parallel ​         python3.8.0 ​      ​torch 
 +clang-llvm ​       cuda-toolkit-9.2 ​ gcc               ​java8u161 ​        ​nano ​             perl              qsys              verilator 
 +clang-llvm-6.0.0 ​ cudnn             ​gcc-5.5.0 ​        ​java9 ​            ​nccl ​             perl-5.26 ​        ​quartus 
 +clang-llvm-7.1.0 ​ cudnn-7.0.5 ​      ​gcc-6.3.0 ​        ​java9.0.4 ​        ​nccl-10.0 ​        ​perl-6 ​           R 
 +cmake             ​cudnn-7.1.1 ​      ​gcc-7.1.0 ​        ​lammps ​           nccl-9.2 ​         php               ​rstudio 
 + 
 +------------------------------- /​sw/​linux-any/​Modules/​modulefiles ------------------------------------------------------------- 
 +ant               ​cplex ​            ​cplex-opl ​        ​dmd-2.088.0 ​      ​ripgrep ​          ​scons-3.0.1 ​      ​vscode 
 +antlr             ​cplex-concert ​    ​cplex-studio-all ​ go                sbt               ​tmux 
 +apktool ​          ​cplex-cpoptimizer dmd               ​matlab ​           scons             ​tmux-2.7
  
------------------------- /​sw/​centos/​Modules/​modulefiles ------------------------ 
-altera_pro ​      ​modelsim_ae ​     php7             ​python3.6.2 
-cuda-toolkit-9.0 modelsim_ase ​    ​php7.1.10 ​       qsys 
-emacs            nios2eds ​        ​python ​          ​quartus 
-matlab ​          ​php ​             python3 
 </​code>​ </​code>​
  
-After deciding what software you want to use, load the package ​using''​%%module load%%''​. ​ Now the software is ready to use.+Load the software module ​using''​%%module load%%''​. ​ Now the software is ready to use.
  
 <​code>​ <​code>​
-ktm5j@applecake ​~ $ module load python3 +abc1de@portal04 ​~ $ module load python3 ​             # Load the module for use in your current session 
-ktm5j@applecake ​~ $ which python+abc1de@portal04 ​~ $ which python
 /​sw/​centos/​python/​3.6.2/​bin/​python /​sw/​centos/​python/​3.6.2/​bin/​python
 +abc1de@portal04 ~ $ python ​                          # When you run `python` you get the version that has been loaded
 +Python 3.6.2 (default, Oct  6 2017, 13:​03:​05) ​
 +[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
 +Type "​help",​ "​copyright",​ "​credits"​ or "​license"​ for more information.
 +>>>​
 </​code>​ </​code>​
  
-After loading several modules in your session, ​''​%%module list%%''​ will show what modules have been loaded.+''​%%module list%%''​ will show what modules ​you have loaded.
  
 <​code>​ <​code>​
-ktm5j@applecake ​~ $ module load matlab +abc1de@portal ​~ $ module load matlab 
-ktm5j@applecake ​~ $ module load php +abc1de@portal ​~ $ module load php 
-ktm5j@applecake ​~ $ module list+abc1de@portal ​~ $ module list
 Currently Loaded Modulefiles:​ Currently Loaded Modulefiles:​
   1) matlab ​   2) python3 ​  3) php   1) matlab ​   2) python3 ​  3) php
 </​code>​ </​code>​
  
-''​%%module display%%''​ will show specific information about the environment variables ​that will be modified ​by a module.+''​%%module display%%''​ will show specific information about the environment variables ​setup by a module.
  
 <​code>​ <​code>​
-ktm5j@applecake ​~ $ module display python+abc1de@portal ​~ $ module display python
 ------------------------------------------------------------------- -------------------------------------------------------------------
 /​sw/​centos/​Modules/​modulefiles/​python:​ /​sw/​centos/​Modules/​modulefiles/​python:​
Line 52: Line 71:
 prepend-path MANPATH /​sw/​centos/​python/​current/​share/​man ​ prepend-path MANPATH /​sw/​centos/​python/​current/​share/​man ​
 ------------------------------------------------------------------- -------------------------------------------------------------------
-ktm5j@applecake ​~ $ echo $LD_LIBRARY_PATH ​+abc1de@portal ​~ $ echo $LD_LIBRARY_PATH ​
 /​sw/​centos/​python/​3.6.2/​lib /​sw/​centos/​python/​3.6.2/​lib
 </​code>​ </​code>​
  
-You can ''​%%switch%%''​, changing ​one version of software for another, or ''​%%unload%%''​ modules when you are done or no longer want this software.+Use ''​%%module ​switch%%'' ​to change ​one version of software for another ​and ''​%%module ​unload%%'' ​to unload ​modules when you are done or no longer want this software.
  
 <​code>​ <​code>​
-ktm5j@applecake ​~ $ module switch python3 python2 +abc1de@portal ​~ $ module switch python3 python2 
-ktm5j@applecake ​~ $ module unload php +abc1de@portal ​~ $ module unload php 
-</​code>​+</​code>​\\ 
 + 
 +=== Shell Scripting/​Slurm Batch === 
 + 
 +If you write shell scripts or slurm batch files that make use of these software packages, remember that you must still load modules from inside of your script. ​ For example, your script must contain the line ''​%%module load python3%%''​ before you can use python from inside of your script.  
 + 
 +\\ 
 + 
 +=== Initialize Modules System === 
 + 
 +To make modules work from inside SLURM or scripts, you must including the following line in your script **before** loading any modules:
  
-===== Things to keep in mind =====+<code bash> 
 +source /​etc/​profile.d/​modules.sh 
 +</​code>​\\
  
-==== Shell Scripting/​Slurm Batch ====+=== Virtual Environments ​=== 
 +For most Python related packages (i.e. pip install ...), it is often more applicable to utilize a virtual environment. This can be done with [[ https://​docs.conda.io/​projects/​conda/​en/​latest/​user-guide/​tasks/​manage-environments.html | Anaconda]] 
 +or [[ https://​docs.python.org/​3/​library/​venv.html | venv ]]. The former requires loading an anaconda module, and the latter requires the python3 module.
  
-If you write shell scripts or slurm batch files that make use of these software packages, remember that you must still load modules from inside of your script. ​ For example, your script must contain the line ''​%%module load python3%%''​ before you can use python from inside of your script.+\\
  
 +=== Module Requests ===
 +If a module is needed but not listed, please send an email to cshelpdesk@virginia.edu with the following:​\\
 +1. In the subject line, write "​Software Module Request: // software name and the version //"\\
 +2. In the body of the email, please include information about the software being requested (version and any associated links)
  • linux_environment_modules.1539968132.txt.gz
  • Last modified: 2018/10/19 16:55
  • by ktm5j