Site Tools


Action disabled: media
linux_environment_modules

Software Modules

Software Modules make it easy for you to use software installed on our Linux systems. We provide a wide range of software across all our servers so that you don't have to install software on your own. We also offer different versions of software, letting users choose which they want to use.


Virtual Environments

For most Python related packages (i.e. pip install …), it is often more applicable to utilize a virtual environment. This can be done with Anaconda or venv. The former requires loading an anaconda module, and the latter requires the python3 module.


Jupyter Notebook

A Jupyter Notebook interface can be loaded using a virtual environment (as described above). This is best done using a NX virtual desktop. Once you've logged into your NX virtual desktop, load python3 or anaconda3 as a software module:

[abc1de@labsrv03 ~]$ module load anaconda3

Then, load Jupyter Notebook:

[abc1de@labsrv03 ~]$ Jupyter Notebook

Follow the link given to you by the Notebook Server to access the interface:

[W 12:52:23.489 NotebookApp] Loading JupyterLab as a classic notebook (v6) extension.
[I 12:52:23.492 NotebookApp] Serving notebooks from local directory: /u/abc1de
[I 12:52:23.492 NotebookApp] Jupyter Notebook 6.5.4 is running at:
[I 12:52:23.492 NotebookApp] http://localhost:8888/?token=eb9345040d71bbeb1eddc75fb88504f98e25beb207907f6d
[I 12:52:23.492 NotebookApp]  or http://127.0.0.1:8888/?token=eb9345040d71bbeb1eddc75fb88504f98e25beb207907f6d
[I 12:52:23.492 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 12:52:23.634 NotebookApp] 
    
    To access the notebook, open this file in a browser:
        file:///u/<youruserid>/.local/share/jupyter/runtime/nbserver-20880-open.html
    for example...
        file:///u/abc1de/.local/share/jupyter/runtime/nbserver-20880-open.html

How Modules Work

Environment modules modify Environment Variables of the Linux Shell. Environment variables typically are paths to executables, version specifications, or terminal colors. For example, when an SSH session is initialized, variables are set such as the PATH variable for the location of various executable commands such as vim, nano, and ssh.

Using Software Modules

Type module avail to get a list of available software (this list is just a small example, many more modules are available).

[abc1de@portal04 ~]$ module avail
---- /sw/ubuntu/Modules/modulefiles ---
anaconda3       cuda-toolkit-11.7.0     cudnn-8.4.1     git-2.38.1 
java17-17.0.1   maven-3.8.3             nano-6.0        openmpi-4.1.4 
python3         ruby-3.0.3              vscode-1.61.2

Load the software module using module load <module name>. Now the software is ready to use.

abc1de@portal04 ~ $ module load python3              # Load the module for use in your current session
abc1de@portal04 ~ $ which python3
/sw/ubuntu-22.04/python/current/bin/python3
abc1de@portal04 ~ $ python3                          # When you run `python` you get the version that has been loaded
Python 3.10.12 (main, Jun 14 2023, 10:18:49) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

module list will show what modules you have loaded.

abc1de@portal ~ $ module load matlab
abc1de@portal ~ $ module load php
abc1de@portal ~ $ module list
Currently Loaded Modulefiles:
  1) matlab    2) python3   3) php

module display <module name> will show specific information about the environment variables setup by a module.

abc1de@portal ~ $ module display python
-------------------------------------------------------------------
/sw/ubuntu/Modules/modulefiles/python:
 
module-whatis	 Python current 
prepend-path	 PATH /sw/ubuntu/python/current/bin 
prepend-path	 LD_LIBRARY_PATH /sw/ubuntu/python/current/lib 
prepend-path	 LD_INCLUDE_PATH /sw/ubuntu/python/current/include 
prepend-path	 MANPATH /sw/ubuntu/python/current/share/man 
-------------------------------------------------------------------
abc1de@portal ~ $ echo $LD_LIBRARY_PATH 
/sw/ubuntu/python/3.10.12/lib

Use module switch <current module name> <new module name> to change one version of software for another and module unload <module name> to unload modules when you are done or no longer want this software.

abc1de@portal ~ $ module switch python3 python2
abc1de@portal ~ $ module unload php

Software Module Requests

If a module is needed but not listed, please send an email to cshelpdesk@virginia.edu with the following:
1. In the subject line, write “Software Module Request: software name and the version
2. In the body of the email, please include information about the software being requested (version and any associated links)

Shell Scripting/Slurm Batch

If you write shell scripts or slurm batch files that make use of these software packages, remember that you must still load modules from inside of your script. For example, your script must contain the line module load python3 before you can use python from inside of your script.

Using modules in a non-interactive script

When executing a shell script non-interactively, initialization files are not loaded from /etc/profile.d. This prevents the initialization of the Environment Modules system and will prevent you from loading software modules.

To fix this, simply include the following line in your scripts:

source /etc/profile.d/modules.sh
linux_environment_modules.txt · Last modified: 2023/09/18 15:50 by 127.0.0.1