Welcome! This workshop is designed for users that are new or reviewing basic Linux principles. This workshop also covers details on connecting to CS servers.
Note, the portal SSH login cluster servers are not GPU accelerated systems.
There are no network or VPN requirements for connecting to the SSH portal login cluster. These servers can be accessed on or off grounds, and do not require a VPN connection to access.
To SSH to the portal login cluster, from your personal computer, open a terminal and enter the following command, replacing <userid> with your computing ID.
Temporary passwords are sent via email when your CS account is created initially. After logging in for the first time, you will be required to change your password.
On Linux / MacOS, one application for connecting to portal via SSH is named Terminal
~$ ssh <userid>@portal.cs.virginia.edu
On Windows, two applications for connecting to portal via SSH are Command Prompt or PowerShell
C:\Users\username> ssh <userid>@portal.cs.virginia.edu
Note, for the first time you connect via SSH, you will see the following output
Are you sure you want to continue connecting (yes/no)?
Enter yes to continue
For more information, see our wiki page about our (Logging in via SSH)
Note, the NoMachine login cluster servers are not GPU accelerated systems.
There are no network or VPN requirements for connecting to the NoMachine desktop GUI cluster. These servers can be accessed on or off grounds, and do not require a VPN connection to access.
After downloading the NoMachine client application from the (NoMachine Website), open the NoMachine application and follow the steps listed below
Name: UVA CS Remote Linux DesktopHost: nxfront.cs.virginia.eduPort: 4000Protocol: NX<userid> with your computing IDUsername: <userid>Password: <cs password>Thereafter, you'll be prompted for a few configuration options that are user specific.
For more information, see our wiki page about our (NoMachine cluster)
Note, the RDP Windows server is not a GPU enabled system.
Network Requirements:
For windows systems, the application Remote Desktop Connection or RDP is installed by default. To connect to the remote desktop server
Computer: winsrv01User name: CSDOM\<userid>For more information, see our wiki page about our general purpose (Windows Server)
Remote development is possible for most common software. Generally, you simply need to enter your CS credentials to log in. Some software is available through the UVA software gateway, and it is recommended to obtain software from the UVA software gateway if possible.
For further reading about SSH connections, see our (SSH wiki page)
For further reading about remote development, see our (Remote Development wiki page)
Below are some example commands that can be run within a terminal over an SSH connection or using the application “Terminal” while using a NoMachine desktop GUI.
Note, for all applicable examples, replace <path> with the location of a file or directory
To print the current working directory (i.e. your current location)
~$ pwd
To change/move to another directory
~$ cd <path>
Note, the tilde character ~ specifies to use your home directory location. Further, simply using the command ~$ cd will move you to your home directory. The following are all equivalent to move to your home directory
~$ cd OR ~$ cd ~ OR ~$ cd ~/ OR ~$ cd /u/<userid>
When using a dot . character, this specifies to reference relative to your current location (the current working directory)
~$ cd ./my_directory
When using two dot .. characters, this specifies to reference the parent directory relative to your current location (the current working directory)
~$ pwd /u/<userid>/my_directory ~$ cd ../ ~$ pwd /u/<userid>
The relative path is the location relative to the current working directory. For example, assuming the current working directory is /u/<userid>, and the directory /u/<userid>/my_directory exists, then either of the following can be used to change the working directory to my_directory
~$ cd ./my_directory/ OR ~$ cd my_directory/
The absolute path is the full path to a file or directory location, regardless of the current working directory. For example, if a directory with the path /u/<userid>/my_directory/ exists, and provided that the directory exists on the system, the following can be used regardless of the current working directory
~$ cd /u/<userid>/my_directory/
To list all files and directories under a given path
~$ ls <path> ~$ ls cs_workshop/ dir0 file0 file1
To list all files and directories in the current working directory
~$ ls .
To list all files and directories of a directory relative to your current working directory
~$ ls ./my_directory
To list all files and directories of a directory regardless of your current working directory, use the absolute path
~$ ls /u/<userid>/my_directory
The following modifiers can also be applied to alter the output of an ls command
To list one filename or directory on each output line
~$ ls -1 ~/my_directory dir0 file0 file1
To list one file or directory for each line, including file/directory permissions, ownership, and modified timestamps
~$ ls -lh ~/my_directory total 10 drwx------ 2 <userid> <groupname> 2 Jun 11 00:01 dir0 -rw------- 1 <userid> <groupname> 4 Jun 11 00:02 file0 -rw------- 1 <userid> <groupname> 6 Jun 11 00:03 file1
Note, the flag h is added to make the file and directory sizes print in human readable format. Without this, using only -l will show file and directory sizes in Bytes
To list ALL files and directories, including hidden files or hidden directories
~$ ls -al ~/cs_workshop/ total 23 drwx------ 3 <userid> <groupname> 6 Jun 11 00:01 . drwx--x--- 36 <userid> <groupname> 68 Jun 11 00:01 .. drwx------ 2 <userid> <groupname> 2 Jun 11 00:01 dir0 -rw------- 1 <userid> <groupname> 4 Jun 11 00:02 file0 -rw------- 1 <userid> <groupname> 6 Jun 11 00:03 file1 -rw------- 1 <userid> <groupname> 7 Jun 11 00:05 .hidden_file0
To list ALL files and directories, placing the most recent at the top
~$ ls -alt ~/cs_workshop/ total 23 drwx--x--- 36 <userid> <groupname> 68 Jun 11 00:01 .. -rw------- 1 <userid> <groupname> 7 Jun 11 00:05 .hidden_file0 -rw------- 1 <userid> <groupname> 6 Jun 11 00:03 file1 -rw------- 1 <userid> <groupname> 4 Jun 11 00:02 file0 drwx------ 3 <userid> <groupname> 6 Jun 11 00:01 . drwx------ 2 <userid> <groupname> 2 Jun 11 00:01 dir0
To show information about a specific file or directory
~$ stat <path> ~$ stat ~/cs_workshop
Note, for all applicable examples, replace <src path> with the starting location or source, replace <dest path> with the destination location
To create an empty directory
~$ mkdir <path> ~$ mkdir ~/my_new_directory/
To create an empty file
~$ touch <path> ~$ touch ~/my_new_directory/my_new_file
To move an individual file
~$ mv <src path> <dest path> ~$ mv ~/my_old_directory/my_file ~/my_new_directory/
To rename a file
~$ mv <path to current file> <path to new file> ~$ mv ~/my_new_directory/my_file ~/my_new_directory/new_filename
To move a directory and its contents, for example, moving the directory ~/java_programs/ into the directory ~/code/
~$ mv <src path> <dest path> ~$ mv ~/java/ ~/code_projects/
To rename a directory
~$ mv <path to current directory> <path to new directory> ~$ mv ~/my_old_directory ~/my_new_directory
To copy a file
Note, if you specify the full path and/or a different file name, the file will copy to the path under the name specified
~$ cp <src path> <dest path> // preserve the filename ~$ cp ~/my_directory/my_file ~/my_other_directory/ ~$ cp ~/my_directory/my_file ~/my_new_directory/my_file // change the filename ~$ cp ~/my_directory/my_old_file ~/my_directory/my_new_file
To copy a directory and its contents to another directory, include the recursive option -r
~$ cp -r <src path> <dest path> ~$ ls ~/code_projects/ java python c ~$ cp -r ~/code_projects/ ~/code_submission/ ~$ ls ~/code_submission/ code_projects
To copy a directory's contents (not the directory itself), include the recursive option and a trailing /. at the end of the directory name
~$ cp -r <src path>/. <dest path> ~$ ls ~/code_projects/ java python c ~$ cp -r ~/code_projects/. ~/code_submission/ ~$ ls ~/code_submission/ java python c
Files/directories stored in a HOME directory (/u) are assigned default permissions and rarely need to be modified.
Files/directories stored in a PROJECT directory (/p) are assigned default permissions and rarely need to be modified.
Files/directories stored in SCRATCH/VOLATILE storage (/bigtemp) are NOT assigned default permissions and may need to be modified.
For a given file, we'll use example output from a stat command, omitting several details to focus on file permissions
~$ stat my_file Access: (0700/-rwx------) Uid: (12345/ <userid>) Gid: (54321/<groupname>)
Note, the syntax for Access: (0700/-rwx------) shows both forms of a file or directory's permissions. The first '0' is for the file's SUID/SGID/Sticky bit. For the purpose of this explanation, we will ignore this part and focus on the last three digits. The first dash '-' shows if the referenced item is a file (denoted with a dash '-' character) or directory (denoted with a 'd' character).
A linux file or directory has three octals to determine access. The three octals for access/permissions, using the above example, we'll separate each octal
rwx------
=> rwx --- ---
=> rwx --- ---
| | |
The first octal | |
is the OWNER's access, | |
which is <userid> | |
| |
The second octal |
is the GROUP's access, |
members of <groupname> |
|
The third octal
is for OTHER/EVERYONE else
(global) access
File permissions may be represented using characters
Equivalently, file permissions octals may be represented numerically
Using the above example for my_file, rewritten using numeric values for the octals, we'll separate each octal
700
=> 7 0 0
=> 7 0 0
| | |
The OWNER (<userid>) has | |
Read (4 or r), | |
Write (2 or w), | |
and Execute (1 or x) | |
| |
The GROUP <groupname> has |
no access |
|
Other (everyone else) has
no access
Using the above example for my_file, where the permissions are rwx------, we'll separate each octal
rwx------
=> rwx --- ---
=> rwx --- ---
| | |
The OWNER (<userid>) has | |
Read (4 or r), | |
Write (2 or w), | |
and Execute (1 or x) | |
| |
The GROUP <groupname> has |
no access |
|
Other (everyone else) has
no access
For another example, observe the file index.html
~$ stat index.html Access: (0604/-rw----r--) Uid: (12345/ <userid>) Gid: (54321/<groupname>)
Separating the octals, we can see that
=> rw- --- r--
=> rw- --- r--
| | |
The OWNER (<userid>) has | |
Read (4 or r), | |
and Write (2 or w) | |
| |
The GROUP <groupname> has |
no access |
|
Other (everyone else) has
Read (4 or r)
To change the permissions of a respective file or directory, the form is to set each of the three respective octals using numeric values
~$ chmod <permission octals> <path> // set so that only the owner can read/write ~$ chmod 600 ~/my_file // set so that only the owner can read/write/execute ~$ chmod 700 ~/my_program.py // set so that only the owner and group members can read/write to a file ~$ chmod 660 /p/my_project/shared_file // set so that the owner can read/write, and make it globally accessible (i.e. readable by a web server) ~$ chmod 604 ~/public_html/index.html
To view the total usage of an individual file or directory
~$ du -h <path>
To view all usage of files and directories, showing usage for each, in a given directory (i.e. your home directory)
~$ du -h --max-depth=1 <path> ~$ du -h --max-depth=1 /u/<userid>
A process in Linux is started when a command is entered. In other words, running a command such as ~$ ls -al ~/my_directory starts a short-lived process.
When a command is entered to start a process, it is started in the foreground. Meaning, the terminal will wait until the command or process completes before becoming usable again.
At times, it can be helpful to run a process in the background instead of the foreground. The benefit here is that the terminal becomes usable while the process is running so that other commands may be entered or to be able to start additional processes.
To launch a process in the background, without output redirection, append the & character to a command
~$ ./<command or program> &
Alternatively, you can move a process from the foreground to the background by pressing CTRL + z
~$ ./my_python_script.py // ctrl + z, entered [1]+ Stopped ./my_python_script.py
Then, you can resume a process in the background
~$ bg 1 [1]+ ./my_python_script.py
Note, without output redirection (described below), all output from background processes will be sent to either standard out (STDOUT), or standard error (STDERR), which is written to your terminal's foreground.
To list all running processes belonging to your user along with each respective Process ID (PID) number
~$ ps -aux | grep <userid> <userid> 12344 0.3 0.4 19164 3552 pts/1 R+ 15:32 0:00 ./my_python_script.py <userid> 12345 0.0 0.0 19164 3552 pts/1 R+ 15:45 0:00 ps -aux <userid> 12346 0.0 0.0 19164 3552 pts/1 R+ 15:45 0:00 grep <userid>
Notice that there are two processes, one for the command ps -aux (PID=12345), and the other for grep <userid> (PID=12346).
When a process has been moved to the background, you can list these as jobs as well
~$ jobs [1]+ Running ./my_python_script.py
To stop a process in the foreground, hold the CTRL key, and select the C key (CTRL + C). On MacOS, this is the Command Key and . (Command Key + .).
This example starts an infinite loop in the foreground, then CTRL + C is entered to stop the process
~$ while true; do echo "42"; sleep 1; done 42 42 42 ^C
To stop a process in the background, using the respective PID number
~$ kill <pid>
For example, to stop ./my_python_script.py which has a PID of 12344
~$ kill 12344
A job number of a process can also be used. For example, to stop ./my_python_script.py which has a job number of 1
~$ kill %1
Note, these examples send a signal to the process called a SIGTERM, or graceful stop. If a process doesn't stop from a SIGTERM, a forceful terminate signal SIGKILL can be sent using -9 in the kill command. Process cleanup does not take place when a SIGKILL is used.
This is used as a last resort option, when a process isn't responding to a SIGTERM
~$ kill -9 12344
At times, it can be helpful to redirect output of a command or process to a log file
To redirect STDOUT (standard output)
~$ <command or program> > ~/output.log ~$ ./a.out > ~/output.log
To redirect STDERR (standard error)
~$ <command or program> 2> ~/error.log ~$ ./a.out 2> ~/error.log
To redirect both STDOUT and STDERR
~$ <command or program> > ~/full_output.log 2>&1 ~$ ./my_python_script.py > ~/full_output.log 2>&1
To suppress or ignore output, you can send either or both streams to /dev/null as shown
~$ <command or program> > /dev/null 2>&1 ~$ ./my_python_script.py > /dev/null 2>&1
There are several options for editing files in a terminal. Two common text editors are nano and vim
For example, to open a file for editing using nano
~$ nano my_file
Note, vim is a powerful tool and has a learning curve, be sure to look up a reference cheat sheet
To exit without saving from editing a file with vim, hold shift, then press the colon / semi-colon key :, then enter q!
~$ vim my_file
There is a command named vimtutor that can be used for practicing and learning vim interactively
~$ vimtutor
To view the resources you're using on a given system, you can either start a process in the background, or open a second terminal to the server that your process is running on to run some of the following commands listed below.
This command will show the respective CPU and Memory usage of your processes. Note, you can also replace the environment variable $USER with your userid
~$ top -U $USER OR ~$ top -U <userid>
Another option is to list processes that you are using
~$ ps -aux | grep <userid>
Note, this section is for GPU enabled systems only. Login clusters such as portal and NoMachine do not have GPUs.
The GPUs available in CS are manufactured by Nvidia only. The information below pertains specifically to Nvidia GPUs.
The most common command for viewing GPU usage and information is
~$ nvidia-smi
More in-depth output can be obtained using nvtop, which requires loading the software module
~$ module load gcc ~$ module load nvtop ~$ nvtop
Note, this data is updated hourly.
To view home directory usage
~$ storagecheck -h
To view project directory usage, enter the path to the project directory
~$ storagecheck -p /p/<project directory name>