Welcome! This workshop is designed for users that are new or reviewing basic Linux principles. This workshop also covers details on connecting to CS servers.

Note, the portal SSH login cluster servers are not GPU accelerated systems.

There are no network or VPN requirements for connecting to the SSH portal login cluster. These servers can be accessed on or off grounds, and do not require a VPN connection to access.

To SSH to the portal login cluster, from your personal computer, open a terminal and enter the following command, replacing <userid> with your computing ID.

Temporary passwords are sent via email when your CS account is created initially. After logging in for the first time, you will be required to change your password.

On Linux / MacOS, one application for connecting to portal via SSH is named Terminal

~$ ssh <userid>@portal.cs.virginia.edu

On Windows, two applications for connecting to portal via SSH are Command Prompt or PowerShell

C:\Users\username> ssh <userid>@portal.cs.virginia.edu

Note, for the first time you connect via SSH, you will see the following output

Are you sure you want to continue connecting (yes/no)?

Enter yes to continue

For more information, see our wiki page about our (Logging in via SSH)

Note, the NoMachine login cluster servers are not GPU accelerated systems.

There are no network or VPN requirements for connecting to the NoMachine desktop GUI cluster. These servers can be accessed on or off grounds, and do not require a VPN connection to access.

After downloading the NoMachine client application from the (NoMachine Website), open the NoMachine application and follow the steps listed below

1. Select Add
2. Under Address enter the following. Note, 'Name' can be anything
   1. Name: UVA CS Remote Linux Desktop
   2. Host: nxfront.cs.virginia.edu
   3. Port: 4000
   4. Protocol: NX
3. Select Add to complete the adding of the CS NoMachine host information
4. Double left-click, or right click to select “Start Connection”
5. Then ender your user credentials, replacing <userid> with your computing ID
   1. Username: <userid>
   2. Password: <cs password>
6. For the first time connecting, select New desktop
7. Select Create a new virtual desktop

Thereafter, you'll be prompted for a few configuration options that are user specific.

For more information, see our wiki page about our (NoMachine cluster)

Note, the RDP Windows server is not a GPU enabled system.

Network Requirements:

• If you are on grounds connected to the eduroam Wi-Fi, then you can connect directly
  • Eduroam information can be found here: https://in.virginia.edu/eduroam

• If you are off grounds, a UVA VPN is required to connect
  • UVA VPN information can be found here: https://in.virginia.edu/vpn

For windows systems, the application Remote Desktop Connection or RDP is installed by default. To connect to the remote desktop server

1. Ensure that you are either on grounds connected to eduroam or using a UVA VPN
2. Open the Remote Desktop Connection application
3. Select the Show Options drop down to expand the Logon settings
4. Enter the following information, note the domain name CSDOM\ that precedes your userid
   1. Computer: winsrv01
   2. User name: CSDOM\<userid>
5. Select Connect and enter your password

For more information, see our wiki page about our general purpose (Windows Server)

Remote development is possible for most common software. Generally, you simply need to enter your CS credentials to log in. Some software is available through the UVA software gateway, and it is recommended to obtain software from the UVA software gateway if possible.

• VSCode (Windows/MacOs/Linux)
  • Instructions for VSCode: https://code.visualstudio.com/docs/remote/ssh

• SecureCRT (Windows)
  • Available through the UVA Software Gateway: https://in.virginia.edu/software

• SecureFX (Windows/MacOS)
  • Available through the UVA Software Gateway: https://in.virginia.edu/software

For further reading about SSH connections, see our (SSH wiki page)

For further reading about remote development, see our (Remote Development wiki page)

Below are some example commands that can be run within a terminal over an SSH connection or using the application “Terminal” while using a NoMachine desktop GUI.

Note, for all applicable examples, replace <path> with the location of a file or directory

To print the current working directory (i.e. your current location)

~$ pwd

To change/move to another directory

~$ cd <path>

Note, the tilde character ~ specifies to use your home directory location. Further, simply using the command ~$ cd will move you to your home directory. The following are all equivalent to move to your home directory

~$ cd

OR

~$ cd ~

OR

~$ cd ~/

OR

~$ cd /u/<userid>

When using a dot . character, this specifies to reference relative to your current location (the current working directory)

~$ cd ./my_directory

When using two dot .. characters, this specifies to reference the parent directory relative to your current location (the current working directory)

~$ pwd
/u/<userid>/my_directory

~$ cd ../
~$ pwd
/u/<userid>

The relative path is the location relative to the current working directory. For example, assuming the current working directory is /u/<userid>, and the directory /u/<userid>/my_directory exists, then either of the following can be used to change the working directory to my_directory

~$ cd ./my_directory/

OR

~$ cd my_directory/

The absolute path is the full path to a file or directory location, regardless of the current working directory. For example, if a directory with the path /u/<userid>/my_directory/ exists, and provided that the directory exists on the system, the following can be used regardless of the current working directory

~$ cd /u/<userid>/my_directory/

To list all files and directories under a given path

~$ ls <path>

~$ ls cs_workshop/
dir0  file0  file1

To list all files and directories in the current working directory

~$ ls .

To list all files and directories of a directory relative to your current working directory

~$ ls ./my_directory

To list all files and directories of a directory regardless of your current working directory, use the absolute path

~$ ls /u/<userid>/my_directory

The following modifiers can also be applied to alter the output of an ls command

To list one filename or directory on each output line

~$ ls -1 ~/my_directory
dir0
file0
file1

To list one file or directory for each line, including file/directory permissions, ownership, and modified timestamps

~$ ls -lh ~/my_directory
total 10
drwx------  2 <userid> <groupname> 2 Jun 11 00:01 dir0
-rw-------  1 <userid> <groupname> 4 Jun 11 00:02 file0
-rw-------  1 <userid> <groupname> 6 Jun 11 00:03 file1

Note, the flag h is added to make the file and directory sizes print in human readable format. Without this, using only -l will show file and directory sizes in Bytes

• The total is the disk usage of the listed files/directories in blocks. Each block is 1024 bytes

• The first column is the file/directory permissions

• The second column is the number of links to the file/directory

• The third column is the owner of the file/directory

• The fourth column is the group of the file/directory

• The fifth column shows the size of the file/directory. For directories, this is not the sum of the contents

• The sixth, seventh, and eighth columns show the modification date

• The nineth column shows the file/directory name

To list ALL files and directories, including hidden files or hidden directories

~$ ls -al ~/cs_workshop/
total 23
drwx------   3 <userid> <groupname>  6 Jun 11 00:01 .
drwx--x---  36 <userid> <groupname> 68 Jun 11 00:01 ..
drwx------   2 <userid> <groupname>  2 Jun 11 00:01 dir0
-rw-------   1 <userid> <groupname>  4 Jun 11 00:02 file0
-rw-------   1 <userid> <groupname>  6 Jun 11 00:03 file1
-rw-------   1 <userid> <groupname>  7 Jun 11 00:05 .hidden_file0

To list ALL files and directories, placing the most recent at the top

~$ ls -alt ~/cs_workshop/
total 23
drwx--x---  36 <userid> <groupname> 68 Jun 11 00:01 ..
-rw-------   1 <userid> <groupname>  7 Jun 11 00:05 .hidden_file0
-rw-------   1 <userid> <groupname>  6 Jun 11 00:03 file1
-rw-------   1 <userid> <groupname>  4 Jun 11 00:02 file0
drwx------   3 <userid> <groupname>  6 Jun 11 00:01 .
drwx------   2 <userid> <groupname>  2 Jun 11 00:01 dir0

To show information about a specific file or directory

~$ stat <path>

~$ stat ~/cs_workshop

Note, for all applicable examples, replace <src path> with the starting location or source, replace <dest path> with the destination location

To create an empty directory

~$ mkdir <path>

~$ mkdir ~/my_new_directory/

To create an empty file

~$ touch <path>

~$ touch ~/my_new_directory/my_new_file

To move an individual file

~$ mv <src path> <dest path>

~$ mv ~/my_old_directory/my_file ~/my_new_directory/

To rename a file

~$ mv <path to current file> <path to new file>

~$ mv ~/my_new_directory/my_file ~/my_new_directory/new_filename

To move a directory and its contents, for example, moving the directory ~/java_programs/ into the directory ~/code/

~$ mv <src path> <dest path>

~$ mv ~/java/ ~/code_projects/

To rename a directory

~$ mv <path to current directory> <path to new directory>

~$ mv ~/my_old_directory ~/my_new_directory

To copy a file

Note, if you specify the full path and/or a different file name, the file will copy to the path under the name specified

~$ cp <src path> <dest path>

// preserve the filename
~$ cp ~/my_directory/my_file ~/my_other_directory/
~$ cp ~/my_directory/my_file ~/my_new_directory/my_file

// change the filename
~$ cp ~/my_directory/my_old_file ~/my_directory/my_new_file

To copy a directory and its contents to another directory, include the recursive option -r

~$ cp -r <src path> <dest path>


~$ ls ~/code_projects/
java
python
c

~$ cp -r ~/code_projects/ ~/code_submission/

~$ ls ~/code_submission/
code_projects

To copy a directory's contents (not the directory itself), include the recursive option and a trailing /. at the end of the directory name

~$ cp -r <src path>/. <dest path>

~$ ls ~/code_projects/
java
python
c

~$ cp -r ~/code_projects/. ~/code_submission/

~$ ls ~/code_submission/
java
python
c

Files/directories stored in a HOME directory (/u) are assigned default permissions and rarely need to be modified.

Files/directories stored in a PROJECT directory (/p) are assigned default permissions and rarely need to be modified.

Files/directories stored in SCRATCH/VOLATILE storage (/bigtemp) are NOT assigned default permissions and may need to be modified.

For a given file, we'll use example output from a stat command, omitting several details to focus on file permissions

~$ stat my_file
Access: (0700/-rwx------)  Uid: (12345/ <userid>)   Gid: (54321/<groupname>)

Note, the syntax for Access: (0700/-rwx------) shows both forms of a file or directory's permissions. The first '0' is for the file's SUID/SGID/Sticky bit. For the purpose of this explanation, we will ignore this part and focus on the last three digits. The first dash '-' shows if the referenced item is a file (denoted with a dash '-' character) or directory (denoted with a 'd' character).

A linux file or directory has three octals to determine access. The three octals for access/permissions, using the above example, we'll separate each octal

rwx------
=> rwx --- ---
=> rwx                      ---                      ---
   |                        |                        |
   The first octal          |                        |
   is the OWNER's access,   |                        |
   which is <userid>        |                        |
                            |                        |
                            The second octal         |
                            is the GROUP's access,   |
                            members of <groupname>   |
                                                     |
                                                     The third octal 
                                                     is for OTHER/EVERYONE else 
                                                     (global) access

File permissions may be represented using characters

• r for Read (open a file)
• w for Write (write to a file)
• x for Execute (run a program or list a directory)

Equivalently, file permissions octals may be represented numerically

• 4 for Read (open a file)
• 2 for Write (write to a file)
• 1 for Execute (run a program or list a directory)
• The resulting permissions are the sum of the allowed access

Using the above example for my_file, rewritten using numeric values for the octals, we'll separate each octal

700
=> 7 0 0
=> 7                          0                           0
   |                          |                           |
   The OWNER (<userid>) has   |                           |
   Read (4 or r),             |                           |
   Write (2 or w),            |                           |
   and Execute (1 or x)       |                           |
                              |                           |
                              The GROUP <groupname> has   |
                              no access                   |
                                                          |
                                                          Other (everyone else) has
                                                          no access
   

Using the above example for my_file, where the permissions are rwx------, we'll separate each octal

rwx------
=> rwx --- ---
=> rwx                        ---                         ---
   |                          |                           |
   The OWNER (<userid>) has   |                           |
   Read (4 or r),             |                           |
   Write (2 or w),            |                           |
   and Execute (1 or x)       |                           |
                              |                           |
                              The GROUP <groupname> has   |
                              no access                   |
                                                          |
                                                          Other (everyone else) has
                                                          no access
   

For another example, observe the file index.html

~$ stat index.html
Access: (0604/-rw----r--)  Uid: (12345/ <userid>)   Gid: (54321/<groupname>)

Separating the octals, we can see that

=> rw- --- r--
=> rw-                        ---                         r--
   |                          |                           |
   The OWNER (<userid>) has   |                           |
   Read (4 or r),             |                           |
   and Write (2 or w)         |                           |
                              |                           |
                              The GROUP <groupname> has   |
                              no access                   |
                                                          |
                                                          Other (everyone else) has
                                                          Read (4 or r)

To change the permissions of a respective file or directory, the form is to set each of the three respective octals using numeric values

~$ chmod <permission octals> <path>

// set so that only the owner can read/write
~$ chmod 600 ~/my_file

// set so that only the owner can read/write/execute
~$ chmod 700 ~/my_program.py

// set so that only the owner and group members can read/write to a file
~$ chmod 660 /p/my_project/shared_file

// set so that the owner can read/write, and make it globally accessible (i.e. readable by a web server)
~$ chmod 604 ~/public_html/index.html

To view the total usage of an individual file or directory

~$ du -h <path>

To view all usage of files and directories, showing usage for each, in a given directory (i.e. your home directory)

~$ du -h --max-depth=1 <path>

~$ du -h --max-depth=1 /u/<userid>

A process in Linux is started when a command is entered. In other words, running a command such as ~$ ls -al ~/my_directory starts a short-lived process.

When a command is entered to start a process, it is started in the foreground. Meaning, the terminal will wait until the command or process completes before becoming usable again.

At times, it can be helpful to run a process in the background instead of the foreground. The benefit here is that the terminal becomes usable while the process is running so that other commands may be entered or to be able to start additional processes.

To launch a process in the background, without output redirection, append the & character to a command

~$ ./<command or program> &

Alternatively, you can move a process from the foreground to the background by pressing CTRL + z

~$ ./my_python_script.py

// ctrl + z, entered
[1]+ Stopped ./my_python_script.py

Then, you can resume a process in the background

~$ bg 1
[1]+ ./my_python_script.py

Note, without output redirection (described below), all output from background processes will be sent to either standard out (STDOUT), or standard error (STDERR), which is written to your terminal's foreground.

To list all running processes belonging to your user along with each respective Process ID (PID) number

~$ ps -aux | grep <userid>
<userid>   12344  0.3  0.4  19164  3552 pts/1    R+   15:32   0:00 ./my_python_script.py
<userid>   12345  0.0  0.0  19164  3552 pts/1    R+   15:45   0:00 ps -aux
<userid>   12346  0.0  0.0  19164  3552 pts/1    R+   15:45   0:00 grep <userid>

Notice that there are two processes, one for the command ps -aux (PID=12345), and the other for grep <userid> (PID=12346).

When a process has been moved to the background, you can list these as jobs as well

~$ jobs
[1]+  Running     ./my_python_script.py

To stop a process in the foreground, hold the CTRL key, and select the C key (CTRL + C). On MacOS, this is the Command Key and . (Command Key + .).

This example starts an infinite loop in the foreground, then CTRL + C is entered to stop the process

~$ while true; do echo "42"; sleep 1; done
42
42
42
^C

To stop a process in the background, using the respective PID number

~$ kill <pid>

For example, to stop ./my_python_script.py which has a PID of 12344

~$ kill 12344

A job number of a process can also be used. For example, to stop ./my_python_script.py which has a job number of 1

~$ kill %1

Note, these examples send a signal to the process called a SIGTERM, or graceful stop. If a process doesn't stop from a SIGTERM, a forceful terminate signal SIGKILL can be sent using -9 in the kill command. Process cleanup does not take place when a SIGKILL is used.

This is used as a last resort option, when a process isn't responding to a SIGTERM

~$ kill -9 12344

At times, it can be helpful to redirect output of a command or process to a log file

To redirect STDOUT (standard output)

~$ <command or program> > ~/output.log

~$ ./a.out > ~/output.log

To redirect STDERR (standard error)

~$ <command or program> 2> ~/error.log

~$ ./a.out 2> ~/error.log

To redirect both STDOUT and STDERR

~$ <command or program> > ~/full_output.log 2>&1 

~$ ./my_python_script.py > ~/full_output.log 2>&1 

To suppress or ignore output, you can send either or both streams to /dev/null as shown

~$ <command or program> > /dev/null 2>&1

~$ ./my_python_script.py > /dev/null 2>&1

There are several options for editing files in a terminal. Two common text editors are nano and vim

For example, to open a file for editing using nano

~$ nano my_file

Note, vim is a powerful tool and has a learning curve, be sure to look up a reference cheat sheet

To exit without saving from editing a file with vim, hold shift, then press the colon / semi-colon key :, then enter q!

~$ vim my_file

There is a command named vimtutor that can be used for practicing and learning vim interactively

~$ vimtutor

To view the resources you're using on a given system, you can either start a process in the background, or open a second terminal to the server that your process is running on to run some of the following commands listed below.

This command will show the respective CPU and Memory usage of your processes. Note, you can also replace the environment variable $USER with your userid

~$ top -U $USER

OR

~$ top -U <userid>

Another option is to list processes that you are using

~$ ps -aux | grep <userid>

Note, this section is for GPU enabled systems only. Login clusters such as portal and NoMachine do not have GPUs.

The GPUs available in CS are manufactured by Nvidia only. The information below pertains specifically to Nvidia GPUs.

The most common command for viewing GPU usage and information is

~$ nvidia-smi

More in-depth output can be obtained using nvtop, which requires loading the software module

~$ module load gcc
~$ module load nvtop
~$ nvtop

Note, this data is updated hourly.

To view home directory usage

~$ storagecheck -h

To view project directory usage, enter the path to the project directory

~$ storagecheck -p /p/<project directory name>