Differences

This shows you the differences between two versions of the page.

--- compute_resources [2021/08/12 14:18] – pgh5a
+++ compute_resources [2023/09/29 18:41] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
+==== Computing Resources ====
+The CS Dept. deploys general purpose compute and GPU compute servers, as well as specialized GPU servers. We also have database, [[nx_lab|NX/Nomachine (remote Linux desktop)]], [[windows_server|a Windows desktop server]], and Docker servers. This section describes the general use and research use servers (see other sections for database and other server information).
+=== portal load balanced servers ===
+The //portal// servers are general purpose servers into which anyone can login. They are available for general use. They are also the "jump off" point for off Grounds connections to the CS network. The servers are in a load balanced cluster, and are accessed through //ssh// to ''%%portal.cs.virginia.edu%%''. See: [[compute_portal|The portal cluster]]
+Use these servers to code, compile, test, etc.. However these are //not// meant for long running processes that will use excessive resources.  Computationally expensive processes should be run on other servers listed below.
+//(all GPU, Memory, CPU, etc. counts are per node)//.
+^ Hostname ^ Memory (GB) ^ CPU Type ^ CPUs ^ Cores/CPU ^ Threads/Core ^ Total Cores ^ OS ^
+| portal[01-10] | 132-256 | Intel | 1 | 8 | 2 | 16 | Ubuntu |
+=== GPU servers ===
+The gpusrv* servers are general purpose servers that contain GPUs into which anyone can login (via 'ssh'). They are intended for code development, testing, and short computations. Long running computations are discouraged, and are better suited to one of the GPU servers controlled by the job scheduler (see the SLURM section below).
+//(all GPU, Memory, CPU, etc. counts are per node)//.
+^ Hostname ^ Memory (GB) ^ CPU Type ^ CPUs ^ Cores/CPU ^ Threads/Core ^ Total Cores ^ GPUs ^ GPU Type ^ GPU RAM (GB) ^
+| gpusrv[01-08] | 256 | Intel | 1 | 10 | 2 | 20 | 4 | Nvidia RTX 2080Ti | 11 |
+| gpusrv[09-16] | 512 | Intel | 2 | 10 | 2 | 40 | 4 | Nvidia RTX 4000   | 8  |
+| gpusrv[17-18] | 128 | Intel | 1 | 10 | 2 | 20 | 4 | Nvidia RTX 2080Ti | 11 |
+=== Docker servers ===
+The docker servers allow users to instantiate a docker container without the need for super-user (root) privileges. Once a user uses 'ssh' to login to the server, the user will execute ''%%sudo /usr/bin/docker%%'' to create a Docker container. Note that the //slurm[1-5]// nodes, available through the SLURM job scheduler, also allow general users to create docker containers (see below).
+//(all GPU, Memory, CPU, etc. counts are per node)//.
+^ Hostname ^ Memory (GB) ^ CPU Type ^ CPUs ^ Cores/CPU ^ Threads/Core ^ Total Cores ^
+| docker01 | 256 | Intel | 2 | 10 | 2 | 40 |
+| docker02 | 64 | Intel | 1 | 4 | 2 | 8 |
+| docker03 | 8 | Intel | 1 | 2 | 2 | 4 |
+=== Nodes controlled by the SLURM Job Scheduler ===
+//See our main article on [[compute_slurm|Slurm]] for more information.//
+These servers are available by submitting a job through the SLURM job scheduler. They are not available for direct logins via 'ssh'. However, users can login to these servers without a job script using the 'srun -i' direct login command. See the SLURM section for more information.
+Per CS Dept. policy, servers are placed into the SLURM job scheduling queues and are available for general use. Also, per that policy, if a user in a research group that originally purchased the hardware requires exclusive use of that hardware, they can be given a reservation for that exclusive use for a specified time. Otherwise, the systems are open for use by anyone with a CS account. This policy was approved by the CS Dept. Computing Committee comprised of CS Faculty.
+This policy allows servers to be used when the project group is not using them. So instead of sitting idle and consuming power and cooling, other Dept. users can benefit from the use of these systems.
+//(all GPU, Memory, CPU, etc. counts are per node)//.
+//This list may not be up to date. Use the 'sinfo' command for current server inventory.//
+^ Hostname        ^ Mem (GB) ^ CPU   ^ #CPUs ^ Cores ^ Threads / Core ^ Total Threads ^ GPUs  ^ GPU Type           ^ GPU RAM (GB) ^ OS                  ^
+| adriatic[01-06] | 1024     | Intel | 2     | 8     | 2              | 32            | 4     | Nvidia RTX 4000    | 8            | Ubuntu Server 22.04 |
+| affogato[01-10] | 128      | Intel | 2     | 8     | 2              | 32            | 0     |                    |              | Ubuntu Server 22.04 |
+| affogato[11-15] | 128      | Intel | 2     | 8     | 2              | 32            | 4     | Nvidia GTX1080Ti   | 11           | Ubuntu Server 22.04 |
+| ai[01-06]       | 64       | Intel | 2     | 8     | 2              | 32            | 4     | Nvidia GTX1080Ti   | 11           | Ubuntu Server 22.04 |
+| ai[07-10]       | 128      | Intel | 2     | 8     | 2              | 32            | 4     | Nvidia GTX1080Ti   | 11           | Ubuntu Server 22.04 |
+| cheetah01       | 256      | AMD   | 2     | 8     | 2              | 32            | 4     | Nvidia A100        | 40           | Ubuntu Server 22.04 |
+| cheetah[02-03]  | 1024(2)  | Intel | 2     | 18    | 2              | 72            | 2     | Nvidia RTX 2080Ti  | 11           | Ubuntu Server 22.04 |
+| cheetah04       | 1024     | AMD   | 2     | 64    | 2              | 256           | 4     | Nvidia A100	       | 80(5)        | Ubuntu Server 22.04 |
+| cortado[01-10]  | 512      | Intel | 2     | 12    | 2              | 48            | 0     |                    |              | Ubuntu Server 22.04 |
+| doppio[01-05]   | 128      | Intel | 2     | 16    | 2              | 64            | 0     |                    |              | Ubuntu Server 22.04 |
+| epona           | 64       | Intel | 1     | 4     | 2              | 8             | 0     |                    |              | Ubuntu Server 22.04 |
+| heartpiece      | 160      | Intel | 2     | 10    | 2              | 40            | 0     |                    |              | Ubuntu Server 22.04 |
+| hydro           | 256      | Intel | 2     | 16    | 2              | 64            | 0     |                    |              | Centos Linux 7      |
+| jaguar01        | 1024     | Intel | 2     | 16    | 2              | 64            | 4     | Nvidia A40         | 48 (4)       | Ubuntu Server 22.04 |
+| jaguar02        | 1024     | Intel | 2     | 8     | 2              | 32            | 8     | Nvidia A16         | 16           | Ubuntu Server 22.04 |
+| jaguar03        | 256      | AMD   | 2     | 56    | 2              | 224           | 8     | Nvidia RTX A4500   | 20           | Ubuntu Server 22.04 |
+| jaguar04        | 256      | Intel | 2     | 16    | 2              | 64            | 4     | Nvidia A40         | 48 (4)       | Ubuntu Server 22.04 |
+| jaguar05        | 256      | Intel | 1     | 8     | 2              | 16            | 4     | Quadro RTX 4000    | 8            | Ubuntu Server 22.04 |
+| jaguar06        | 128      | Intel | 2     | 24    | 2              | 48            | 2     | Nvidia A40         | 48           | Ubuntu Server 22.04 |
+| lotus           | 256      | Intel | 2     | 20    | 2              | 80            | 8     | Nvidia RTX 6000    | 24           | Ubuntu Server 22.04 |
+| lynx[01-04]     | 64       | Intel | 4     | 8     | 2              | 32            | 4     | Nvidia GTX1080Ti   | 11           | Ubuntu Server 22.04 |
+| lynx[05-07]     | 64       | Intel | 4     | 8     | 2              | 32            | 4     | Nvidia P100        | 16           | Ubuntu Server 22.04 |
+| lynx[08-09]     | 64       | Intel | 4     | 8     | 2              | 32            | 0     |                    |              | Ubuntu Server 22.04 |
+| lynx10	      | 64       | Intel | 4     | 8     | 2              | 32            | 3     | Nvidia GTX1080     | 8            | Ubuntu Server 22.04 |
+| lynx[11,12]     | 64       | Intel | 4     | 8     | 2              | 32            | 4     | Nvidia Titan X     | 12           | Ubuntu Server 22.04 |
+| optane01        | 1024(1)  | Intel | 2     | 16    | 2              | 64            | 0     |                    |              | Ubuntu Server 22.04 |
+| panther01       | 512      | Intel | 1     | 8     | 2              | 16            | 0     |                    |              | Ubuntu Server 22.04 |
+| pegasusboots    | 192      | Intel | 2     | 10    | 2              | 40            | 0     |                    |              | Ubuntu Server 22.04 |
+| ristretto[01-04]| 128      | Intel | 2     | 6     | 1              | 12            | 8     | Nvidia GTX1080Ti   | 11           | Ubuntu Server 22.04 |
+| sds[01-02]	  | 512      | Intel | 2     | 10    | 2              | 40            | 4     | Nvidia RTX A4000   | 16           | Ubuntu Server 22.04 |
+| slurm[1-5](3)   | 512      | Intel | 2     | 12    | 2              | 24            | 0     |                    |              | Ubuntu Server 22.04 |
+| titanx[01-03]	  | 256      | Intel | 1     | 8     | 2              | 16            | 1     | Nvidia Titan X     | 12           | Ubuntu Server 22.04 |
+| titanx[04-06]	  | 64       | Intel | 1     | 6     | 2              | 12            | 1     | Nvidia Titan X     | 12           | Ubuntu Server 22.04 |
+(1) 512GB Intel Optane memory, 512 DDR4 memory
+(2) In addition to 1TB of DDR4 RAM, these servers also house a 900GB Optane NVMe SSD and a 1.6TB NVMe regular SSD drive
+(3) All users have 'sudo' privileges to execute /usr/bin/docker on these nodes
+(4) Scaled to 96GB by pairing GPUs with NVLink technology that provides a maximum bi-directional bandwidth of 112GB/s between the paired A40s
+(5) Scaled to 320GB by aggregating all four GPUs with NVLink technology
+=== Job Scheduler Queues ===
+//See our [[compute_slurm#partitions|main article on Slurm]] for more information on using queues ("partitions")//
+The main partition contains nodes without GPUs, the gpu partition contains nodes with gpus, the nolom partition has nodes with no gpus and no time limit on jobs, and the gnolim partition has nodes with GPUs but no time limit on jobs. The time limit on jobs in the main and gpu partitions is shown using the 'sinfo' command.
+^ Queue ^ Nodes ^
+| main | cortado[01-10], hydro, optane01, slurm[1-5] |
+| gpu | adriatic[01-06], affogato[11-15], ai[01-08], cheetah[01-03], jaguar[01-03], lotus, lynx[01-12], ristretto[01-04], sds[01-02], titanx[01-06] |
+| nolim | doppio[01-05], epona, heartpiece, pegasusboots |
+| gnolim | ai[01-10], titanx[01-06] |
+=== Group specific computing resources ===
+Several groups in CS deploy servers that are used exclusively by that group. Approximately 40 servers are deployed in this fashion, ranging from traditional CPU servers to specialized servers containing GPU accelerators.
+=== Docker Server ===
+//See our main article on [[compute_docker|Docker]] for more information.//
+The Docker server on hostname 'docker01' is currently available to all department members. Docker containers can also be created on nodes //slurm[1-5]//.