Permission is granted to copy and distribute this manual so long as this copyright page accompanies any copies. The Legion system software herein described is intended for research and is available free-of-charge for that purpose. Permission is not granted for distributing the Legion system software outside of your site.
In no event shall the University of Virginia be liable to any party for direct, indirect, special, incidental, or consequential damages arising out of the use of the Legion system software and its documentation.
The University of Virginia specifically disclaims any warranties, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. The software provided hereunder is on an "as is" basis, and the University of Virginia has no obligation to provide maintenance, support, updates, enhancements, or modifications.
This work partially supported by DARPA (Navy) contract # N66001-96-C-8527, DOE grant DE-FD02-96ER25290, DOE contract Sandia LD-9391, Northrup-Grumman (for the DoD HPCMOD/PET program), DOE D459000-16-3C and DARPA (GA) SC H607305A
This manual is an introduction to Legion 1.2 and is written for users with little or no previous experience with Legion. It explains the purpose, history, and design of Legion and gives a high-level view of Legion context space and an introduction to navigating and manipulating contexts. There are also instructions for logging in to an active Legion system. Users who need to install a new system or to start-up a previously installed system should consult the System Administrator Manual ( See Downloading Legion 1.2 ). If you need more help please contact us ( See Getting help ).
There are three other Legion manuals, each aimed at a specific user, that can be consulted for more information. The System Administrator Manual provides information and documentation for administrators of Legion systems, including installing and running a Legion system, configuring security features, and managing resources. The Developer Manual provides information and documentation for programmers working in Legion and includes information about languages, libraries, core objects, and implementing new objects. The Reference Manual is a more specialized resource, providing detailed information about specific elements of the Legion system. There are also man pages for all Legion commands, included with the system files, and several on-line tutorials on the Legion web site (http://legion.virginia.edu).
Developed at the University of Virginia, Legion is an integrated software system for distributed parallel computation. While fully supporting existing codes written in MPI and PVM, Legion provides features and services that allow users to take advantage of much larger, more complex resource pools. With Legion, for example, a user can easily run a computation on a supercomputer at a national center while dynamically visualizing the results on a local machine. As another example, Legion makes it trivial to schedule and run a large parameter space study on several workstation farms simultaneously. Legion permits computational scientists to use cycles wherever they are, allowing bigger jobs to run in shorter times through higher degrees of parallelization.
These features also make Legion attractive to administrators looking for ways to increase and simplify the use of shared high-performance machines. The Legion implementation emphasizes extensibility, and multiple policies for resource use can be embedded in a single Legion system that spans multiple resources or even administrative domains.
This manual assumes that you are working on a previously installed, compiled, and running system. 1 (See the System Administrator Manual for information on installing, compiling, and starting a new Legion system.) Before going any further, be sure that the system is properly installed and running.
Please note that while your path will change as you move through Legion's binary files, it will not change as you move through context space. We are currently developing a graphic user interface to help users negotiate this space more easily. For the time being, though, you will be working from your command line and typing in commands to run programs and manipulate contexts.
A user who logs in to a Legion system is identified by a special object called the AuthenticationObject : it contains the user's password, initial implicit parameters (the Legion equivalent of a the Unix "environment"), and other information. AuthenticationObjects are created when a user creates a user id. A set of Legion commands ( legion_set_acl , legion_get_acl , legion_passwd , legion_set_implicit_params , and legion_get_ implicit_params ) can be used to retrieve or change this information (see section See Security in the Legion Reference Manual or the man pages for a discussion of how to use these commands). When an authenticated user runs a Legion process a certificate confirming his or her identify is passed along to verify that the user has permission to run the process. This certificate is created and signed by the user's AuthenticationObject.
AuthenticationObjects must be permanent in order to be useful. If an AuthenticationObject is destroyed, its associated LOID, which identifies the user to the rest of the system, is lost. There is no way to generate an identical LOID for a new AuthenticationObject.
In order to log in to the system, you will need a user id, which can be created by either the user or system administrator. You can then log in, with the legion_login command. The system will then request your password and check it with your AuthenticationObject, which verifies your identity and security privileges.
If you do not already have a user id, you can create one. Normally, the system administrator creates user ids and simultaneously enters them in the groups that have appropriate security rights on the system. If you create your own id you may not have the necessary permissions to enter the id in such groups and use the system resources that the groups control. However, your new user id will otherwise function normally and can be used to protect your resources from other users of the system.
Note that your working context will not change when you enter the new shell, but your access privileges are different. Objects created in bob 's shell will by default require bob 's access privileges to use them.
While logged in, you can change your password and other parameters of your environment. The password may be changed with legion_passwd . Note that in this release you must specify the user whose password you are changing, even if it is yourself:
To change your implicit parameters, which can be used to control the behavior of security as well as other Legion tools and objects, use the legion_set_implicit_params tools documented in the reference manual. The access permissions of existing objects can be changed with legion_set_acl (please see also See Using security features , in the System Administrator Manual).
Legion is an object-based system organized by classes and metaclasses (classes of classes). Every part of Legion, including classes and metaclasses, is represented by a Legion object and users manipulate various elements by controlling their representative objects. All Legion objects are automatically assigned an identifying number, called a LOID (see section See About the LOID below), and can also be assigned a user-chosen string name, called a context name . Users can organize these context names into a discrete context space, and from that space can run programs, use far-flung resources, and take advantage of Legion's other features.
Legion users will probably spend a great deal of time working in context space, so it is important to understand the initial geography of your system's context space and how to negotiate that space. A typical newly installed system has the basic elements of context space: there are four sub-contexts and several starter context names for commonly used objects. See Contents of a typical new Legion system context space , below, shows the contents a newly installed Legion system's context space. Note that your system may differ, since a system administrator may have already customized your space to take advantage of your system's resources.
To see the contents of your context space, you can use either the legion_context_list or the legion_ls command. (See section See Command-line functions in the Legion Reference Manual for a list of all Legion commands' usage.) The full context path name for BootstrapHost in the system shown in See Contents of a typical new Legion system context space would be /hosts/BootstrapHost . 2
For more information on viewing context space and using the other context-related commands, please see See Working in context space . A new Legion system includes four sub-contexts, called class , hosts , vaults , and home . These in turn contain context names that refer to the initial contents of a new system: the class sub-context contains context names of particular classes of Legion objects, the hosts sub-context contains names of the host objects currently in your system, and the vaults sub-context contains names of the vault object currently in your system. The home sub-context is initially empty.
When newly installed, hosts contains two names, BootstrapHost and the DNS name of the bootstrap host. (A host is a machine in a Legion system. A vault is a file storage system used to store inactive Legion objects. A host object represents and manages a given host in a Legion system, and a vault object represents and manages a given vault in a Legion system.) Both of these names point to the bootstrap host object, which represents the bootstrap host. On the other hand, vaults contains only one name, BootstrapVault , which points to the bootstrap vault object. (See See Host and vault objects for more information on hosts and vaults.)
A newly started class sub-context contains context names for all of the basic Legion object classes, such as BasicFileClass and ttyObjectClass . Note that these classes are themselves objects, even though they are responsible for managing a given set of objects. All Legion objects are organized into classes, and all classes are organized into a huarache of metaclasses , with LegionClass as the supreme metaclass. Note also that the context space illustrated in See Contents of a typical new Legion system context space does not reflect this hierarchy. This is a key feature of Legion: the physical location and organization of Legion objects is unrelated to any user's context space. The objects named in a host sub-context do not even have to know of each other's existence or even be able to communicate with each other. They do not even have to be hosts: the user can create, delete, and use contexts as he or she chooses.
Context space is a way of organizing Legion space. It is similar to Unix's system of directories and files but while Unix space is location specific, context space can be arranged to suit the user's convenience. Contexts are string names assigned to Legion objects. The user can assign an object a name (or multiple names) without worrying about the object's physical location. Context space is organized into a series of sub-contexts, similar to Unix directories, and can be assigned mnemonic or logical names as suits the user. A single sub-context can contain context names that refer to objects located on a single machine or separated by large distances.
All Legion objects are automatically assigned a unique system-level identifying name, called a LOID (Legion Object Identifier). The LOID allows the system to find objects scattered on several different machines. Since a LOID consists of several lines of numbers, it can be awkward at best for a user to track objects via their LOIDs. Users can instead assign a string name to an object's LOID, making it easier to find and use any object in Legion. This name, called a context name, resides only in a user's individual context space and refers only to a particular LOID. Context names are grouped into contexts and these contexts are organized into a hierarchy similar to a Unix directory structure.
Each user has a personal context space, which she organizes and maintains to suit her own needs. Objects are named according to each user's preferences, and each user's context space is independent of other users' context spaces, allowing different users to assign multiple context names to a single object without interfering with each other or with the object. This is discussed further in section See Organizing context space , but the next section provides a brief look at the anatomy of the LOID.
A LOID is a variable length binary structure (typically more than 128 bytes), which contains a type field followed by several variable size fields. The first variable field indicates the Legion domain in which the LOID was created, the second is the class identifier, the third is the instance number, and the fourth is a public key. The type, domain, and class identifier fields must be present. A LOID may be appended with any number of other fields, each of whose size and meaning are dependent on the LOID type. LOIDs are represented as strings, using standard "dotted hex" notation. For example, a typical LOID would look something like this:
In this sample the type field, 1 , indicates the LOID's type; the first variable field, 01 , indicates the domain in which the object was created; the second variable field, 07 , indicates the class that the object belongs to; the third variable field, 01 , indicates the object's instance number; and the fourth variable field, 000001fc0d24ab ... , is the object's RSA public key. This notation is obviously too cumbersome for most users to manipulate as strings, so the naming service provided by context spaces allows users to deal with objects using meaningful string names chosen by the user. This separation of concerns between system-level names and user-level names allows Legion LOIDs to contain system-level information, such as an object's public key, without polluting the user-level name-space.
All contexts in Legion are organized into a Unix-like hierarchy, with a root context that contains a set of other contexts. This structure provides each object with an absolute path name. Just as a Unix user organizes files in a series of directories, a Legion user organizes object in a series of contexts: a Legion system has a root context, and within that space each user has a personal root context, a home context in which he or she works. 3 Users can use commands to move between contexts, just as they use commands to change directories in Unix space. A sample context space is shown in See Context space . 4 Notice that each user's context space is autonomous, so that the same name can be used by more than one user. E.g., User1 and User2 can create separate sub-contexts and call both of them ContextA . Note also that, depending on individual security arrangements, a user can work in another user's context space, just as Unix users can work in each other's directories.
Similarly, just as Unix users can use the same name in different directories to refer to different files, users can assign the same context name to different Legion objects: in See Context space the path names /User1/ContextA/Foo and /User3/ContextC/ContextC1/Foo describe different context paths that can refer to different objects. The two users' context spaces are independent of each other, allowing the users to assign names without fear of conflict.
On the other hand, a single Legion object can have multiple context names in multiple contexts, as shown in See Single object with multiple context names : Foo and Bar refer to a single object, ObjectX, but these names are in two different context spaces.
It is important to remember that context space does not reflect object space: ObjectX remains in its native location, regardless of any context path names users chose to give it. User1 might be in the same building as ObjectX or on the other side of the country. Context space is made up of string names organized by user preference. Objects named in a single context may actually be hundreds of miles apart.
Beyond this user-level grouping, linking object groups in graph- or directory-based structures can be useful: just as a Unix directory tree or a collection of hypertext documents provide a logical means for users to organize information, context space provides a structure for finding and organizing Legion objects. For example, to find a host object running on an IBM SP2, one might enter a context called hosts , then a sub-context called ibm , then a sub-context called sp2 , where a list of host objects running on SP2s could be found (in Unix, this would be the equivalent of moving through a series of directories--/hosts/ibm/sp2--to arrive at a directory called sp2).
Legion provides a library interface and utility commands such as legion_context_create , legion_context_lookup , and legion_ context_list for manipulating and negotiating context spaces. The Legion library interface provides a Unix-like path interface for expressing navigation through multiple contexts, and Legion command-line tools support path-like naming of objects within context space. Paths are delimited by the " / " character. For example, the path /usr/adam would map to the object named adam within the context named usr .
Objects contain in their state a root context (implicitly named "/") that provides each object with a base for absolute path names. The root context's LOID is stored in a variable that can altered by the object at runtime. Objects also contain in their state the current context's LOID, which provides the basis for relative context paths. 5
As in Unix space, users move around context space with command-line utilities, or commands. The remainder of this document discusses the basic tools for manipulating context space. Most of these tools are similar to Unix commands, in the hopes that users will be able to understand and use them more easily. There are a variety of resources available for learning The Reference Manual lists all commands and instructions for using them, and man pages for all commands are included in new Legion systems (ask your system administrator if you are unsure how to use man pages). The Legion web site has on-line tutorials and postscript documents that provide information about context space and legion commands (<http://legion.virginia.edu>). In this manual, an alphabetical list of all commands and their syntax is in See Alphabetical list of Legion commands and a subject list is in See Subject listing of Legion commands .
Just as the ls command in Unix allows users to view the files located in a particular directory, the legion_context_list command shows all objects contained in a particular context. The root context of a new system will look something like this:
To view sub-contexts (similar to sub-directories), the name of a context 6 is used an as argument:
The legion_ls command will also list the contents of a context. It is similar to the legion_context_list command, but it can return more specific information. When used without flags, the command will only list the contexts and file objects contained in the current context.
. 1.01.05.01000000.00000... 7
Users can look up the LOID associated with an object named in a context by using the legion_context_lookup command. The output will be a "dotted hex" LOID. Usage of this command requires including the object's context space name (its context path) as a parameter.
The sub-context tmp is now the current working context. Note that this command sets an environment variable in the shell in which it has been invoked. The command is implemented as an alias (as supported by your shell) that sources the script legion_set_context.sh.
Users can create alternate names for objects with legion_ln , a command similar to the Unix ln command. Note that it works only with objects that already have a context name. If no context name has been assigned, you will need to use a different command (such as legion_context_add : see See Assigning a context name to a LOID ) to assign a name to assign a name to the LOID.
Here the new name Foo is assigned to the BootstrapHost object and placed in the current working context (rather than the hosts context, where the object's original context name BootstrapHost is located).
An object's alternate context names can be in different contexts: BootstrapHost is in the hosts context, and Foo is in the current context. Both names refer to the same LOID and either one can be used to control the object.
The legion_context_add command maps an object's LOID to a context name. It can be used to assign a name to a previously unnamed object or to assign an extra name to a previously named object. Either way you need to provide the object's full LOID as an argument.
The name Foo is automatically removed from the current context. This process affects only a specific context path: any other use of the name Foo , whether somewhere else in this user's or any other user's context space, will be unchanged.
Neither of these commands, as used in these two examples, will affect the associated object: removing a context name does not destroy the object. If you wish to destroy or deactivate the object use the legion_rm command with, respectively, the -destroy or -deactivate flags.
The legion_destroy_object command, on the other hand, destroys the object but not its context name(s). This command should therefore not be used with objects that have context names, unless the names are then removed with either legion_rm or legion_context_remov e .
As discussed in section See Organizing context space , the same context name can be used to refer to different Legion objects. For example, the name Foo can be assigned to two different objects in two different contexts, the current working context and the tmp context.
The contents of a local file can be copied to a new Legion file object with the legion_cp command. The example below copies the contents of a local Unix file, UnixTextFile.txt , to a new Legion file object, called newFileObject , in the current working context.
The contentx of a local directory tree can be copied from local file space into context space with the legion_import_tree command. The contents of Unix binary files will automatically be copied into new Legion file objects. In the example below, the contents of the Unix directory legion.practice will be copied into the current Legion context. A listing of the Unix directory's contents shows that the directory contains one sub-directory and three files. The subdirectory nextlevel also contains three files.
The default values of legion_import_tree are the current Unix directory and the current Legion context (in this case, the home context "."), but users can specify the directory path and/or the context (see the Reference Manual for full usage).
A new system contains a single host and vault, called the bootstrap host and bootstrap vault, but in order to expand the resources available to your system's users will want to add host objects and vault objects to your system.
A vault stores inactive Legion objects: it is a persistent storage space that manages the persistent storage space of inert (i.e., inactive) Legion objects and is the virtual representation of a persistent storage space on a processor. A vault can be in a file system, database system, tape drive, CD-rom, etc.
A host object is a Legion object that represents and can run a physical host or collection of hosts in the Legion system. The host object guards the host's resources, and can activate or deactivate other Legion objects.
A vault object is a Legion object that represents and runs the vault. Like the host object, vault objects guard a vault's resources. They do not, however, activate or deactivate the objects that they manage.
See Simple Legion system , right, shows how this division of labor works in a simple Legion system, Host Object1 representing Host1 and Vault ObjectA representing VaultA.
A new Legion system contains one host, known as the bootstrap host, and one vault, known as the bootstrap vault, as well as one host object and one vault object to manage them. The latter are called the bootstrap host object and bootstrap vault object. Legion automatically assigned these objects context names when the legion_initialize is run during the start-up procedure. Note, however, that while the bootstrap vault is assigned only one context name ( BootstrapVault ), the bootstrap host is assigned two context names ( BootstrapHost and your.bootstrap.host.name ).
To add a new host, the main system must be active. If necessary, start it with the first-time start-up or restart procedure, as appropriate. Note that the new host must also have Legion distribution tree installed, so that it can communicate with your Legion system and properly carry out Legion tasks.
You will be adding the host from your current machine using the legion_starthost command. This command uses remote shell ( rsh ) classes. 8
Normal usage is below (please see section See Adding new hosts for legion_starthost 's flags and default settings).
The legion_starthost command can be used to start new host objects on your current host as well as on other hosts: a single machine can contain more than one host object representing and managing the resources available to it. Note that you can create multiple vault objects in a single vault.
You should specify a compatible vault whenever your create a new host object: the command does not require a vault name in order to create a new host object. There are commands for adding and removing vaults from a host's list of compatible vaults, and you can add vaults to this list after creating the host object, but if possible it is simpler to specify at least one compatible vault when running legion_starthost .
In this sample output, a good deal of information is returned. Legion prints out the attributes of the newly created host object, which include its name, context, path name, vault, and LOID. It also shows the binary executable files for basic Legion objects (e.g., an implementation object) being added and configured to the new host. These files allow the new host to start new Legion objects as necessary.
Note that the output lists the new host-vault pairs that are formed when the object is created. Since the command listed one compatible vault, BootstrapVault , the new vault will only be added to BootstrapHost 's list of compatible vaults and only BootstrapVault will be added to the new host's list of compatible vaults. More than one vault can be listed as compatible when creating a host object.
If you prefer to place the new host object in a different sub-context, or if you would like to assign it a specific context name, you can use the -N flag. The example below creates a host object on the same host as above, but assigns it the context name aNewHost . Note that the argument specifies that the context name be put in the hosts sub-context (otherwise it will be put in your current context).
The procedure of starting a new vault is similar to starting a new host. Usage of the legion_startvault command is below (please see section See Adding new vaults for legion_startvault 's flags and default settings).
The output here is similar to the legion_starthost output, and includes the new object's attributes and LOID. Once again, only one host was listed as compatible, so the new vault is added on to BootstrapHost 's list of compatible vaults and only BootstrapHost is added to the new vault's list of compatible hosts. Note that if you do not specify any compatible hosts the vault's list of compatible hosts will be empty. There are commands for adding and removing hosts from a vault's list of compatible hosts, and you can add hosts to this list after creating the vault, but if possible it is simpler to specify at least one compatible host when running legion_startvault .
To add more than one host to the vault object's compatibility list, just add the names of the host objects. The next example repeats the previous command to create a new vault object called aNewVault that will be compatible with BootstrapHost as well as with aNewHost .
Hosts and vaults are always paired: a host or vault is unusable by itself. Adding new hosts and vaults to your system makes multiple processors and storage space available to your system. However, only compatible hosts and vaults can be paired. See Compatible host-vault pair , right, shows a compatible host-vault pair: the host and vault can "see" each other and can therefore pass objects between them, as the objects are activated and deactivated.
All Legion host objects must be paired with at least one compatible vault object in order to carry out Legion processes. There are several commands (see section See Manipulating host-vault pairing ) for adding, deleting, and viewing lists of acceptable hosts or vaults, but before you add a new host or vault to your system you must consider any possible compatibility problems. This is a critical point, since if a paired host object and vault object cannot communicate with each other, the Legion objects that they control will be unusable. In See Compatible host-vault pair , for instance, Host A can "see" Vault B, HostObjectA manages Host A and VaultObjectB manages space in Vault B. Host A is compatible with Vault B, so HostObjectA can be paired with VaultObjectB.
This is not a concern in systems with a single shared vault (e.g., a networked file system, database system, tape drive, CD-rom, etc.), as in See Common persistent storage system , left. Here, any vault objects will represent space in the only available disk storage space (Vault A) accessible to Host 1, Host 2, and Host 3. If the user on Host 1 wishes to add Host 2 to her system, she can either pair the new host object with the BootstrapVault already in her current system or create a new vault object (on Vault A) to manage the new host object's persistent storage needs. Either way, there is no need to worry about non-compatible pairing.
On the other hand, if this same user wanted to add a new host from a foreign system (i.e., a system with a persistent storage space that is not accessible to her bootstrap host object), she would also need to create a new vault object in the foreign system and pair it with the new host object. See Multi-host and multi-vault system (below) shows an example of this situation, with two different file systems and multiple hosts. Hosts 1-3 can see Vault A, and Hosts 4-6 can see Vault B. Again, if the user wanted to add a host from her own system she could either assign the new host object a currently existing BootstrapVault from Vault A or create a new vault object on Vault A without worrying about host-vault compatibility. But if she wants to add a host from the second system, say Host 4, she must pair it with a compatible vault object on Vault B (either by creating a new vault object with legion_startvault or by using one that has already been created and that she has permission to use).
See Adding new resources to a Legion system (below) shows how this might work. The user's Legion system creates HostObject4 and VaultObjectB , objects that represent and manage the user's Legion processes on Host 4 and Vault B. HostObject4 will manage the user's Legion work on Host 4, and VaultObjectB will manage persistent storage of any objects that HostObject4 creates or uses. Assuming that there are no conflicts in architecture, environment, etc., you can add a new host to your system with the legion_starthost command and the new host's DNS name and a new host object (representing new.host.DNS.name) will be created on the new host using the current values of $LEGION and $LEGION_OPR from your current environment. If the new host has a different architecture or has different Legion environment variables, or if the user wishes to specify a different user id, the user can specify this with legion_starthost flags.
Note that if you must include the context path of the class which will parent the new object: if you wish to create an instance of BasicFileClass you must include the BasicFileClass context path name ( /class/BasicFileClass in the example below). To create the object on a different host or vault, use the -h or -v flags plus the host's or vault's context path. The -H and -V flags will cause the class to create an instance on a host or vault of a specified class (note that users can create their own classes of hosts as is most convenient to organize the types of hosts that they wish to work with). If the -Ch or -Cv flag is used, the class will create an instance of itself on a randomly choosen host or vault from among the hosts or vaults listed in a specified context.
The newly created object's context name, file , will appear in the current context (since we did not specify another context) but, as the -h flag specified, it was placed on aNewHost . This is an important feature of context space: an object's context path does not reflect its actual placement. To find an object's actual location at any time, use the legion_get_host command. The output of this command is the host object's LOID.
There are a group of Legion commands that allow users to control placement of a class's instances or of a specific instance. You can use the legion_class_host_list or legion_class_vault_list commands to control the list of hosts or vaults upon which a given class can place its instances, or the legion_instance_host_list and legion_ instance_vault_list to control where a given object can be placed. There are also command that will display lists of a specified object's acceptable hosts ( legion_get_candidate_hosts ), vaults ( legion_ get_candidate_vaults ), and host-vault pairings ( legion_get_ candidate_placements ). These commands are likely to be most useful for resource scheduling and management. Please see section See Command-line functions of the Reference Manual for information on the usage of these commands, look on the man pages, or the on-line tutorials (at <http://legion.virginia.edu>).
The Legion graphic user interface ( GUI) is a Java application that allows users to run context-related commands in a graphic interface. The Legion GUI uses icons to represent different parts of context space (file objects, sub-contexts, etc.) and allows users to run most context-related commands from within the GUI.
The Context Manager can be run from the command-line of any platform compatible with the Java Development Kit (JDK) 1.1.3. A Windows95 client application, called the Legion Server, allows users to run the Context Manager from Windows95.
The Context Manager can also be run in the background, so that the command line will still be active. The window will look something like See Command-line Context Manager window . Note that you cannot start the ContextMgr process from outside the $LEGION/src/UserInterface directory.
Before you can run the Legion Server, a Windows95 client application, you need to follow the above procedure on a machine running a Legion session. The make command, run in step 3 above, created a file called Legion.zip in your $LEGION/src/UserInterface path. The zip file contains the classes needed to run the Server.
You also need to have either the Java Runtime Environment (JRE) version 1.1.3 or later, or JDK version 1.1.3 or later, installed on your Windows95 machine. You can download these from <http://java.sun.com/products/>.
We recommend that you run this in the background, although it is not necessary. Once this process has started the Legion Server it will output the name of the machine it is running on and a port number to your command line.
The Legion Java client will open in a separate window, titled the Legion Context Manager, and can be manipulated like a normal Windows95 window. The window will look something like See Windows95 Context Manager window .
To restart the command-line Context Manager, repeat step 4 in section See Starting the Context Manager from the command-line . To restart the Windows 95 Legion Context Manager, repeat steps 3 and 4 described in section See Starting the Context Manager in Windows95 . You will get a different port number each time you run the Context Manager.
The GUI can be used in place of context-related commands to manage context space: all context-related objects such as contexts, file objects, and objects are represented by icons and can be manipulated with the right and left mouse buttons.
Import a copy of a local file into Legion space. A separate window will appear, with the option to browse through local file space and specify a filter to use when copying the file into Legion space.
There are five different icons in the GUI, each representing an element of context space. The context name associated with each icon will appear beneath the icon. Objects that do not have a context name will not be represented in the GUI. Clicking the right or left mouse button on icons will allow users to carry out various actions, explained below.
The context icon represents a context, similar to a Unix directory. You can change your current context by double-clicking the desired context icon with the left mouse button. Click the right mouse button to get the following options:
Destroy the selected context (i.e., remove its name and LOID from Legion) and destroy any context names kept in the selected context. Note, however, that this will not destroy the objects named in that context, nor will it remove any other context names from other contexts.
The parent context icon represents the current context's parent context. Double-click the left mouse button on this icon to change your current context to the parent context. Clicking the right mouse button will produce the same options as with the context icon (above).
Display all instances of a selected class in a separate window. See See Context Manager listing a class's instances , below, for more information about this option.
Executes a previously registered executable program class. Please see See Executing remote programs for information about registering remote programs, and See Running a remote program from the GUI for information about running remote programs in the GUI. This option will produce a separate window, which can be used to specify input and output file names. The Ctrl-p key combination will also run this process.
The file object icon represents a file object. Double-click the left mouse button on this icon to open a text editor window displaying the file's contents. The text editor allows users to edit file objects. Click the right mouse button to get the following options:
Destroy the selected object (i.e., remove its name and LOID from Legion). A "Destroy" window will appear, where you should specify whether you want to destroy just the object's context name or the object (both its context name and its LOID).
In all contexts, users can refresh the current context by clicking on the upper left hand "." current context icon ( See Switching contexts in the Context Manager a). Similarly, users can always get to a sub-context's parent context by double-clicking on the ".." parent context icon, to the right of the "." icon ( See Switching contexts in the Context Manager b). Note that the parent context icon will not appear if you are currently in your home context.
Note that changes to context space made from the command line are not automatically updated in the GUI. Users should refresh their current context as necessary. Changes made in the GUI, however, are reflected in the context space and can be viewed with command utilities (i.e., legion_ls or legion_context_list ).
Create a new File Object. A New File window will open, where you can enter the new object's context path name, then a text editor window will open. The Ctrl-n key combination will also run this process.
Import a copy of a local file into Legion space. A separate window will appear, with the option to browse through local file space and specify a filter to use when copying the file into Legion space. The Ctrl-i key combination will also run this process.
Executes a previously registered executable program class. Please see See Executing remote programs for information about registering remote programs, and See Running a remote program from the GUI for information about running remote programs in the GUI. This option will produce a separate window, which can be used to specify input and output file names. The Ctrl-p key combination will also run this process.
Display all instances of a selected class in a separate window. The "Show Instances" option returns several pieces of information about the selected class's instances, as shown in See Context Manager listing a class's instances (below): it lists each instance's LOID, status, Object Address (OA: an active object's location), Host, and Vault. The "Instance" pull-down menu allows users to activate, deactivate, and destroy selected or all instances, as well as to refresh the window. The Ctrl-p key combination will also run this process.
Destroy a selected object's path name and/or the object itself. A "Destroy" window will appear, where you should specify whether you want to destroy just the object's context name or the object (both its context name and its LOID). The Ctrl-Delete key combination will also run this process.
The mechanics of running a Legion application will, of course, vary widely from one application to another. Generally, however, users will need to compile, register, and run individual applications with commands written specifically for that purpose. For example, MPI applications use the legion_mpi_register and legion_mpi_run commands, and PVM applications use the legion_pvm_register command. Some commands may require users to select a group of hosts that the application should use before running the application, others may just ask how many hosts should be used.
Users can arrange parts of their context space around the needs of specific applications or to organize certain resources (a sub-context of names of hosts that can be used for a specific MPI application, for instance).
Information about Legion's MPI and PVM libraries can be found in sections See PVM and See MPI . Information about Legion's Basic Fortran Support can be found in the Legion Developer Manual (section See Fortran ).
The current Legion implementation supports a core PVM interface See A. Geist, A Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V.S. Sunderam, PVM: Parallel Virtual Machine, MIT Press, 1994. See V.S. Sunderam, PVM: A framework for parallel distributed computing, Concurrency: Practice and Experience, vol. 2(4), pp. 315-339, Dec. 1990. , which includes functions for data marshaling, message passing, and task creation and control. Legion supports legacy PVM codes, and provides an enhanced PVM environment using Legion features such as security and placement services. For example, PVM applications can be run with encrypted message passing on Legion's PVM implementation.
A link-in replacement PVM library uses the primitive services provided by Legion, to support the PVM interface. For example, PVM tasks map directly to Legion objects: thus pvm_spawn() uses Legion.CreatObject() , and PVM task ids are mapped to Legion LOIDs. PVM Buffers are implemented by using Legion Buffers (the fundamental data container in the Legion runtime library)-- pvm_pmkint() thus uses LegionBuffer:: put_int() . Similarly, PVM message-passing maps to Legion method invocation: the PVM-Legion library exports a new method, LegionPVMSend() . Sending a message to a task maps to calling the LegionPVMSend() method on the object corresponding to that task. A LegionBuffer containing the message becomes the parameter to the method.
Whereas PVM tasks are identified by integer task identifiers ( Tids), Legion objects are identified by LOIDs, which come in varying sizes but are generally much larger than an int. The Legion-PVM library must therefore provide a mapping between Tids and LOIDs. Legion's contexts are used to manage this mapping, and the Legion library caches the mappings that have already been looked up. Since Tid-to-LOID mappings are invariant, caching can be performed without any consistency management costs.
Whereas PVM spawn requires a "task name" parameter to identify the type of task to spawn, Legion uses class LOIDs to specify an object's type. The Legion-PVM library utilizes a single class object for each type of task: therefore, Legion-PVM tasks are instances of Legion classes. Mapping between task names (i.e., as specified to pvm_spawn() ) and class LOIDs is managed in Legion context space.
Legion classes have associated "implementation objects," which provide their instances with binary implementations. Because of this, PVM-Legion does not spawn tasks stored in a "well-known" directory location, as standard PVM does. A mechanism is provided for "registering" task implementations with the appropriate classes. Once executables are registered, the user does not need to explicitly manage the copying of binaries between hosts on disjoint file systems, since Legion automatically manages this, unlike standard PVM.
Task object code files can be compiled as before. Link against libLegionPVM.3, and basic Legion libraries (the final linkage must be performed using a C++ compiler, or using C++ appropriate flags for ld , since the Legion library has C++ link-time requirements).
A sample Legion PVM makefile is shown in See Sample Legion PVM make file .
To register the example task with the Legion/PVM system, use the legion_pvm_register command. The first time this is run Legion will create /pvm and /pvm/tasks directories. Your output will look something like this:
The current Legion implementation supports a core MPI interface, which includes messages passing, data marshaling and heterogeneous conversion. Legion supports legacy MPI codes, and provides an enhanced MPI environment using Legion features such as security and placement services.
MPI implementations generally require that MPI executables reside in a given place on disk. Legion's implementation objects serve a similar role. We have provided a tool, legion_mpi_register , which registers executables of different architectures for use with Legion MPI.
A sample Legion MPI makefile is shown in See Sample Legion MPI make file .
Note that if the -f flag is used the file named in < file name > will be read first, and the -n flag and program name (if it is included) will be ignored. The legion_mpi_run utility will expect one command per line in the file: each line can contain any of the legion_mpi_run flags (except the -f flag). The MPI application named in the legion_mpi_run command will then be started, with the given flags. Note that when multiple MPI programs are run with a legion_mpi_run command, via the -f flag, any other flags used (e.g., -h to specify a set of hosts) will be applied to all of the programs run unless otherwise directed by the program itself.
Note that the < command > argument here is the Legion context space name for the class created by legion_mpi_register in the previous section. In this example the default path mpi/programs is used with the class name vdelay .
This context will have an entry for each object in this MPI application. If you run multiple versions of the application simultaneously, use the -p flag with legion_mpi_run to specify an alternate context in which to put the list of objects.
You must then register the program with the legion_mpi_register command. The sample output below shows Legion creating three new contexts ( mpi , programs , and instances , the last two being sub-contexts of mpi ), creating and tagging a task class, and registering the implementation of the mpitest_c program.
In order to view the output of the program, you will need to create and set a tty object. The example below creates a tty object, sets it as an environmental variable, and then directs its output to the background. (For more information on using these commands, please see the Legion Reference Manual.)
Note that this sequence adds the context name mytty in your current context. Users might actually find it more convenient to create a context to hold tty objects, to help organize their context space.
Having done all of this, you can now run the program, with the legion_mpi_run command to run the program. You must specify how many processes the program should use to run the program with the -n flag.
If you do not specify which hosts the program should run its processes on, Legion will randomly choose the hosts from the host objects that are in your system. Here it runs three processes on three different hosts.
As noted above, Legion may not choose the same number of hosts objects as processes specified in the -n parameter. If you specify three processes, Legion will randomly choose to run them on one, two, or three hosts.
If you wish to specify which hosts are used to run your processor, use the -h flag. In the example below, we will run mpitest_c on four hosts, which we will assume are already part of the system (see See Expanding the system's resources in the System Administrator Manual for information about adding hosts to a Legion system) and which, in this example, are named in a single context called mpi-hosts .
We can now assign new context names to the four hosts that we wish to use for this program, using the legion_ln command. These new names map to the host objects' LOIDs, and provide a convenient way to specify different hosts for different MPI programs or to execute the same program on different groups of hosts.
Note also that the -n parameter is still used, even though there are only four host objects named in the mpi-hosts context. The number of processes and number of host object do not have to match: if a user wanted to run only two processes on the hosts named in mpi-hosts , he or she could use -n 2 . The program would then run on the first two hosts listed in the mpi-hosts context. 9
Output from a Legion MPI object to stdout or Fortran unit 6 is automatically directed to a tty object. Note, though, that this output will be mixed with other output in your tty object, and you may wish to segregate MPI output. Legion MPI supports a special set of library routines to print on the Legion tty, which is created using the legion_set_tty and legion_tty_watch commands. This will cause the MPI object's output to be printed in segregated blocks.
There are a few limitations in legion_mpi_debug : If an MPI object doesn't respond, it will hang, and it won't go on to query additional objects. An MPI object can only respond when it enters the MPI library; if it is in an endless computational loop, it will never reply.
Legion's remote execution allows users to run remotely executed programs from the Legion command line, taking advantage of Legion's distributed resources. The user's workload can be spread out on several processors, improving job turn-around time and performance. Users can execute multiple instances of a single program or execute several different programs in parallel.
A remote program is an executable process that can be started with a command-line utility. Note that a remote program is not part of the Legion system, but is an independent program that may or may not be compatible with Legion. Remote programs might be shell scripts, compilers, existing executables, etc. The term "remote" here means that the program does not need to be executed on the local host.
Legion can run both Legion-compatible and non-Legion-compatible programs. A compatible program is linked to the Legion libraries. A non-compatible program does not have these links. The legion_register_program command, which registers non-compatible programs with the Legion system, creates a proxy to handle a particular program. When the program is run, with the legion_run command, the proxy is activated and can instruct the program to begin execution.
Legion can run both Legion and non-Legion (i.e., programs that are not linked to the Legion libraries, such as shell scripts) remote program since it relies on two different types of interfaces to pass Legion's execution order to the program. Before either a compatible or non-compatible remote program can be run from Legion it must be registered . The legion_register_runnable command registers Legion-compatible programs, and the legion_register_program command registers a non-Legion program. A single command, legion_run , runs both types of remote programs. Both of these commands have parameters that include the remote program's binary path name and architecture and assign the program a context path name: this allows Legion to know where the program's executable files are located, what platform the files use, and how to find the program's Legion interface.
Non-compatible remote programs are registered with the legion_register_program command. The user includes information on the program's architecture (e.g., linux, sgi, solaris, etc.), its binary path name, and a program class.
The < program class > parameter is a context path name for the class that will manage Legion objects related to a particular program. Note that since the user decides how to organize his or her own context space, the path name can be whatever the user finds most convenient (although you might want to use the name of the program in the < program class > parameter, so that it will be easier to remember). The context path name can be new or can use a previously created path (if the user is registering multiple versions of a the same program), and will refer to a (new or previously created) Legion object that functions as a proxy for executing a non-compatible remote program.
Note also that if multiple programs are registered with the same program class and architecture, the most recently registered version will be used when the program is run on that particular architecture. Once the program has been registered, it can be run with the legion_run command.
The legion_register_runnable command registers programs that are linked to the Legion libraries, and export a runnable object interface. This is, the command creates an interface between the Legion system and the remote program and this interface transfers the execution command from Legion to the remote program. Like legion_register_program , this command specifies the remote program's program class, executable binary path, and architecture. Its usage is:
As with legion_register_program , the < program class > parameter is a context path name for the Legion objects that will handle a particular program. And, again, the user is free to choose a context path that best suits his or her organizational scheme, or use a previously created context (although you might want to use the name of the program in the < program class > parameter, so that it will be easier to remember). In that case, note that Legion will use the most recently registered program class and architecture when running a remote program. Once the program has been registered, it can be run with the legion_run command.
Use legion_run to remotely execute programs. but note that you must first register the program with either legion_register_program for non-Legion compatible programs or legion_register_runnable for Legion compatible programs. The usage of this command is:
The legion_run command executes a single instance of the program associated with the < program class > parameter. Legion will randomly choose a host with a suitable architecture (specified when the program was registered) to execute the program.
There are several optional parameters associated with this command. The -help flag displays the command's syntax and provides a brief description of how to use its options. The -w flag directs the command's output to your tty object. Note that if you have not created or set a tty object for your current window, you will not be able to see the command's output and an error message will appear. You can use the legion_create_object command to create a tty object, and the legion_set_tty command to set the tty as an environmental variable:
The -in < context path name > option allows you to select a Legion file that should be copied into the remote program's current working directory before the program executes. The -out < context path name > option instructs Legion to copy the program's output from the remote program's current working directory to the Legion file named in < context path name >, after the program terminates (partial results will be available if the program crashes).
The other two options operate on a local file: -IN < local file name > tells Legion to copy the files named in < local file name > into the remote program's current working directory before the program executes, and -OUT < local file name > tells Legion to copy the program's output to the file named in < local file name >.
Files that are listed with the -in / -out and -IN / -OUT options can be named with either full or relative path names. Note, however, that the remote program will use the file path's basename (i.e., as relative path names from the current working directory) to gain access to the named file. New files will be given names corresponding to the basename of the current Legion context path or local directory path.
If your program requires extensive use of these options you may want to use the -f flag, which allows user to list the options in a separate file rather than on the command line. The file can contain any of the options discussed above and can be delimited with spaces or new lines. 10
The remote Unix program cp is first registered, and the program class " cp " is created in the user's local context, /bin/cp is declared to be its local executable path name, and linux is declared to be its architecture type.
The program is then remotely executed on a randomly chosen linux host, and a series of events takes place: first, the -in option tells Legion to copy the Legion file /dir1/src into the remote program's cp 's current working directory and call it src . The remote program cp then executes, copying src to a new Unix file, dest . Finally, once cp has terminated, Legion copies the contents of Unix file dest to a new Legion file called dest , and adds the context name dest to /dir2 (i.e., if the user runs legion_context_list , dest will now be listed and will refer to the dest file).
Once a program has been registered, it can be run from the Legion GUI (please see See Graphic user interface for information about starting and running the GUI).
This will bring up a new window, which users should use to specify arbitrary command-line arguments for running the program and the context names of any input or output files (the -in / -out parameters). Users can use currently existing output files or create new ones. Users can also specify the program's architecture from a list of acceptable architectures at the bottom of the Run window: this list contains the names of the architectures that Legion currently accepts. The any option uses the default architecture.
Full documentation of these commands can be found in the Reference Manual, in See Command-line functions . There are also man pages for all Legion commands included in the system release package, and on-line tutorials on the Legion web site (<http://legion.virginia.edu>).
Full documentation of these commands can be found in the Reference Manual, in See Command-line functions , and on the Legion web site at <http://legion.virginia.edu/documentation/tutorials/commands_all.html>. There are also man pages for all Legion commands included in the system release package.
An OPA (Object Persistent representation Address) is used to track a specific OPR (Object Persistent Representation, the inert state of an object): in this case, the OPA will represent the persistent state of the new host object. The binary path is the binary path of the program that will be run on the new host in order to start the new host object. Currently, this path can be altered only by editing the legion_start source file, in the $LEGION/src/CoreObjects/ClassObjects/rshStartClassObject directory.
These flags and defaults need to be carefully considered when adding new hosts and vaults to your system. If the new host has a different architecture or a different directory structure, use the -A , -L , or -O flags to specify these parameters. The -U flag allows you to specify a Unix user id for the new object, so that a system administrator can add host objects to another user on the same Unix system. This can be useful should you wish to create a guest user id that has limited access privileges to the new host. The -C flag allows you to start an instance of a different class, so that users can create new host classes and have more flexibility in managing their resources. (Legion currently comes with only one host class, UnixHostClass, but users can add more host classes as necessary, either by creating instances of the UnixHostClass or by writing new classes.)
If the new host object's architecture is not the same as the current host object's architecture, you should run the legion_init_arch tool in order to create implementation objects to match the new architecture (see section 9.0 of the System Administrator Manual, for information about implementations, and section 9.2 of the System Administrator Manual, for information about running legion_init_arch ).
This commands creates a new vault object on the storage system of a specified host (named in < host name >). The flags are similar to the legion_starthost flags. The -L , -O , or -A flags can be used to specify a different architecture or a different directory structure. The -N flag allows you to specify a context name. The -U flag allows you to specify a Unix user id for the new object, so that a system administrator (or anyone with root privileges) can add vault objects to another user on the same Unix system. This can be useful should you wish to create a guest user id that has limited access privileges to the new host. The -C flag allows you to start an instance of a different class. That is, if you do not use this flag, the new host object will be an instance of class UnixVaultClass. This flag allows users to create new vault classes, so as to give users more flexibility in managing their resources.
The legion_host_vault_list command will list a host object's list of compatible vaults. The example below lists the compatible vaults for the BootstrapHost. Note the -p flag, which signals that the list should be printed out.
Both of these commands can add and delete hosts or vaults from the list with specific flags ( -a and -d , respectively). For example, to remove aNewVault from BootstrapHost 's list of acceptable vaults, you would enter the following:
In the future, a FAQ page listing common Legion problems will be posted on the Legion web site, but for the time being please contact us at one of the addresses listed below if you have trouble with the system. Please be sure to include any relevant information about the state of your system before and during the problem.
1. It is actually not always necessary to have a Legion system running in order to use Legion: some Legion hosts can run in "consumer mode." Consumer mode hosts do not require the full set of Legion system binaries to be installed and running; a subset of binaries can access a Legion system that runs on different remote hosts, and can potentially even use that system to execute parts of Legion programs. A consumer mode host cannot itself be used to carry out parts of Legion programs unless those programs are started directly on that host by mechanisms outside of Legion (for example, from a shell running on the host's operating system).
2. Legion uses a the Unix-style slash ("/") in the default context space path names, so the examples here will follow that style. However, it is possible to change this to the DOS-style backslash ("\") via the primitive context manipulation routines provided by the Legion library.
5. It is important to note that this default path service provided by the Legion library is completely replaceable. For example, if a user wished to manipulate context spaces via DOS-style "\" delimited path names, the service could easily be implemented using the primitive context manipulation routines provided by the Legion library.
8. An alternative procedure, using command-line utilities, is explained in the Reference Manual on See Starting extra hosts and vaults from the command line . If possible, we recommend using the rsh procedure explained here, since it is faster and easier.