Legion 1.2

Basic User Manual

The Legion Research Group

Department of Computer Science

University of Virginia

Charlottesville, VA 22903

legion@virginia.edu

http://legion.virginia.edu/

Introduction

About Legion 6

Getting started

Before you start 7
  1. Preparing the Legion environment 7
  2. Logging in to a Legion system 7
What am I looking at? 10
Context space 12
  1. What is context space? 12
  2. How context space works 12
  3. About the LOID 12
  4. Organizing context space 13
Working in context space 15
  1. View a context's contents 15
  2. Looking up an object's LOID 17
  3. Create a new context 18
  4. Changing the working context 18
  5. Assigning alternate names to existing contexts 18
  6. Assigning a context name to a LOID 19
  7. Renaming an object 19
  8. Removing names 19
  9. Using the same name in different contexts 19
  10. Copying a local file to a Legion file object 20
  11. Copying a Legion file object 20
  12. Viewing a file object's contents 20
  13. Importing a local Unix directory tree 20
Host and vault objects 22
  1. What is a host object and vault object? 22
  2. About the bootstrap host and vault 22
  3. Adding a new host 23
  4. Adding a new vault 25
  5. About host-vault pairs 27
  6. Creating objects on new hosts 29
  7. Controlling instance placement on hosts and vaults 29
Graphic user interface 31
  1. Starting the Context Manager from the command-line 31
  2. Running the GUI on Windows95 32
  3. Shutting down the Context Manager 33
  4. Restarting the Context Manager 33
  5. Using the Legion GUI 33
Running a Legion application 39
PVM 40
  1. Core PVM interface 40
  2. Tids & LOIDs 40
  3. Task classes 40
  4. Using PVM-Legion 41
  5. Installation 41
  6. Compilation 41
  7. Registering compiled tasks 41
  8. Examples 42
MPI 43
  1. Task classes 43
  2. Using Legion MPI 43
  3. Example 45
  4. Scheduling MPI processes 46
  5. Debugging support 48
  6. Functions supported 48
Executing remote programs 49
  1. Introduction 49
  2. Registering non-Legion-compatible programs 49
  3. Registering Legion-compatible programs 50
  4. Running a remote program 51
  5. Converting a C/C++ program 52
  6. Summary 52
  7. Running a remote program from the GUI 53

Appendices

Alphabetical list of Legion commands 54
Subject listing of Legion commands 59
  1. Calls on Objects 59
  2. Calls on Class Objects 59
  3. Calls on LegionClass 60
  4. Calls on File and Context Objects 60
  5. Start-Up and Shutdown Functions 61
  6. Scheduling Support 61
  7. General Functions About the State of the System 62
  8. Security 62
  9. Application Development 63
  10. Program Support 63
More information on adding host and vault objects 64
  1. Adding new hosts 64
  2. Adding new vaults 65
  3. Manipulating host-vault pairing 65

Getting help 67

References 68

Index 70

Introduction

This manual is an introduction to Legion 1.2 and is written for users with little or no previous experience with Legion. It explains the purpose, history, and design of Legion and gives a high-level view of Legion context space and an introduction to navigating and manipulating contexts. There are also instructions for logging in to an active Legion system. Users who need to install a new system or to start-up a previously installed system should consult the System Administrator Manual ( See Downloading Legion 1.2 ). If you need more help please contact us ( See Getting help ).

There are three other Legion manuals, each aimed at a specific user, that can be consulted for more information. The System Administrator Manual provides information and documentation for administrators of Legion systems, including installing and running a Legion system, configuring security features, and managing resources. The Developer Manual provides information and documentation for programmers working in Legion and includes information about languages, libraries, core objects, and implementing new objects. The Reference Manual is a more specialized resource, providing detailed information about specific elements of the Legion system. There are also man pages for all Legion commands, included with the system files, and several on-line tutorials on the Legion web site (http://legion.virginia.edu).

About Legion

Developed at the University of Virginia, Legion is an integrated software system for distributed parallel computation. While fully supporting existing codes written in MPI and PVM, Legion provides features and services that allow users to take advantage of much larger, more complex resource pools. With Legion, for example, a user can easily run a computation on a supercomputer at a national center while dynamically visualizing the results on a local machine. As another example, Legion makes it trivial to schedule and run a large parameter space study on several workstation farms simultaneously. Legion permits computational scientists to use cycles wherever they are, allowing bigger jobs to run in shorter times through higher degrees of parallelization.

Key capabilities include the following:

  • Legion eliminates the need to move and install binaries manually on multiple platforms. After Legion schedules a set of tasks over multiple remote machines, it automatically transfers the appropriate binaries to each host. A single job can run on multiple heterogeneous architectures simultaneously; Legion will ensure that the right binaries go to each, and that it only schedules onto architectures for which it has binaries.
  • Legion provides a virtual file system that spans all the machines in a Legion system. Input and output files can be seen by all the parts of a computation, even when the computation is split over multiple machines that don't share a common file system. Different users can also use the virtual file system to collaborate, sharing data files and even accessing the same running computations.
  • Legion's object-based architecture dramatically simplifies building add-on tools for tasks such as visualization, application steering, load monitoring, and job migration.
  • Legion provides optional privacy and integrity of communications for applications distributed over public networks. Multiple users in a Legion system are protected from one another.

These features also make Legion attractive to administrators looking for ways to increase and simplify the use of shared high-performance machines. The Legion implementation emphasizes extensibility, and multiple policies for resource use can be embedded in a single Legion system that spans multiple resources or even administrative domains.

Getting started

Before you start

This manual assumes that you are working on a previously installed, compiled, and running system. 1 (See the System Administrator Manual for information on installing, compiling, and starting a new Legion system.) Before going any further, be sure that the system is properly installed and running.

Please note that while your path will change as you move through Legion's binary files, it will not change as you move through context space. We are currently developing a graphic user interface to help users negotiate this space more easily. For the time being, though, you will be working from your command line and typing in commands to run programs and manipulate contexts.

The next few sections deal with setting up the proper environment variables and logging in to a Legion system as a Legion user.

Preparing the Legion environment

Depending on how your system is set up, you may need to set up your access to your system before you can run Legion commands. This will probably involve running a script such as this:

$ . ~LEGION/setup.sh

or

$ source ~LEGION/setup.csh

The exact syntax will depend on what kind of shell you are using and on where your Legion files are installed. Consult your system administrator for more information.

Logging in to a Legion system

A user who logs in to a Legion system is identified by a special object called the AuthenticationObject : it contains the user's password, initial implicit parameters (the Legion equivalent of a the Unix "environment"), and other information. AuthenticationObjects are created when a user creates a user id. A set of Legion commands ( legion_set_acl , legion_get_acl , legion_passwd , legion_set_implicit_params , and legion_get_ implicit_params ) can be used to retrieve or change this information (see section See Security in the Legion Reference Manual or the man pages for a discussion of how to use these commands). When an authenticated user runs a Legion process a certificate confirming his or her identify is passed along to verify that the user has permission to run the process. This certificate is created and signed by the user's AuthenticationObject.

AuthenticationObjects must be permanent in order to be useful. If an AuthenticationObject is destroyed, its associated LOID, which identifies the user to the rest of the system, is lost. There is no way to generate an identical LOID for a new AuthenticationObject.

Logging in as a user

In order to log in to the system, you will need a user id, which can be created by either the user or system administrator. You can then log in, with the legion_login command. The system will then request your password and check it with your AuthenticationObject, which verifies your identity and security privileges.

There are two different ways to use this command.

  1. You can run the command with just the user id or with no arguments at all. The legion_login process will put you in a new sub-shell, and will continue to run. Any processes that you start from within this shell will be accompanied by a copy of your AuthenticationObject's certificate. This method is explained in See Log in as a new user .
  2. Or, the legion_login command can set the necessary environment variables so that, instead of running a shell, you can tell legion_login to execute a specific command (similar to the behavior of Unix rsh ). The syntax for this is:

legion_login [-l userloid | user_id] [-e command]

  1. You can add another Legion command as an argument and the legion_login process will securely obtain your certificate from your AuthenticationObject and pass it along. This is useful if you need to certify your identity for a specific process and you do not wish to operate in a legion_login sub-shell.

Note that even if your system doesn't require user ids to gain access to Legion, using legion_login has a side benefit in that command-line tools run faster in a user id shell.

Create a new user ID

If you do not already have a user id, you can create one. Normally, the system administrator creates user ids and simultaneously enters them in the groups that have appropriate security rights on the system. If you create your own id you may not have the necessary permissions to enter the id in such groups and use the system resources that the groups control. However, your new user id will otherwise function normally and can be used to protect your resources from other users of the system.

A user name is simply an entry in context space for an AuthenticationObject. You can create users with the legion_create_user command. To create user "bob" in the current context, enter:

$ legion_create_user bob

To create the bob entry elsewhere, enter:

$ legion_create_user /my/path/bob

The command will prompt for a password for the new user. Note that the context where the user name is placed has nothing to do with whether that user can access anything in that context space.

Log in as a new user

To log in as the new user "bob," enter

$ legion_login bob

Password: xxxx

$

You do not need to include the user id as a parameter, if you prefer:

$ legion_login

Legion login: bob

Password: xxxx

$

Note that your working context will not change when you enter the new shell, but your access privileges are different. Objects created in bob 's shell will by default require bob 's access privileges to use them.

To exit from bob 's shell, enter exit :

$ exit

exit

$

While logged in, you can change your password and other parameters of your environment. The password may be changed with legion_passwd . Note that in this release you must specify the user whose password you are changing, even if it is yourself:

$ legion_passwd bob

New Legion password: xxxx

Retype new password: xxxx

$

To change your implicit parameters, which can be used to control the behavior of security as well as other Legion tools and objects, use the legion_set_implicit_params tools documented in the reference manual. The access permissions of existing objects can be changed with legion_set_acl (please see also See Using security features , in the System Administrator Manual).

What am I looking at?

Legion is an object-based system organized by classes and metaclasses (classes of classes). Every part of Legion, including classes and metaclasses, is represented by a Legion object and users manipulate various elements by controlling their representative objects. All Legion objects are automatically assigned an identifying number, called a LOID (see section See About the LOID below), and can also be assigned a user-chosen string name, called a context name . Users can organize these context names into a discrete context space, and from that space can run programs, use far-flung resources, and take advantage of Legion's other features.

Legion users will probably spend a great deal of time working in context space, so it is important to understand the initial geography of your system's context space and how to negotiate that space. A typical newly installed system has the basic elements of context space: there are four sub-contexts and several starter context names for commonly used objects. See Contents of a typical new Legion system context space , below, shows the contents a newly installed Legion system's context space. Note that your system may differ, since a system administrator may have already customized your space to take advantage of your system's resources.

To see the contents of your context space, you can use either the legion_context_list or the legion_ls command. (See section See Command-line functions in the Legion Reference Manual for a list of all Legion commands' usage.) The full context path name for BootstrapHost in the system shown in See Contents of a typical new Legion system context space would be /hosts/BootstrapHost . 2

For more information on viewing context space and using the other context-related commands, please see See Working in context space . A new Legion system includes four sub-contexts, called class , hosts , vaults , and home . These in turn contain context names that refer to the initial contents of a new system: the class sub-context contains context names of particular classes of Legion objects, the hosts sub-context contains names of the host objects currently in your system, and the vaults sub-context contains names of the vault object currently in your system. The home sub-context is initially empty.

When newly installed, hosts contains two names, BootstrapHost and the DNS name of the bootstrap host. (A host is a machine in a Legion system. A vault is a file storage system used to store inactive Legion objects. A host object represents and manages a given host in a Legion system, and a vault object represents and manages a given vault in a Legion system.) Both of these names point to the bootstrap host object, which represents the bootstrap host. On the other hand, vaults contains only one name, BootstrapVault , which points to the bootstrap vault object. (See See Host and vault objects for more information on hosts and vaults.)

A newly started class sub-context contains context names for all of the basic Legion object classes, such as BasicFileClass and ttyObjectClass . Note that these classes are themselves objects, even though they are responsible for managing a given set of objects. All Legion objects are organized into classes, and all classes are organized into a huarache of metaclasses , with LegionClass as the supreme metaclass. Note also that the context space illustrated in See Contents of a typical new Legion system context space does not reflect this hierarchy. This is a key feature of Legion: the physical location and organization of Legion objects is unrelated to any user's context space. The objects named in a host sub-context do not even have to know of each other's existence or even be able to communicate with each other. They do not even have to be hosts: the user can create, delete, and use contexts as he or she chooses.

There are also several binary files which are included with Legion. These files contain programs for running the commands utilities, library functions, and various programs.

Context space
What is context space?

Context space is a way of organizing Legion space. It is similar to Unix's system of directories and files but while Unix space is location specific, context space can be arranged to suit the user's convenience. Contexts are string names assigned to Legion objects. The user can assign an object a name (or multiple names) without worrying about the object's physical location. Context space is organized into a series of sub-contexts, similar to Unix directories, and can be assigned mnemonic or logical names as suits the user. A single sub-context can contain context names that refer to objects located on a single machine or separated by large distances.

How context space works

All Legion objects are automatically assigned a unique system-level identifying name, called a LOID (Legion Object Identifier). The LOID allows the system to find objects scattered on several different machines. Since a LOID consists of several lines of numbers, it can be awkward at best for a user to track objects via their LOIDs. Users can instead assign a string name to an object's LOID, making it easier to find and use any object in Legion. This name, called a context name, resides only in a user's individual context space and refers only to a particular LOID. Context names are grouped into contexts and these contexts are organized into a hierarchy similar to a Unix directory structure.

Each user has a personal context space, which she organizes and maintains to suit her own needs. Objects are named according to each user's preferences, and each user's context space is independent of other users' context spaces, allowing different users to assign multiple context names to a single object without interfering with each other or with the object. This is discussed further in section See Organizing context space , but the next section provides a brief look at the anatomy of the LOID.

About the LOID

A LOID is a variable length binary structure (typically more than 128 bytes), which contains a type field followed by several variable size fields. The first variable field indicates the Legion domain in which the LOID was created, the second is the class identifier, the third is the instance number, and the fourth is a public key. The type, domain, and class identifier fields must be present. A LOID may be appended with any number of other fields, each of whose size and meaning are dependent on the LOID type. LOIDs are represented as strings, using standard "dotted hex" notation. For example, a typical LOID would look something like this:

1.01.07.01.000001fc0d24ab3512f63533f3f96b2bba0572aed8d47efec7c24618fca3a3d49326fa84eaa320724c10264f225c97820aced74586194974256ee86c2ae0565d55c8e623

In this sample the type field, 1 , indicates the LOID's type; the first variable field, 01 , indicates the domain in which the object was created; the second variable field, 07 , indicates the class that the object belongs to; the third variable field, 01 , indicates the object's instance number; and the fourth variable field, 000001fc0d24ab ... , is the object's RSA public key. This notation is obviously too cumbersome for most users to manipulate as strings, so the naming service provided by context spaces allows users to deal with objects using meaningful string names chosen by the user. This separation of concerns between system-level names and user-level names allows Legion LOIDs to contain system-level information, such as an object's public key, without polluting the user-level name-space.

Organizing context space

All contexts in Legion are organized into a Unix-like hierarchy, with a root context that contains a set of other contexts. This structure provides each object with an absolute path name. Just as a Unix user organizes files in a series of directories, a Legion user organizes object in a series of contexts: a Legion system has a root context, and within that space each user has a personal root context, a home context in which he or she works. 3 Users can use commands to move between contexts, just as they use commands to change directories in Unix space. A sample context space is shown in See Context space . 4 Notice that each user's context space is autonomous, so that the same name can be used by more than one user. E.g., User1 and User2 can create separate sub-contexts and call both of them ContextA . Note also that, depending on individual security arrangements, a user can work in another user's context space, just as Unix users can work in each other's directories.

Similarly, just as Unix users can use the same name in different directories to refer to different files, users can assign the same context name to different Legion objects: in See Context space the path names /User1/ContextA/Foo and /User3/ContextC/ContextC1/Foo describe different context paths that can refer to different objects. The two users' context spaces are independent of each other, allowing the users to assign names without fear of conflict.

On the other hand, a single Legion object can have multiple context names in multiple contexts, as shown in See Single object with multiple context names : Foo and Bar refer to a single object, ObjectX, but these names are in two different context spaces.

It is important to remember that context space does not reflect object space: ObjectX remains in its native location, regardless of any context path names users chose to give it. User1 might be in the same building as ObjectX or on the other side of the country. Context space is made up of string names organized by user preference. Objects named in a single context may actually be hundreds of miles apart.

Beyond this user-level grouping, linking object groups in graph- or directory-based structures can be useful: just as a Unix directory tree or a collection of hypertext documents provide a logical means for users to organize information, context space provides a structure for finding and organizing Legion objects. For example, to find a host object running on an IBM SP2, one might enter a context called hosts , then a sub-context called ibm , then a sub-context called sp2 , where a list of host objects running on SP2s could be found (in Unix, this would be the equivalent of moving through a series of directories--/hosts/ibm/sp2--to arrive at a directory called sp2).

Legion provides a library interface and utility commands such as legion_context_create , legion_context_lookup , and legion_ context_list for manipulating and negotiating context spaces. The Legion library interface provides a Unix-like path interface for expressing navigation through multiple contexts, and Legion command-line tools support path-like naming of objects within context space. Paths are delimited by the " / " character. For example, the path /usr/adam would map to the object named adam within the context named usr .

Objects contain in their state a root context (implicitly named "/") that provides each object with a base for absolute path names. The root context's LOID is stored in a variable that can altered by the object at runtime. Objects also contain in their state the current context's LOID, which provides the basis for relative context paths. 5

Working in context space

As in Unix space, users move around context space with command-line utilities, or commands. The remainder of this document discusses the basic tools for manipulating context space. Most of these tools are similar to Unix commands, in the hopes that users will be able to understand and use them more easily. There are a variety of resources available for learning The Reference Manual lists all commands and instructions for using them, and man pages for all commands are included in new Legion systems (ask your system administrator if you are unsure how to use man pages). The Legion web site has on-line tutorials and postscript documents that provide information about context space and legion commands (<http://legion.virginia.edu>). In this manual, an alphabetical list of all commands and their syntax is in See Alphabetical list of Legion commands and a subject list is in See Subject listing of Legion commands .

View a context's contents

Just as the ls command in Unix allows users to view the files located in a particular directory, the legion_context_list command shows all objects contained in a particular context. The root context of a new system will look something like this:

$ legion_context_list

.

class

hosts

vaults

home

$

But a user logging into a running Legion system might see something like this in his or her home context:

$ legion_context_list

.

class

hosts

vaults

home

Foofile

Fooclass

Fooobject

$

To view sub-contexts (similar to sub-directories), the name of a context 6 is used an as argument:

$ legion_context_list /hosts

.

..

BootstrapHost

your.host.DNS.name

$

The legion_ls command will also list the contents of a context. It is similar to the legion_context_list command, but it can return more specific information. When used without flags, the command will only list the contexts and file objects contained in the current context.

$ legion_ls

.

class

hosts

vaults

home

Foofile

$

There are three optional flags, which will return varying levels of detail. The -a flag will list all objects contained in the context.

$ legion_ls -a

.

class

hosts

vaults

home

Foofile

Fooclass

Fooobject

$

The -l flag includes objects' type. File objects return their file size. Notice that when used without the -a flag it returns only contexts and file objects.

$ legion_ls -l

. (context)

class (context)

hosts (context)

vaults (context)

home (context)

Foofile 43 bytes

$

$ legion_ls -la

. (context)

class (context)

hosts (context)

vaults (context)

home (context)

Foofile 43 bytes

Fooclass (class)

Fooobject

$

The -L flag will list the LOIDs associated with the objects contained in the current context. Notice that, again, it lists only contexts and file objects in the context:

$ legion_ls -L

. 1.01.05.01000000.00000... 7

class 1.01.05.02000000.00000...

hosts 1.01.05.03000000.00000...

vaults 1.01.05.04000000.00000...

home 1.01.05.05000000.00000...

Foofile 1.01.66000000.02000000.00000...

$

As with legion_context_list , this command can be used with path names to list the contents of sub-contexts.

$ legion_ls -la /class

. (context)

.. (context)

LegionClass (class)

VanillaMetaClass (class)

BootstrapMetaClass (class)

BasicFileClass (class)

ttyObjectClass (class)

JobProxyClass (class)

BatchQueueMetaClass (class)

StatTreeClass (class)

AuthenticationObjectClass (class)

UnixHostClass (class)

UnixVaultClass (class)

BindingAgentClass (class)

CommandLineClass (class)

ContextClass (class)

UnixImplementationClass (class)

UnixImplementationCacheClass (class)

StatelessProxyMetaClass (class)

StatelessClasses (context)

$

Looking up an object's LOID

Users can look up the LOID associated with an object named in a context by using the legion_context_lookup command. The output will be a "dotted hex" LOID. Usage of this command requires including the object's context space name (its context path) as a parameter.

$ legion_context_lookup /hosts/BootstrapHost

1.01.07.01000000.000001fc0a8e60138a9...

$

Create a new context

New contexts can be created with the legion_context_create command. This is similar to the Unix mkdir command. Note that the output includes the new context's location and its LOID.

$ legion_context_create /tmp

Creating context "tmp" in parent "/".

New context LOID = "1.01.05.07000000.000001fc0d02..."

$

Changing the working context

The legion_set_context command will change the current working context, as can be seen in the following example.

$ legion_set_context /tmp

The sub-context tmp is now the current working context. Note that this command sets an environment variable in the shell in which it has been invoked. The command is implemented as an alias (as supported by your shell) that sources the script legion_set_context.sh.

Assigning alternate names to existing contexts

Users can create alternate names for objects with legion_ln , a command similar to the Unix ln command. Note that it works only with objects that already have a context name. If no context name has been assigned, you will need to use a different command (such as legion_context_add : see See Assigning a context name to a LOID ) to assign a name to assign a name to the LOID.

Here the new name Foo is assigned to the BootstrapHost object and placed in the current working context (rather than the hosts context, where the object's original context name BootstrapHost is located).

$ legion_ln /hosts/BootstrapHost Foo

$ legion_ls -la

. (context)

.. (context)

Foo

$

An object's alternate context names can be in different contexts: BootstrapHost is in the hosts context, and Foo is in the current context. Both names refer to the same LOID and either one can be used to control the object.

$ legion_context_lookup Foo

1.01.07.01000000.000001fc0a8e60138a9...

$

$ legion_context_lookup /hosts/BootstrapHost

1.01.07.01000000.000001fc0a8e60138a9...

$

Assigning a context name to a LOID

The legion_context_add command maps an object's LOID to a context name. It can be used to assign a name to a previously unnamed object or to assign an extra name to a previously named object. Either way you need to provide the object's full LOID as an argument.

In the example below, the name newName is assigned to a LOID.

$ legion_context_add 1.01.66000000.05... newName

Renaming an object

An object can be renamed with the legion_mv command (similar to the Unix mv command). In the example below, object Foo is renamed Bar .

$ legion_mv Foo Bar

The name Foo is automatically removed from the current context. This process affects only a specific context path: any other use of the name Foo , whether somewhere else in this user's or any other user's context space, will be unchanged.

Removing names

Users can remove context names with either legion_rm or legion_context_remove .

$ legion_rm Bar

or

$ legion_context_remove Bar

Neither of these commands, as used in these two examples, will affect the associated object: removing a context name does not destroy the object. If you wish to destroy or deactivate the object use the legion_rm command with, respectively, the -destroy or -deactivate flags.

The legion_destroy_object command, on the other hand, destroys the object but not its context name(s). This command should therefore not be used with objects that have context names, unless the names are then removed with either legion_rm or legion_context_remov e .

Using the same name in different contexts

As discussed in section See Organizing context space , the same context name can be used to refer to different Legion objects. For example, the name Foo can be assigned to two different objects in two different contexts, the current working context and the tmp context.

The name Foo is assigned to an object in the current context, with the legion_ln command.

$ legion_ln 1.01.05.6e0c2... Foo

The same name, in the tmp context, is then assigned to a different object.

$ legion_ln 1.01.66000... /tmp/Foo

Alternatively, the same name can be used in different context paths to refer to the same object:

$ legion_ln 1.01.05.6e0c2... Foo

$ legion_ln 1.01.05.6e0c2... /tmp/Foo

Copying a local file to a Legion file object

The contents of a local file can be copied to a new Legion file object with the legion_cp command. The example below copies the contents of a local Unix file, UnixTextFile.txt , to a new Legion file object, called newFileObject , in the current working context.

$ legion_cp -localsource UnixTextFile.txt newFileObject

The -localsource flag indicates that UnixTextFile.txt can be found in the local file system.

Copying a Legion file object

Use the legion_cp command to copy of a file object.

$ legion_cp newFileObject newFileObject_copy

The -localdest flag will copy the contents of a Legion file object to a new file in the local file system.

$ legion_cp -localdest newFileObject anotherTextFile.txt

Viewing a file object's contents

Users can view the contents of a Legion file object with the legion_cat command, similar to the Unix cat command. The contents are printed to standard output.

$ legion_cat newFileObject

This is a test, just a test, nothing more.

$

Importing a local Unix directory tree

The contentx of a local directory tree can be copied from local file space into context space with the legion_import_tree command. The contents of Unix binary files will automatically be copied into new Legion file objects. In the example below, the contents of the Unix directory legion.practice will be copied into the current Legion context. A listing of the Unix directory's contents shows that the directory contains one sub-directory and three files. The subdirectory nextlevel also contains three files.

$ ls -Fa legion.practice

./ ../ nextlevel/ nothing nothing2 nothing3

$ ls -Fa legion.practice/nextlevel

./ ../ nextnothing nextnothing2 nextnothing3

$

The default values of legion_import_tree are the current Unix directory and the current Legion context (in this case, the home context "."), but users can specify the directory path and/or the context (see the Reference Manual for full usage).

Note that Legion copied the contents of the nextlevel subdirectory into a new sub-context called nextlevel :

$ legion_ls -la

. (context)

class (context)

hosts (context)

vaults (context)

home (context)

nextlevel (context)

nothing 43 bytes

nothing2 46 bytes

nothing3 46 bytes

$ legion_ls -la /nextlevel

nextnothing3 46 bytes

nextnothing2 46 bytes

nextnothing 43 bytes

.. (context)

. (context)

$

The legion_cat command shows that the new file objects contain the same information as the Unix files:

$ cat legion.practice/nextlevel/nextnothing3

3: This is a test, just a test, nothing more.

$

$ legion_cat /nextlevel/nextnothing3

3: This is a test, just a test, nothing more.

$

Host and vault objects

A new system contains a single host and vault, called the bootstrap host and bootstrap vault, but in order to expand the resources available to your system's users will want to add host objects and vault objects to your system.

What is a host object and vault object?

A host is a physical machine (workstation, PC, etc.) that contains at least one processor and can contain active Legion objects.

A vault stores inactive Legion objects: it is a persistent storage space that manages the persistent storage space of inert (i.e., inactive) Legion objects and is the virtual representation of a persistent storage space on a processor. A vault can be in a file system, database system, tape drive, CD-rom, etc.

A host object is a Legion object that represents and can run a physical host or collection of hosts in the Legion system. The host object guards the host's resources, and can activate or deactivate other Legion objects.

A vault object is a Legion object that represents and runs the vault. Like the host object, vault objects guard a vault's resources. They do not, however, activate or deactivate the objects that they manage.

See Simple Legion system , right, shows how this division of labor works in a simple Legion system, Host Object1 representing Host1 and Vault ObjectA representing VaultA.

About the bootstrap host and vault

A new Legion system contains one host, known as the bootstrap host, and one vault, known as the bootstrap vault, as well as one host object and one vault object to manage them. The latter are called the bootstrap host object and bootstrap vault object. Legion automatically assigned these objects context names when the legion_initialize is run during the start-up procedure. Note, however, that while the bootstrap vault is assigned only one context name ( BootstrapVault ), the bootstrap host is assigned two context names ( BootstrapHost and your.bootstrap.host.name ).

To see this use the legion_context_list command:

$ legion_context_list /hosts

.

..

BootstrapHost

your.bootstrap.host.name

$

$ legion_context_list -la /vaults

.

..

BootstrapVault

$

Note that you may see other host or vault objects listed in the hosts and vaults contexts, especially if your system administrator has customized your context space for you.

Adding a new host

To add a new host, the main system must be active. If necessary, start it with the first-time start-up or restart procedure, as appropriate. Note that the new host must also have Legion distribution tree installed, so that it can communicate with your Legion system and properly carry out Legion tasks.

You will be adding the host from your current machine using the legion_starthost command. This command uses remote shell ( rsh ) classes. 8

Normal usage is below (please see section See Adding new hosts for legion_starthost 's flags and default settings).

legion_starthost [<flags>] {<new host name>} [<compatible vault list>]

The legion_starthost command can be used to start new host objects on your current host as well as on other hosts: a single machine can contain more than one host object representing and managing the resources available to it. Note that you can create multiple vault objects in a single vault.

You should specify a compatible vault whenever your create a new host object: the command does not require a vault name in order to create a new host object. There are commands for adding and removing vaults from a host's list of compatible vaults, and you can add vaults to this list after creating the host object, but if possible it is simpler to specify at least one compatible vault when running legion_starthost .

In the example below, the system's default BootstrapVault is used as the new host's vault.

In this sample output, a good deal of information is returned. Legion prints out the attributes of the newly created host object, which include its name, context, path name, vault, and LOID. It also shows the binary executable files for basic Legion objects (e.g., an implementation object) being added and configured to the new host. These files allow the new host to start new Legion objects as necessary.

Note that the output lists the new host-vault pairs that are formed when the object is created. Since the command listed one compatible vault, BootstrapVault , the new vault will only be added to BootstrapHost 's list of compatible vaults and only BootstrapVault will be added to the new host's list of compatible vaults. More than one vault can be listed as compatible when creating a host object.

One other important detail is that the host's DNS name is assigned to the new object as its context name and placed in the hosts context.

$ legion_ls -la /hosts

. (context)

.. (context)

BootstrapHost

bootstrap.host.DNS.name

new.host.DNS.name

$

If you prefer to place the new host object in a different sub-context, or if you would like to assign it a specific context name, you can use the -N flag. The example below creates a host object on the same host as above, but assigns it the context name aNewHost . Note that the argument specifies that the context name be put in the hosts sub-context (otherwise it will be put in your current context).

Run the legion_ls command to see the hosts that have been added to the system.

$ legion_ls -la hosts

. (context)

.. (context)

BootstrapHost

bootstrap.host.DNS.name

aNewHost

$

Note that the new host object's specified context name ( aNewHost ) is listed but not its DNS name (new.host.DNS.name).

Adding a new vault

The procedure of starting a new vault is similar to starting a new host. Usage of the legion_startvault command is below (please see section See Adding new vaults for legion_startvault 's flags and default settings).

legion_startvault [<flags>] {<host name>} [<compatible host list>]

The example on the next page adds a vault object on the host, uses the -N flag to assign it the context name aNewVault , and lists it as compatible with BootstrapHost .

The output here is similar to the legion_starthost output, and includes the new object's attributes and LOID. Once again, only one host was listed as compatible, so the new vault is added on to BootstrapHost 's list of compatible vaults and only BootstrapHost is added to the new vault's list of compatible hosts. Note that if you do not specify any compatible hosts the vault's list of compatible hosts will be empty. There are commands for adding and removing hosts from a vault's list of compatible hosts, and you can add hosts to this list after creating the vault, but if possible it is simpler to specify at least one compatible host when running legion_startvault .

To add more than one host to the vault object's compatibility list, just add the names of the host objects. The next example repeats the previous command to create a new vault object called aNewVault that will be compatible with BootstrapHost as well as with aNewHost .

The end of the output shows aNewVault being added to BootstrapHost 's and aNewHost 's lists of compatible vaults, and those two hosts being added to aNewVault 's list of compatible hosts.

About host-vault pairs

Hosts and vaults are always paired: a host or vault is unusable by itself. Adding new hosts and vaults to your system makes multiple processors and storage space available to your system. However, only compatible hosts and vaults can be paired. See Compatible host-vault pair , right, shows a compatible host-vault pair: the host and vault can "see" each other and can therefore pass objects between them, as the objects are activated and deactivated.

All Legion host objects must be paired with at least one compatible vault object in order to carry out Legion processes. There are several commands (see section See Manipulating host-vault pairing ) for adding, deleting, and viewing lists of acceptable hosts or vaults, but before you add a new host or vault to your system you must consider any possible compatibility problems. This is a critical point, since if a paired host object and vault object cannot communicate with each other, the Legion objects that they control will be unusable. In See Compatible host-vault pair , for instance, Host A can "see" Vault B, HostObjectA manages Host A and VaultObjectB manages space in Vault B. Host A is compatible with Vault B, so HostObjectA can be paired with VaultObjectB.

This is not a concern in systems with a single shared vault (e.g., a networked file system, database system, tape drive, CD-rom, etc.), as in See Common persistent storage system , left. Here, any vault objects will represent space in the only available disk storage space (Vault A) accessible to Host 1, Host 2, and Host 3. If the user on Host 1 wishes to add Host 2 to her system, she can either pair the new host object with the BootstrapVault already in her current system or create a new vault object (on Vault A) to manage the new host object's persistent storage needs. Either way, there is no need to worry about non-compatible pairing.

On the other hand, if this same user wanted to add a new host from a foreign system (i.e., a system with a persistent storage space that is not accessible to her bootstrap host object), she would also need to create a new vault object in the foreign system and pair it with the new host object. See Multi-host and multi-vault system (below) shows an example of this situation, with two different file systems and multiple hosts. Hosts 1-3 can see Vault A, and Hosts 4-6 can see Vault B. Again, if the user wanted to add a host from her own system she could either assign the new host object a currently existing BootstrapVault from Vault A or create a new vault object on Vault A without worrying about host-vault compatibility. But if she wants to add a host from the second system, say Host 4, she must pair it with a compatible vault object on Vault B (either by creating a new vault object with legion_startvault or by using one that has already been created and that she has permission to use).

See Adding new resources to a Legion system (below) shows how this might work. The user's Legion system creates HostObject4 and VaultObjectB , objects that represent and manage the user's Legion processes on Host 4 and Vault B. HostObject4 will manage the user's Legion work on Host 4, and VaultObjectB will manage persistent storage of any objects that HostObject4 creates or uses. Assuming that there are no conflicts in architecture, environment, etc., you can add a new host to your system with the legion_starthost command and the new host's DNS name and a new host object (representing new.host.DNS.name) will be created on the new host using the current values of $LEGION and $LEGION_OPR from your current environment. If the new host has a different architecture or has different Legion environment variables, or if the user wishes to specify a different user id, the user can specify this with legion_starthost flags.

Creating objects on new hosts

New objects can be created on the new host with the legion_create_object command.

The full syntax of this command is:

legion_create_object

{-l <class loid> | -c <legion space path>}

<context path for new object>

[-h <host on which to place new object>]

[-v <vault on which to place new object>]

[-H <context path of preferred host class>]

[-V <context path of preferred vault class>]

[-Ch <context containing list of preferred hosts>]

[-Cv <context containing list of preferred vaults>]

Note that if you must include the context path of the class which will parent the new object: if you wish to create an instance of BasicFileClass you must include the BasicFileClass context path name ( /class/BasicFileClass in the example below). To create the object on a different host or vault, use the -h or -v flags plus the host's or vault's context path. The -H and -V flags will cause the class to create an instance on a host or vault of a specified class (note that users can create their own classes of hosts as is most convenient to organize the types of hosts that they wish to work with). If the -Ch or -Cv flag is used, the class will create an instance of itself on a randomly choosen host or vault from among the hosts or vaults listed in a specified context.

The example below creates an instance of the BasicFileClass on aNewHost and assigns it the context name file . The command's output is the new object's LOID.

$ legion_create_object -c /class/BasicFileClass file \

-h hosts/aNewHost 1.01.66000000.01000000.000001fc0b0eec4e02...

$

The newly created object's context name, file , will appear in the current context (since we did not specify another context) but, as the -h flag specified, it was placed on aNewHost . This is an important feature of context space: an object's context path does not reflect its actual placement. To find an object's actual location at any time, use the legion_get_host command. The output of this command is the host object's LOID.

$ legion_get_host -c file

1.01.07.3eb53908.000001fc0d9b155044fb5...

$

Controlling instance placement on hosts and vaults

There are a group of Legion commands that allow users to control placement of a class's instances or of a specific instance. You can use the legion_class_host_list or legion_class_vault_list commands to control the list of hosts or vaults upon which a given class can place its instances, or the legion_instance_host_list and legion_ instance_vault_list to control where a given object can be placed. There are also command that will display lists of a specified object's acceptable hosts ( legion_get_candidate_hosts ), vaults ( legion_ get_candidate_vaults ), and host-vault pairings ( legion_get_ candidate_placements ). These commands are likely to be most useful for resource scheduling and management. Please see section See Command-line functions of the Reference Manual for information on the usage of these commands, look on the man pages, or the on-line tutorials (at <http://legion.virginia.edu>).

Graphic user interface

The Legion graphic user interface ( GUI) is a Java application that allows users to run context-related commands in a graphic interface. The Legion GUI uses icons to represent different parts of context space (file objects, sub-contexts, etc.) and allows users to run most context-related commands from within the GUI.

The Context Manager can be run from the command-line of any platform compatible with the Java Development Kit (JDK) 1.1.3. A Windows95 client application, called the Legion Server, allows users to run the Context Manager from Windows95.

Starting the Context Manager from the command-line

Before compiling and running the Context Manager be sure that your machine has JDK.1.1.3 or later installed.

  1. Add the following to your CLASSPATH environment variable:
  1. Add the JDK bin directory to your path (i.e., /home/jdk1.1.3/bin).
  2. To compile the application and the necessary classes, go to the $LEGION/src/UserInterface directory and enter:

$ make

  1. You must have a Legion session already running. Be sure to export the CLASSPATH variable and set your DISPLAY variable. Use the following command to start the application:

$ legion_java ContextMgr

The Context Manager can also be run in the background, so that the command line will still be active. The window will look something like See Command-line Context Manager window . Note that you cannot start the ContextMgr process from outside the $LEGION/src/UserInterface directory.

Running the GUI on Windows95

Before you can run the Legion Server, a Windows95 client application, you need to follow the above procedure on a machine running a Legion session. The make command, run in step 3 above, created a file called Legion.zip in your $LEGION/src/UserInterface path. The zip file contains the classes needed to run the Server.

You also need to have either the Java Runtime Environment (JRE) version 1.1.3 or later, or JDK version 1.1.3 or later, installed on your Windows95 machine. You can download these from <http://java.sun.com/products/>.

Starting the Context Manager in Windows95

Note that you will actually be running two separate applications, one from the command line of the machine running your Legion session and one from a DOS window of your Windows95 machine.

  1. Follow the installation instructions for Windows95. You must add the bin directory for either package to your path in the AUTOEXEC.BAT file (e.g, PATH=C:\jre\bin).
  2. Copy the file Legion.zip to your Windows95 machine and unzip it (be sure that your unzip program maintains extended filenames).This will create a new directory called UserInterface, which contains the appropriate classes for running the client.
  3. On the machine running your Legion session start a Java Legion Server by typing:

$ javaLegionServer &

We recommend that you run this in the background, although it is not necessary. Once this process has started the Legion Server it will output the name of the machine it is running on and a port number to your command line.

  1. To start the client on your Windows95 machine, open a DOS window, and change the directory to $LEGION/UserInterface and type the following:

$ Legion [machine name] [port]

Use the machine name and port from the Legion output in step 3.

The Legion Java client will open in a separate window, titled the Legion Context Manager, and can be manipulated like a normal Windows95 window. The window will look something like See Windows95 Context Manager window .

You can continue to work on the machine running your Legion session if the Java client is running in the background. Closing the DOS window will also close the Legion Context Manager window.

Shutting down the Context Manager

To shut down the Legion Context Manager, simply close the Context Manager window.

Restarting the Context Manager

To restart the command-line Context Manager, repeat step 4 in section See Starting the Context Manager from the command-line . To restart the Windows 95 Legion Context Manager, repeat steps 3 and 4 described in section See Starting the Context Manager in Windows95 . You will get a different port number each time you run the Context Manager.

Using the Legion GUI

The GUI can be used in place of context-related commands to manage context space: all context-related objects such as contexts, file objects, and objects are represented by icons and can be manipulated with the right and left mouse buttons.

Click the right mouse button in the Context Manager window to reveal three basic options:

Create Context
Create a new sub-context in the current context

Import File
Import a copy of a local file into Legion space. A separate window will appear, with the option to browse through local file space and specify a filter to use when copying the file into Legion space.

Import Directory
Import a local directory into Legion space or from Legion space into local disk space. A separate window, showing the local file system, will appear.

The GUI icons

There are five different icons in the GUI, each representing an element of context space. The context name associated with each icon will appear beneath the icon. Objects that do not have a context name will not be represented in the GUI. Clicking the right or left mouse button on icons will allow users to carry out various actions, explained below.

The context icon represents a context, similar to a Unix directory. You can change your current context by double-clicking the desired context icon with the left mouse button. Click the right mouse button to get the following options:

Get Interface
View the selected context object's interface.The information will appear in a separate window.

Get Attributes
View the selected context object's attributes. The information will appear in a separate window.

Destroy
Destroy the selected context (i.e., remove its name and LOID from Legion) and destroy any context names kept in the selected context. Note, however, that this will not destroy the objects named in that context, nor will it remove any other context names from other contexts.

Activate
Activate the selected context object.

Deactivate
Deactivate the selected context object.

The parent context icon represents the current context's parent context. Double-click the left mouse button on this icon to change your current context to the parent context. Clicking the right mouse button will produce the same options as with the context icon (above).

The class icon represents a class. You can double-click the left mouse button on a class icon to see a list of the class's instances. Click the right mouse button to get the following options:

Show Instances
Display all instances of a selected class in a separate window. See See Context Manager listing a class's instances , below, for more information about this option.

Run.. .
Executes a previously registered executable program class. Please see See Executing remote programs for information about registering remote programs, and See Running a remote program from the GUI for information about running remote programs in the GUI. This option will produce a separate window, which can be used to specify input and output file names. The Ctrl-p key combination will also run this process.

Get Interface
View the selected context object's interface.The information will appear in a separate window.

Get Attributes
View the selected context object's attributes. The information will appear in a separate window.

Destroy
Destroy the selected object (i.e., remove its name and LOID from Legion).

Activate
Activate the selected context object.

Deactivate
Deactivate the selected context object.

The instance icon represents an instance of a class. Click the right mouse button to get the following options:

Get Interface
View the selected context object's interface.The information will appear in a separate window.

Get Attributes
View the selected context object's attributes. The information will appear in a separate window.

Destroy
Destroy the selected object (i.e., remove its name and LOID from Legion).

Activate
Activate the selected instance.

Deactivate
Deactivate the selected instance.

The file object icon represents a file object. Double-click the left mouse button on this icon to open a text editor window displaying the file's contents. The text editor allows users to edit file objects. Click the right mouse button to get the following options:

View File
Display contents of selected file object in a separate window.

Copy
Make a copy of the file object. This will create a new object, with a new LOID and context name but with the same content.

Move
Move the selected file object's contents to a new file object.

Export File
Export a copy of a local file to local disk space.

Get Interface
View the selected context object's interface.This information will appear in a separate window.

Get Attributes
View the selected context object's attributes. This information will appear in a separate window.

Destroy
Destroy the selected object (i.e., remove its name and LOID from Legion). A "Destroy" window will appear, where you should specify whether you want to destroy just the object's context name or the object (both its context name and its LOID).

Activate
Activate the selected instance.

Deactivate
Deactivate the selected instance.

In all contexts, users can refresh the current context by clicking on the upper left hand "." current context icon ( See Switching contexts in the Context Manager a). Similarly, users can always get to a sub-context's parent context by double-clicking on the ".." parent context icon, to the right of the "." icon ( See Switching contexts in the Context Manager b). Note that the parent context icon will not appear if you are currently in your home context.

Note that changes to context space made from the command line are not automatically updated in the GUI. Users should refresh their current context as necessary. Changes made in the GUI, however, are reflected in the context space and can be viewed with command utilities (i.e., legion_ls or legion_context_list ).

The GUI pull-down menus

The pull-down menus can be used to do the following:

File -> New -> Open New Window
Open a new Context Manager window. This will point to the same context space as the original Context Manager window.

New File
Create a new File Object. A New File window will open, where you can enter the new object's context path name, then a text editor window will open. The Ctrl-n key combination will also run this process.

View File
Display contents of a selected textual file object in a separate window.

Import File
Import a copy of a local file into Legion space. A separate window will appear, with the option to browse through local file space and specify a filter to use when copying the file into Legion space. The Ctrl-i key combination will also run this process.

Import Directory
Import a local directory into Legion space or from Legion space into local disk space. A separate window, showing the local file system, will appear.

Export File
Export a copy of a local file to local disk space. The Ctrl-e key combination will also run this process.

Quit
Close the current window. If only one window is open, the program will exit. The Ctrl-x key combination will also run this process.

Edit -> File -> View File
Opens a Text Editor window, where you can edit the file's contents.

Copy File
Make a copy of the file object. This will create a new object, with a new LOID and context name but with the same content. The Ctrl-c key combination will also run this process.

Export File
Export a copy of a local file to local disk space. The Ctrl-e key combination will also run this process.

Class -> Run...
Executes a previously registered executable program class. Please see See Executing remote programs for information about registering remote programs, and See Running a remote program from the GUI for information about running remote programs in the GUI. This option will produce a separate window, which can be used to specify input and output file names. The Ctrl-p key combination will also run this process.

Show Instances
Display all instances of a selected class in a separate window. The "Show Instances" option returns several pieces of information about the selected class's instances, as shown in See Context Manager listing a class's instances (below): it lists each instance's LOID, status, Object Address (OA: an active object's location), Host, and Vault. The "Instance" pull-down menu allows users to activate, deactivate, and destroy selected or all instances, as well as to refresh the window. The Ctrl-p key combination will also run this process.

Activate All Instances
Activate all instances of a selected class. The Ctrl-a key combination will also run this process.

Deactivate All Instances
Deactivate all instances of a selected class. The Ctrl-d key combination will also run this process.

Get Interface
View the selected context object's interface.This information will appear in a separate window. The Ctrl-g key combination will also run this process.

Get Attributes
View the selected context object's attributes. This information will appear in a separate window. The Ctrl-h key combination will also run this process.

Destroy
Destroy a selected object's path name and/or the object itself. A "Destroy" window will appear, where you should specify whether you want to destroy just the object's context name or the object (both its context name and its LOID). The Ctrl-Delete key combination will also run this process.

Activate
Activate a selected, currently inert, object.

Deactivate
Deactivate a selected, currently active, object, i.e. move it to an inert state.

Running a Legion application

The mechanics of running a Legion application will, of course, vary widely from one application to another. Generally, however, users will need to compile, register, and run individual applications with commands written specifically for that purpose. For example, MPI applications use the legion_mpi_register and legion_mpi_run commands, and PVM applications use the legion_pvm_register command. Some commands may require users to select a group of hosts that the application should use before running the application, others may just ask how many hosts should be used.

Users can arrange parts of their context space around the needs of specific applications or to organize certain resources (a sub-context of names of hosts that can be used for a specific MPI application, for instance).

Users can view the output of a program by creating and setting tty objects to send output to a specific file or a specific window, so that they can view multiple outputs simultaneously.

Information about Legion's MPI and PVM libraries can be found in sections See PVM and See MPI . Information about Legion's Basic Fortran Support can be found in the Legion Developer Manual (section See Fortran ).

PVM
Core PVM interface

The current Legion implementation supports a core PVM interface See A. Geist, A Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V.S. Sunderam, PVM: Parallel Virtual Machine, MIT Press, 1994. See V.S. Sunderam, PVM: A framework for parallel distributed computing, Concurrency: Practice and Experience, vol. 2(4), pp. 315-339, Dec. 1990. , which includes functions for data marshaling, message passing, and task creation and control. Legion supports legacy PVM codes, and provides an enhanced PVM environment using Legion features such as security and placement services. For example, PVM applications can be run with encrypted message passing on Legion's PVM implementation.

A link-in replacement PVM library uses the primitive services provided by Legion, to support the PVM interface. For example, PVM tasks map directly to Legion objects: thus pvm_spawn() uses Legion.CreatObject() , and PVM task ids are mapped to Legion LOIDs. PVM Buffers are implemented by using Legion Buffers (the fundamental data container in the Legion runtime library)-- pvm_pmkint() thus uses LegionBuffer:: put_int() . Similarly, PVM message-passing maps to Legion method invocation: the PVM-Legion library exports a new method, LegionPVMSend() . Sending a message to a task maps to calling the LegionPVMSend() method on the object corresponding to that task. A LegionBuffer containing the message becomes the parameter to the method.

Tids & LOIDs

Whereas PVM tasks are identified by integer task identifiers ( Tids), Legion objects are identified by LOIDs, which come in varying sizes but are generally much larger than an int. The Legion-PVM library must therefore provide a mapping between Tids and LOIDs. Legion's contexts are used to manage this mapping, and the Legion library caches the mappings that have already been looked up. Since Tid-to-LOID mappings are invariant, caching can be performed without any consistency management costs.

Task classes

Whereas PVM spawn requires a "task name" parameter to identify the type of task to spawn, Legion uses class LOIDs to specify an object's type. The Legion-PVM library utilizes a single class object for each type of task: therefore, Legion-PVM tasks are instances of Legion classes. Mapping between task names (i.e., as specified to pvm_spawn() ) and class LOIDs is managed in Legion context space.

Legion classes have associated "implementation objects," which provide their instances with binary implementations. Because of this, PVM-Legion does not spawn tasks stored in a "well-known" directory location, as standard PVM does. A mechanism is provided for "registering" task implementations with the appropriate classes. Once executables are registered, the user does not need to explicitly manage the copying of binaries between hosts on disjoint file systems, since Legion automatically manages this, unlike standard PVM.

Using PVM-Legion
Installation

The PVM-Legion library can be installed using the source files in $LEGION/src/ServiceObjects/PVM3.

Compilation

Task object code files can be compiled as before. Link against libLegionPVM.3, and basic Legion libraries (the final linkage must be performed using a C++ compiler, or using C++ appropriate flags for ld , since the Legion library has C++ link-time requirements).

A sample Legion PVM makefile is shown in See Sample Legion PVM make file .

Registering compiled tasks

This step can be skipped for tasks that will be started only from the command line (tasks that will not be "spawned").

Run legion_pvm_register :

legion_pvm_register <class name> <binary path> <platform type>

This creates PVM-specific contexts, a class object for the task class, registers the name in Legion context space, and creates an implementation object for the class.

You can now run the PVM application. If necessary, you can examine Legion context space to get information about the PVM-Legion state using either

legion_context_list /pvm

which will list the Tids of running PVM tasks. Use the -L flag to also list the LOIDs, or

legion_context_list /pvm/tasks

which lists registered task classes. You can also use Legion class utilities to examine the class state (e.g., legion_list_instances ).

Examples

There are several examples of PVM programs in an example file in your Legion system. To get there, enter:

$ cd $LEGION/src/ServiceObjects/PVM3/examples

To compile the example code, enter:

$ make

To register the example task with the Legion/PVM system, use the legion_pvm_register command. The first time this is run Legion will create /pvm and /pvm/tasks directories. Your output will look something like this:

$ legion_pvm_register hello_other \

$LEGION/bin/$LEGION_ARCH/hello_other $LEGION_ARCH

"/pvm" does not exist - creating it

"/pvm/tasks" does not exist - creating it

Task class "/pvm/tasks/hello_other" does not exist - creating it

$

The hello_other class is now registered in context space, and can be seen with the legion_ls command.

$ legion_ls -la /pvm/tasks

. (context)

.. (context)

hello_other (class)

$

You can now run a simple example, with the new class. This example creates a new task, and sends a single message. Notice that the command to run the example does not have a "legion_" prefix.

$ hello

i'm t34c8e665

from t64d7b9e5: hello, world from t64d7b9e5 (parent t34c8e665) on xxx.xxx.xxx.xxx

$

The other examples can be registered and run as well, following the same procedure.

MPI

The current Legion implementation supports a core MPI interface, which includes messages passing, data marshaling and heterogeneous conversion. Legion supports legacy MPI codes, and provides an enhanced MPI environment using Legion features such as security and placement services.

A link-in replacement MPI library uses the primitive services provided by Legion to support the MPI interface. MPI processes map directly to Legion objects.

Task classes

MPI implementations generally require that MPI executables reside in a given place on disk. Legion's implementation objects serve a similar role. We have provided a tool, legion_mpi_register , which registers executables of different architectures for use with Legion MPI.

Using Legion MPI
Installation

The Legion MPI library can be found in $LEGION/src/ServiceObjects/MPI.

Compilation

MPI code may be compiled as before. Link it against libmpi and the basic Legion libraries. The final linkage must be performed using a C++ compiler or by passing C++ appropriate flags to ld .

A sample Legion MPI makefile is shown in See Sample Legion MPI make file .

Register Compiled Tasks

Run legion_mpi_register . Usage of this tool is:

legion_mpi_register <class name> <binary path> <platform type>

The example below registers xxx/myMPIprograms/vdelay (the binary path) as using a Linux architecture.

$ legion_mpi_register vdelay xxx/myMPIprograms/vdelay linux

This creates MPI-specific contexts ( /mpi , /mpi/programs , and /mpi/instances ), a class object for this program, an implementation for this program, and registers the name in Legion context space.

The legion_mpi_register command may be executed several times if you have compiled your program on several architectures.

Running the MPI Application

MPI programs are started using the program legion_mpi_run :

legion_mpi_run {-n <number of processes> <command>} {<flags>}
[arguments]

supported < flags > are:

-h < host context path > Specify the set of hosts on which the

program will run

(default is the system's default placement)

-0 < legion context name > Runs the 0th process on this node

-n < number of processes > Specify the number of processes that the

program will run

-p < PID context > Specify a name for PIDs

(default is /mpi/instances/program_name)

-s Print statistics at exit

-v Verbose option

-d < Unix path name > Specify that all children change to the

same directory before they begin to run.

-f < file name > Allows different MPI programs to be

started with a common

MPI_COMM_WORLD.

Note that if the -f flag is used the file named in < file name > will be read first, and the -n flag and program name (if it is included) will be ignored. The legion_mpi_run utility will expect one command per line in the file: each line can contain any of the legion_mpi_run flags (except the -f flag). The MPI application named in the legion_mpi_run command will then be started, with the given flags. Note that when multiple MPI programs are run with a legion_mpi_run command, via the -f flag, any other flags used (e.g., -h to specify a set of hosts) will be applied to all of the programs run unless otherwise directed by the program itself.

For example:

$ legion_mpi_run -n 2 mpi/programs/vdelay

Note that the < command > argument here is the Legion context space name for the class created by legion_mpi_register in the previous section. In this example the default path mpi/programs is used with the class name vdelay .

You can examine the running objects of your application using

$ legion_context_list mpi/instances/program_name

This context will have an entry for each object in this MPI application. If you run multiple versions of the application simultaneously, use the -p flag with legion_mpi_run to specify an alternate context in which to put the list of objects.

To view all instances of the program's class, use legion_ list_instances :

$ legion_list_instances -c mpi/programs/program_name

To destroy all of the program class's instances, use the legion_ destroy_instances command. Note, though, that this command will search out and destroy all of the class's instances.

Note that MPI programs cannot save their state or be deactivated. They simply die. This may have an impact on fault-tolerance, although we will it in a future release.

Example

There are several sample MPI programs in your Legion system. First, go to the directory containing the program's binary files:

$ cd $LEGION/src/examples

To compile the example code, enter:

$ make $LEGION/bin/$LEGION_ARCH/mpitest_c

You must then register the program with the legion_mpi_register command. The sample output below shows Legion creating three new contexts ( mpi , programs , and instances , the last two being sub-contexts of mpi ), creating and tagging a task class, and registering the implementation of the mpitest_c program.

In order to view the output of the program, you will need to create and set a tty object. The example below creates a tty object, sets it as an environmental variable, and then directs its output to the background. (For more information on using these commands, please see the Legion Reference Manual.)

$ legion_create_object -c class/ttyObjectClass mytty

1.01.67000000.01000000.000001fc0a6876...

$ legion_set_tty mytty

$ legion_tty_watch &

Note that this sequence adds the context name mytty in your current context. Users might actually find it more convenient to create a context to hold tty objects, to help organize their context space.

Having done all of this, you can now run the program, with the legion_mpi_run command to run the program. You must specify how many processes the program should use to run the program with the -n flag.

If you do not specify which hosts the program should run its processes on, Legion will randomly choose the hosts from the host objects that are in your system. Here it runs three processes on three different hosts.

$ legion_mpi_run -n 3 /mpi/programs/mpitest_c

Hello from node 0 of 3 hostname Host2

Hello from node 1 of 3 hostname Host3

Node 0 has a secret, and it's 42

Node 1 thinks the secret is 42

Hello from node 2 of 3 hostname BootstrapHost

Node 2 thinks the secret is 42

Node 0 thinks the secret is 42

Node 0 exits barrier and exits.

Node 1 exits barrier and exits.

Node 2 exits barrier and exits.

$

Another time it might run the three processes on two hosts.

$ legion_mpi_run -n 3 /mpi/programs/mpitest_c

Hello from node 1 of 3 hostname BootstrapHost

Hello from node 0 of 3 hostname Host3

Node 0 has a secret, and it's 42

Node 1 thinks the secret is 42

Hello from node 2 of 3 hostname BootstrapHost

Node 2 thinks the secret is 42

Node 0 thinks the secret is 42

Node 0 exits barrier and exits.

Node 1 exits barrier and exits.

Node 2 exits barrier and exits.

$

Scheduling MPI processes

As noted above, Legion may not choose the same number of hosts objects as processes specified in the -n parameter. If you specify three processes, Legion will randomly choose to run them on one, two, or three hosts.

If you wish to specify which hosts are used to run your processor, use the -h flag. In the example below, we will run mpitest_c on four hosts, which we will assume are already part of the system (see See Expanding the system's resources in the System Administrator Manual for information about adding hosts to a Legion system) and which, in this example, are named in a single context called mpi-hosts .

First, you must create the mpi-hosts context with the legion_context_create command:

$ legion_context_create mpi-hosts

Creating context "mpi-hosts" in parent ".".

New context LOID = "1.01.05.44b53908.000001fc0940fbf7

We can now assign new context names to the four hosts that we wish to use for this program, using the legion_ln command. These new names map to the host objects' LOIDs, and provide a convenient way to specify different hosts for different MPI programs or to execute the same program on different groups of hosts.

$ legion_ln /hosts/Host1 /mpi-hosts/mpitest_Host1

$ legion_ln /hosts/Host2 /mpi-hosts/mpitest_Host2

$ legion_ln /hosts/Host3 /mpi-hosts/mpitest_Host3

$ legion_ln /hosts/Host4 /mpi-hosts/mpitest_Host4

The program can now be run with legion_mpi_run , using the -h flag to specify that the four processes should be run on the hosts in mpi-hosts . Note that the nodes are placed on the hosts in order.

$ legion_mpi_run -h /mpi-hosts -n 4 /mpi/programs/emptiest

Hello from node 0 of 4 hostname Host1.xxx.xxx

Hello from node 1 of 4 hostname Host2.xxx.xxx

Node 0 has a secret, and it's 42

Hello from node 2 of 4 hostname Host3.xxx.xxx

Node 1 thinks the secret is 42

Node 2 thinks the secret is 42

Hello from node 3 of 4 hostname Host4.xxx.xxx

Node 3 thinks the secret is 42

Node 0 thinks the secret is 42

Node 0 exits barrier and exits.

Node 1 exits barrier and exits.

Node 2 exits barrier and exits.

Node 3 exits barrier and exits.

$

Note also that the -n parameter is still used, even though there are only four host objects named in the mpi-hosts context. The number of processes and number of host object do not have to match: if a user wanted to run only two processes on the hosts named in mpi-hosts , he or she could use -n 2 . The program would then run on the first two hosts listed in the mpi-hosts context. 9

$ legion_mpi_run -h /mpi-hosts -n 2 /mpi/programs/mpitest

Hello from node 0 of 2 hostname Host1.xxx.xxx

Node 0 has a secret, and it's 42

Hello from node 1 of 2 hostname Host2.xxx.xxx

Node 1 thinks the secret is 42

Node 0 thinks the secret is 42

Node 0 exits barrier and exits.

Node 1 exits barrier and exits.

$

Debugging support

Output from a Legion MPI object to stdout or Fortran unit 6 is automatically directed to a tty object. Note, though, that this output will be mixed with other output in your tty object, and you may wish to segregate MPI output. Legion MPI supports a special set of library routines to print on the Legion tty, which is created using the legion_set_tty and legion_tty_watch commands. This will cause the MPI object's output to be printed in segregated blocks.

LMPI_Console_Output (string) Buffers a string to be output later.

LMPI_Console_Output_Flush () Flushes all buffered strings to the tty.

LMPI_Console_Output_And_Flush (string) Buffers and flushes in one call.

The Fortran version of these routines adds a carriage return at the end of each string, and omits trailing spaces. The C version does not.

A utility program, legion_mpi_debug , allows the user to examine the state of MPI objects and print out their message queues. This is a handy way to debug deadlock.

legion_mpi_debug [-q] {-c <instance context path>}

The -q flag will list the contents of the queues. For example:

$ legion_mpi_debug -q -c /mpi/instances/vdelay

Process 0 state: spinning in MPI_Recv source 1 tag 0 comm?

Process 0 queue:

(queue empty)

Process 1 state: spinning in MPI_Recv source 0 tag 0 comm?

Process 1 queue:

MPI_message source 0 tag 0 len 8

$

Do not be alarmed that process 1 hasn't picked up the matching message in its queue: it will be picked up after the command has finished running.

There are a few limitations in legion_mpi_debug : If an MPI object doesn't respond, it will hang, and it won't go on to query additional objects. An MPI object can only respond when it enters the MPI library; if it is in an endless computational loop, it will never reply.

Functions supported

Please note that data packed/unpacked using the MPI_Pack and MPI_Unpack functions will not be properly converted in heterogeneous networks.

All MPI functions are currently supported in Legion MPI.

Executing remote programs

Legion's remote execution allows users to run remotely executed programs from the Legion command line, taking advantage of Legion's distributed resources. The user's workload can be spread out on several processors, improving job turn-around time and performance. Users can execute multiple instances of a single program or execute several different programs in parallel.

Introduction

A remote program is an executable process that can be started with a command-line utility. Note that a remote program is not part of the Legion system, but is an independent program that may or may not be compatible with Legion. Remote programs might be shell scripts, compilers, existing executables, etc. The term "remote" here means that the program does not need to be executed on the local host.

What kind of programs can Legion run?

Legion can run both Legion-compatible and non-Legion-compatible programs. A compatible program is linked to the Legion libraries. A non-compatible program does not have these links. The legion_register_program command, which registers non-compatible programs with the Legion system, creates a proxy to handle a particular program. When the program is run, with the legion_run command, the proxy is activated and can instruct the program to begin execution.

Can Legion run only compatible remote programs?

Legion can run both Legion and non-Legion (i.e., programs that are not linked to the Legion libraries, such as shell scripts) remote program since it relies on two different types of interfaces to pass Legion's execution order to the program. Before either a compatible or non-compatible remote program can be run from Legion it must be registered . The legion_register_runnable command registers Legion-compatible programs, and the legion_register_program command registers a non-Legion program. A single command, legion_run , runs both types of remote programs. Both of these commands have parameters that include the remote program's binary path name and architecture and assign the program a context path name: this allows Legion to know where the program's executable files are located, what platform the files use, and how to find the program's Legion interface.

Registering non-Legion-compatible programs

Non-compatible remote programs are registered with the legion_register_program command. The user includes information on the program's architecture (e.g., linux, sgi, solaris, etc.), its binary path name, and a program class.

legion_register_program <program class> <executable path> <legion arch>

The < program class > parameter is a context path name for the class that will manage Legion objects related to a particular program. Note that since the user decides how to organize his or her own context space, the path name can be whatever the user finds most convenient (although you might want to use the name of the program in the < program class > parameter, so that it will be easier to remember). The context path name can be new or can use a previously created path (if the user is registering multiple versions of a the same program), and will refer to a (new or previously created) Legion object that functions as a proxy for executing a non-compatible remote program.

Note also that if multiple programs are registered with the same program class and architecture, the most recently registered version will be used when the program is run on that particular architecture. Once the program has been registered, it can be run with the legion_run command.

An example of this command might look like this:

$ legion_register_program myProgram /bin/myProgram linux

Program class "myProgram" does not exist.

Creating class "myProgram".

Registering implementation for class "myProgram"

$

This output shows Legion creating the program class myProgram , as the command requested. If this class had been previously created, the output would look like this:

$ legion_register_program myProgram /bin/myProgram linux

Registering implementation for class "myProgram"

$

Registering Legion-compatible programs

The legion_register_runnable command registers programs that are linked to the Legion libraries, and export a runnable object interface. This is, the command creates an interface between the Legion system and the remote program and this interface transfers the execution command from Legion to the remote program. Like legion_register_program , this command specifies the remote program's program class, executable binary path, and architecture. Its usage is:

legion_register_runnable <program class> <executable path> <legion arch>

As with legion_register_program , the < program class > parameter is a context path name for the Legion objects that will handle a particular program. And, again, the user is free to choose a context path that best suits his or her organizational scheme, or use a previously created context (although you might want to use the name of the program in the < program class > parameter, so that it will be easier to remember). In that case, note that Legion will use the most recently registered program class and architecture when running a remote program. Once the program has been registered, it can be run with the legion_run command.

$ legion_register_runnable myProgram /bin/myProgram linux

Program class "myProgram" does not exist.

Creating class "myProgram".

Registering implementation for class "myProgram"

$

This output shows Legion creating the program class myProgram , as the command requested. If this class had been previously created, the output would look like this:

$ legion_register_runnable myProgram /bin/myProgram linux

Registering implementation for class "myProgram"

$

Running a remote program

Use legion_run to remotely execute programs. but note that you must first register the program with either legion_register_program for non-Legion compatible programs or legion_register_runnable for Legion compatible programs. The usage of this command is:

legion_run [-help] [-w] [-a <architecture>]
[-in <context path name>] [-out <context path name>]

[-IN <local file name>] [-OUT <local file name>]

[-f <options file>] <program class> [<arg1>, <arg2>, ... <argn>]

The legion_run command executes a single instance of the program associated with the < program class > parameter. Legion will randomly choose a host with a suitable architecture (specified when the program was registered) to execute the program.

There are several optional parameters associated with this command. The -help flag displays the command's syntax and provides a brief description of how to use its options. The -w flag directs the command's output to your tty object. Note that if you have not created or set a tty object for your current window, you will not be able to see the command's output and an error message will appear. You can use the legion_create_object command to create a tty object, and the legion_set_tty command to set the tty as an environmental variable:

$ legion_create_object -c class/ttyObjectClass mytty

1.01.67000000.01000000.000001fc0a6876...

$ legion_set_tty mytty

$ legion_tty_watch &

If you prefer not to use the -w flag, you can use the legion_tty_watch command to direct your output to a previously created tty object.

The -a flag allows you to specify the type of platform the program requires. Note that the possible architectures are limited. At the moment they are:

  • linux (corresponds to x86 running Red Hat 4.x Linux)
  • solaris (corresponds to Sun workstations running Solaris 5.x)
  • sgi (corresponds to SGI workstations running IRIX 5.x)
  • alpha_linux (corresponds to DEC Alphas running Red Hat 4.x Linux)
  • alpha_DEC (corresponds to DEC Alphas running OSF1 v4)
  • t90 (corresponds to Cray T90s running Unicos 10.x)
  • rs6000 (corresponds to IBM RS/6000s running AIX 4.2).

The -in < context path name > option allows you to select a Legion file that should be copied into the remote program's current working directory before the program executes. The -out < context path name > option instructs Legion to copy the program's output from the remote program's current working directory to the Legion file named in < context path name >, after the program terminates (partial results will be available if the program crashes).

The other two options operate on a local file: -IN < local file name > tells Legion to copy the files named in < local file name > into the remote program's current working directory before the program executes, and -OUT < local file name > tells Legion to copy the program's output to the file named in < local file name >.

Files that are listed with the -in / -out and -IN / -OUT options can be named with either full or relative path names. Note, however, that the remote program will use the file path's basename (i.e., as relative path names from the current working directory) to gain access to the named file. New files will be given names corresponding to the basename of the current Legion context path or local directory path.

The < arg1 >, < arg2 >, ... < argn > parameters are arbitrary command-line arguments.

If your program requires extensive use of these options you may want to use the -f flag, which allows user to list the options in a separate file rather than on the command line. The file can contain any of the options discussed above and can be delimited with spaces or new lines. 10

Converting a C/C++ program

Any C or C++ program can be easily made into a Legion runnable object using the following steps:

  1. The program should export a C-linkable legion_main function in place of its main function.
  2. The program should be linked with libLegionRun.a (in addition to the standard Legion libraries: libLegion.a and libBasicFiles.a).
Summary

The entire process, using the Unix cp process as a sample program, might look something like this:

$ legion_register_program cp /bin/cp linux

Program class "cp" does not exist.

Creating class "cp".

Registering implementation for class "cp"

$ legion_run -in /dir1/src -out /dir2/dest cp src dest

The remote Unix program cp is first registered, and the program class " cp " is created in the user's local context, /bin/cp is declared to be its local executable path name, and linux is declared to be its architecture type.

The program is then remotely executed on a randomly chosen linux host, and a series of events takes place: first, the -in option tells Legion to copy the Legion file /dir1/src into the remote program's cp 's current working directory and call it src . The remote program cp then executes, copying src to a new Unix file, dest . Finally, once cp has terminated, Legion copies the contents of Unix file dest to a new Legion file called dest , and adds the context name dest to /dir2 (i.e., if the user runs legion_context_list , dest will now be listed and will refer to the dest file).

Note that you can ask for multiple input and output files when you execute a program by repeating the -in / -out and -IN / -OUT options.

Running a remote program from the GUI

Once a program has been registered, it can be run from the Legion GUI (please see See Graphic user interface for information about starting and running the GUI).

Go to the context containing the program class context name, and click the right mouse button on the icon representing the program class. A list of options will appear: click on the Run... option.

This will bring up a new window, which users should use to specify arbitrary command-line arguments for running the program and the context names of any input or output files (the -in / -out parameters). Users can use currently existing output files or create new ones. Users can also specify the program's architecture from a list of acceptable architectures at the bottom of the Run window: this list contains the names of the architectures that Legion currently accepts. The any option uses the default architecture.

A few caveats:

  • The GUI version of legion_run does not include all of the options that the command-line version does: the -IN / -OUT , -f < options file >, and the -w options are not available. Programs that require these options should be run from the command line.
  • The Input Legion Files and Output Legion Files dialog boxes use context path names, just as the -in / -out parameters do in legion_run . In other words, if a file does not have a context name it cannot be used as an input or output file.

Appendices

Alphabetical list of Legion commands

Full documentation of these commands can be found in the Reference Manual, in See Command-line functions . There are also man pages for all Legion commands included in the system release package, and on-line tutorials on the Legion web site (<http://legion.virginia.edu>).

legion_activate_instances {-c <class context path> | -l <class LOID>}

legion_activate_object {-l <object LOID> | -c <context path>}

legion_add_class_mapping <metaclass LOID> <class LOID>

legion_bfs <file> [-o <out file>] [-BackEnd]

legion_cat <context path>

legion_class_host_list [-c <class context path> | -l <class LOID>]

[{-a | -d} <host name>] [-p] [-u]

legion_class_vault_list [-c <class context path> | -l {<class LOID>}

[{-a | -d} <vault name>] [-p] [-u]

legion_classof {-c <context path> | -l <object LOID>}

legion_context_add <object LOID> <context name>

legion_context_create <context path>

legion_context_list [<context path>]

legion_context_lookup <context path>

legion_context_remove <context path>

legion_cp [-localsource | -localdest] <source file> <destination file>

legion_create_class [-c < context path>]

legion_create_implementation <binary path name> <architecture>

[-l <class LOID> | -c <context path>]

legion_create_object

{-l <class LOID> | -c <class context path>}

<new object context path>

[-h <host name on which to place new object>]

[-v <vault on which to place new object>]

[-H <context path of preferred host class>]

[-V <context path of preferred vault class>]

[-Ch <context containing list of preferred hosts>]

[-Cv <context containing list of preferred vaults>]

legion_create_object_r

{-l <class LOID> | -c <class context path>}

<context path> <host name> <host architecture> <$LEGION> <$LEGION_OPR> <$LEGION_OPA> <binary path> [<user id>]

legion_create_stat_tree <base context path>

legion_create_user <user id>

legion_deactivate_instances

{-c <class context path> | -l <class LOID>}

legion_deactivate_object {-l <object LOID> | -c <context path>}

legion_destroy_instances {-c <class context path> | -l <class LOID>}

legion_destroy_object {-l <object LOID> | -c <context path>}

legion_direct_output {-c <object path> | -l <LOID>}

{-c <tty path> | -l <tty LOID>}

legion_exports_interface {-l <LOID> | -c <context path>}

{-w <well known class type> | -f <function signature>}+

legion_get_acl [-l <loid> | -c <context>]

legion_get_candidate_hosts

{-c <object context path> | -l <object LOID>}

legion_get_candidate_placements

{-c <class context path> <instance context path>}

| {-l <class LOID> <instance LOID>} [<max placements>]

legion_get_candidate_vaults

{-c <object context path> | -l <object LOID>}

legion_get_host {-l <object LOID> | -c <object context path>}

legion_get_implicit_params [-l <loid> | -c <context>]]

legion_get_interface {-l <class LOID> | -c <context path>}

legion_get_vault {-l <object LOID> | -c <object context path>}

legion_host_stats {-c <host context path> | -l <host loid>}

legion_host_vault_list {-c <host context path> |-l <host LOID>]

[{-a | -d} <vault name>] [-p] [-u]

legion_import_tree [<unix directory path> [<legion context path>]]

legion_init_arch

legion_initialize

legion_instance_host_list {-c <context path> | -l <LOID>}

[{-a | -d} <host name>] [-p] [-u]

legion_java <java options>

legion_list_attributes {-c <context path> | -l <object LOID>}

[-L] [<attribute name>]

legion_list_instances {-l <class LOID> | -c <context path>}

legion_list_objects {-c <host context path> | -l <host LOID>}

legion_ln <context path> <new alias>

legion_login [-l <user LOID> | <user id>] [-e <command>]

legion_ls [-lLa] <context path>

legion_mplc

legion_mpi_debug [-q] {-c <instance context path>}

legion_mpi_register <class name> <binary local path name>

<platform type>

legion_mpi_run [-h <hosts context path>] [-p <PID context path>]

-n <number of hosts> <command> [<arguments>]

legion_mv <context path> <new context path>

legion_passwd {-l <user LOID> | <user id>}

legion_ping {-c <context path> | -l <object LOID>}

legion_print_config

legion_pvm_register

<class path name> <binary local path name> <platform type>

legion_register_program

<program class> <executable path> <legion arch>

legion_register_runnable

<program class> <executable path> <legion arch>

legion_rm [-deactivate | -destroy] <context path>

legion_run [-help] [-w] [-a <architecture>]

[-in <context path name>] [-out <context path name>]

[-IN <local file name>] [-OUT <local file name>]
[-f <options file>] <program class> [<arg1>, <arg2>, ... <argn>]

legion_set_acl [-c <object context path> | -l <object LOID>]

{-s | filename}

legion_set_context <context path>

legion_set_host {-l <object LOID> | -c <object context path}

{-l <host LOID> | -c <host context path>}

legion_set_implicit_params

[-c <object context path> | -l <object LOID>] {-s | filename}

legion_set_tty <tty context path>

legion_set_vault {-l <object LOID> | -c <object context path}

{-l <vault LOID> | -c <vault context path>}

legion_setup_state [-i]

legion_shutdown [-local] [-f] [-i] [-h]

legion_shutdown_class {-l <class LOID> | -c <context path>}

legion_starthost [-L <$LEGION>] [-O <$LEGION_OPR>] [-A <$LEGION_ARCH>]

[-N <context name>] [-U <user id>]

{<new host name>} [<compatible vault list>]

legion_startup [-local]

legion_startvault [-L <$LEGION>] [-O <$LEGION_OPR>]

[-A <$LEGION_ARCH>] [-N <context name>] [-U <user id>]

{<host name>} [<compatible host list>]

legion_tty_redirect <context path>

legion_tty_unredirect <context path>

legion_tty_watch {-c <tty context path> | -l <tty LOID>}

legion_update_attributes {-c <context path> |-l <object LOID>}

[{-a | -d | -t} <attribute description>] [-u]

legion_vault_host_list {-c <vault context path> | -l <vault LOID>}

[{-a | -d} <host name>] [-p] [-u]

legion_wellknown_class [<wellknown class name>]

Subject listing of Legion commands

Full documentation of these commands can be found in the Reference Manual, in See Command-line functions , and on the Legion web site at <http://legion.virginia.edu/documentation/tutorials/commands_all.html>. There are also man pages for all Legion commands included in the system release package.

Calls on Objects

legion_exports_interface {-l <LOID> | -c <context path>}

{-w <well known class type> | -f <function signature>}+

legion_get_interface {-l <class LOID> | -c <context path>}

legion_ping {-c <context path> | -l <object LOID>}

legion_list_attributes {-c <context path> | -l <object LOID>}

[-L] [<attribute name>]

legion_update_attributes {-c <context path> |-l <object LOID>}

[{-a | -d | -t} <attribute description>] [-u]

Calls on Class Objects

legion_activate_object {-l <object LOID> | -c <context path>}

legion_create_object

{-l <class LOID> | -c <class context path>}

<new object context path>

[-h <host name on which to place new object>]

[-v <vault on which to place new object>]

[-H <context path of preferred host class>]

[-V <context path of preferred vault class>]

[-Ch <context containing list of preferred hosts>]

[-Cv <context containing list of preferred vaults>]

legion_create_object_r

{-l <class LOID> | -c <class context path>}

<context path> <host name> <host architecture> <$LEGION> <$LEGION_OPR> <$LEGION_OPA> <binary path> [<user id>]

legion_deactivate_object {-l <object LOID> | -c <context path>}

legion_destroy_object {-l <object LOID> | -c <context path>}

legion_list_instances {-l <class LOID> | -c <context path>}

legion_get_host {-l <object LOID> | -c <object context path>}

legion_get_vault {-l <object LOID> | -c <object context path>}

legion_set_host {-l <object LOID> | -c <object context path}

{-l <host LOID> | -c <host context path>}

legion_set_vault {-l <object LOID> | -c <object context path}

{-l <vault LOID> | -c <vault context path>}

Calls on LegionClass

legion_add_class_mapping <metaclass LOID> <class LOID>

legion_create_implementation <binary path name> <architecture>

[-l <class LOID> | -c <context path>]

legion_init_arch

Calls on File and Context Objects

legion_activate_instances {-c <class context path> | -l <class LOID>}

legion_cat <context path>

legion_context_add <object LOID> <context name>

legion_context_create <context path>

legion_context_list [<context path>]

legion_context_lookup <context path>

legion_context_remove <context path>

legion_cp [-localsource | -localdest] <source file> <destination file>

legion_deactivate_instances

{-c <class context path> | -l <class LOID>}

legion_destroy_instances {-c <class context path> | -l <class LOID>}

legion_direct_output {-c <object path> | -l <LOID>}

{-c <tty path> | -l <tty LOID>}

legion_get_host {-c <object context path> | -l <object LOID>}

legion_get_vault {-c <object context path> | -l <object LOID>}

legion_import_tree [<unix directory path> [<legion context path>]]

legion_ln <context path> <new alias>

legion_ls [-lLa] <context path>

legion_mv <context path> <new context path>

legion_rm [-deactivate | -destroy] <context path>

legion_set_context <context path>

legion_set_tty <tty context path>

legion_tty_redirect <context path>

legion_tty_unredirect <context path>

legion_tty_watch {-c <tty context path> | -l <tty LOID>}

Start-Up and Shutdown Functions

legion_create_class [-c < context path>]

legion_initialize

legion_print_config

legion_setup_state [-i]

legion_shutdown [-local] [-f] [-i] [-h]

legion_shutdown_class {-l <class LOID> | -c <context path>}

legion_starthost [-L <$LEGION>] [-O <$LEGION_OPR>]

[-A <$LEGION_ARCH>] [-N <context name>] [-U <user id>]

{<new host name>} [<compatible vault list>]

legion_startup [-local]

legion_startvault [-L <$LEGION>] [-O <$LEGION_OPR>]

[-A <$LEGION_ARCH>] [-N <context name>] [-U <user id>]

{<host name>} [<compatible host list>]

Scheduling Support

legion_class_host_list [-c <class context path> | -l <class LOID>]

[{-a | -d} <host name>] [-p] [-u]

legion_class_vault_list [-c <class context path> | -l {<class LOID>}

[{-a | -d} <vault name>] [-p] [-u]

legion_get_candidate_hosts

{-c <object context path> | -l <object LOID>}

legion_get_candidate_placements

{-c <class context path> <instance context path>}

| {-l <class LOID> <instance LOID>} [<max placements>]

legion_get_candidate_vaults

{-c <object context path> | -l <object LOID>}

legion_host_vault_list {-c <host context path> |-l <host LOID>]

[{-a | -d} <vault name>] [-p] [-u]

legion_instance_host_list {-c <context path> | -l <LOID>}

[{-a | -d} <host name>] [-p] [-u]

legion_vault_host_list {-c <vault context path> | -l <vault LOID>}

[{-a | -d} <host name>] [-p] [-u]

General Functions About the State of the System

legion_classof {-c <context path> | -l <object LOID>}

legion_create_stat_tree <base context path>

legion_host_stats {-c <host context path> | -l <host loid>}

legion_list_objects {-c <host context path> | -l <host LOID>}

legion_wellknown_class [<wellknown class name>]

Security

legion_create_user <user id>

legion_get_acl [-l <loid> | -c <context>]

legion_get_implicit_params [-l <loid> | -c <context>]]

legion_login [-l <user LOID> | <user id>] [-e <command>]

legion_passwd {-l <user LOID> | <user id>}

legion_set_acl [-c <object context path> | -l <object LOID>]

{-s | filename}

legion_set_implicit_params

[-c <object context path> | -l <object LOID>] {-s | filename}

Application Development

legion_bfs <file> [-o <out file>] [-BackEnd]

legion_mplc

legion_java <java options>

Program Support

legion_mpi_debug [-q] {-c <instance context path>}

legion_mpi_register <class name> <binary local path name>

<platform type>

legion_mpi_run [-h <hosts context path>] [-p <PID context path>]

-n <number of hosts> <command> [<arguments>]

legion_pvm_register <class path name> <binary local path name>

<platform type>

legion_register_program <program class> <executable path>

<legion arch>

legion_register_runnable <program class> <executable path>

<legion arch>

legion_run [-help] [-w] [-a <architecture>]

[-in <context path name>] [-out <context path name>]

[-IN <local file name>] [-OUT <local file name>]

[-f <options file>] <program class> [<arg1>, <arg2>, ... <argn>]

More information on adding host and vault objects
Adding new hosts

Supported < flags > for legion_starthost are:

-L <$LEGION> Specify $LEGION for the new host

(default is "/home/xxxx/Legion")

-O <$LEGION_OPR> Specify $LEGION_OPR for host

(default is current $LEGION_OPR)

-A <$LEGION_ARCH> Specify the architecture type for the host

(default is current $LEGION_ARCH)

-N <context name> Specify the context name for the host object

(default is "/hosts/<new host name>")

-U <user id> Specify a Unix user id for host

(default is current Unix user id)

-C <host class> Specify the context path of the host class

(default is "/class/UnixHostClass")

This command first creates a new host object on the specified < new host name > (the host's DNS name). It selects the following additional default values for the new object:

<$LEGION_OPA> = $LEGION_OPR/Host-$HOST.OPA

<binary path> = $LEGION/bin/$LEGION_ARCH/UnixHostObject

An OPA (Object Persistent representation Address) is used to track a specific OPR (Object Persistent Representation, the inert state of an object): in this case, the OPA will represent the persistent state of the new host object. The binary path is the binary path of the program that will be run on the new host in order to start the new host object. Currently, this path can be altered only by editing the legion_start source file, in the $LEGION/src/CoreObjects/ClassObjects/rshStartClassObject directory.

These flags and defaults need to be carefully considered when adding new hosts and vaults to your system. If the new host has a different architecture or a different directory structure, use the -A , -L , or -O flags to specify these parameters. The -U flag allows you to specify a Unix user id for the new object, so that a system administrator can add host objects to another user on the same Unix system. This can be useful should you wish to create a guest user id that has limited access privileges to the new host. The -C flag allows you to start an instance of a different class, so that users can create new host classes and have more flexibility in managing their resources. (Legion currently comes with only one host class, UnixHostClass, but users can add more host classes as necessary, either by creating instances of the UnixHostClass or by writing new classes.)

If the new host object's architecture is not the same as the current host object's architecture, you should run the legion_init_arch tool in order to create implementation objects to match the new architecture (see section 9.0 of the System Administrator Manual, for information about implementations, and section 9.2 of the System Administrator Manual, for information about running legion_init_arch ).

Adding new vaults

Supported < flags > for legion_startvault are:

-L <$LEGION> Specify $LEGION for the vaults host

(default is "/home/Legion")

-O <$LEGION_OPR> Specify $LEGION_OPR for the vault's host

(default is "/home/OPR")

-A <$LEGION_ARCH> Specify the architecture of the vault's host

(default is current $LEGION_ARCH)

-N <context name> Specify the context name for the vault object

(default is "/vaults/vault-<host name>")

-U <user id> Specify a Unix user id

(default is current Unix user id)

-C <vault class> Specify the context path of the vault class

(default is "/class/UnixVaultClass")

This commands creates a new vault object on the storage system of a specified host (named in < host name >). The flags are similar to the legion_starthost flags. The -L , -O , or -A flags can be used to specify a different architecture or a different directory structure. The -N flag allows you to specify a context name. The -U flag allows you to specify a Unix user id for the new object, so that a system administrator (or anyone with root privileges) can add vault objects to another user on the same Unix system. This can be useful should you wish to create a guest user id that has limited access privileges to the new host. The -C flag allows you to start an instance of a different class. That is, if you do not use this flag, the new host object will be an instance of class UnixVaultClass. This flag allows users to create new vault classes, so as to give users more flexibility in managing their resources.

Manipulating host-vault pairing

The legion_host_vault_list command will list a host object's list of compatible vaults. The example below lists the compatible vaults for the BootstrapHost. Note the -p flag, which signals that the list should be printed out.

$ legion_host_vault_list -c hosts/BootstrapHost -p

** COMPATIBLE VAULT LISTING:

** 1.01.03.3cb53908.000001fc0bb4fef12ecf6cc...

** 1.01.03.3db53908.000001fc0dd5621fadf70b0...

** 1.01.03.3eb53908.000001fc0d6e9041e262126...

$

There are three vaults listed here: if you are not sure what the context names of these vaults are, use the legion_context_lookup command.

To view the list of a vault's compatible hosts, use legion_vault_host_list . The example on the next page asks for the list of BootstrapVault's compatible hosts.

$ legion_vault_host_list -c vaults/BootstrapVault -p

** COMPATIBLE HOST LISTING:

** 1.01.07.3cb53908.000001fc0c29636eee98d...

** 1.01.07.3eb53908.000001fc0d9b155044fb5...

$

There are two hosts listed here, BootstrapHost and aNewHost .

Both of these commands can add and delete hosts or vaults from the list with specific flags ( -a and -d , respectively). For example, to remove aNewVault from BootstrapHost 's list of acceptable vaults, you would enter the following:

$ legion_host_vault_list -c hosts/BootstrapHost -d \

vaults/aNewVault -p

Deleted 1 vault(s) to host's compatibility set

** COMPATIBLE VAULT LISTING:

** 1.01.03.3cb53908.000001fc0bb4fef12ecf6cc...

** 1.01.03.3db53908.000001fc0dd5621fadf70b0...

$

The next example adds a host to BootstrapVault 's list of compatible hosts.

$ legion_vault_host_list -c vaults/BootstrapVault -a \

hosts/AnotherHost -p

Added 1 host(s) to vault's compatibility set

** COMPATIBLE HOST LISTING:

** 1.01.07.3cb53908.000001fc0c29636eee98d...

** 1.01.07.3eb53908.000001fc0d9b155044fb5...

** 1.01.07.3fb53908.000001fc0c96beaba5730...

$

Getting help

In the future, a FAQ page listing common Legion problems will be posted on the Legion web site, but for the time being please contact us at one of the addresses listed below if you have trouble with the system. Please be sure to include any relevant information about the state of your system before and during the problem.

We would greatly appreciate hearing from you whenever you find errors or bugs in Legion, so that we can avoid similar problems in future releases.

E-mail

For help and general Legion information, please send an e-mail message to legion-help@virginia.edu

Contact Information

Legion Research Group

Department of Computer Science

Olsson Hall

University of Virginia

Charlottesville, VA 22903

http://legion.virginia.edu/

fax: 804-982-2214

On-Line Help

A variety of technical notes, reports, and on-line tutorials are available on the Legion web site, at <http://legion.virginia.edu/>.

References

Several of the Legion-related papers listed in this bibliography are available on the Legion web site, at <http://legion.virginia.edu/>.

  1. A.J. Ferrari, M.J. Lewis, C.L. Viles, A. Nguyen-Tuong, and A.S. Grimshaw, "Implementation of the Legion Runtime Library," University of Virginia CS Technical Report CS-96-16 , November, 1996.
  2. A. Geist, A Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V.S. Sunderam, PVM: Parallel Virtual Machine , MIT Press, 1994.
  3. A.S. Grimshaw, W.A. Wulf, "The Legion Vision of a Worldwide Virtual Computer," Communications of the ACM , pp. 39-45, Vol. 40, No. 1, 1997.
  4. A.S. Grimshaw, W.A. Wulf, "Legion--A View from 50,000 Feet," P roceedings of the Fifth IEEE Symposium on High Performance Distributed Computing , IEEE Computer Society Press, Syracuse, NY, August 6-9, 1996.
  5. A.S. Grimshaw, "Object-Oriented Parallel Processing with Mentat." Postscript version available on the Legion web site.
  6. A.S. Grimshaw, A. Nguyen-Tuong, W.A. Wulf, "Campus-Wide Computing: Results Using Legion at the University of Virginia," University of Virginia CS Technical Report CS-95-19 , March 27, 1995.
  7. A.S. Grimshaw, W.A. Wulf, J.C. French, A.C. Weaver, and P.F. Reynolds Jr., "Legion: The Next Logical Step Toward a Nationwide Virtual Computer," University of Virginia CS Technical Report CS-94-21 , June 8, 1994.
  8. A.S. Grimshaw, J.B. Weissman, E.A. West, and E. Loyot, "Metasystems: An Approach Combining Parallel Processing and Heterogeneous Distributed Computing Systems," Journal of Parallel and Distributed Computing , pp. 257-270, Vol. 21, No. 3, June 1994.
  9. A.S. Grimshaw, W.A. Wulf, J.C. French, A.C. Weaver, P.F. Reynolds Jr., "A Synopsis of the Legion Project," University of Virginia CS Technical Report CS-94-20 , June 8, 1994.
  10. A.S. Grimshaw, "Easy to Use Object-Oriented Parallel Programming with Mentat," IEEE Computer , May, 1993.
  11. J.F. Karpovich, "Support for Object Placement in Wide Area Heterogeneous Distributed Systems," University of Virginia CS Technical Report CS-96-03 , January 16, 1996.
  12. M.J. Lewis, A.S. Grimshaw, "Using Dynamic Configurability to Support Object-Oriented Languages and Systems in Legion," University of Virginia CS Technical Report CS-96-19 , December 4, 1996.
  13. M.J. Lewis, A.S. Grimshaw, "The Core Legion Object Model," Proceedings of the Fifth IEEE High Performance Distributed Computing , IEEE Computer Society Press, pp. 551-561, Syracuse, NY, August 6-9, 1996.
  14. V.S. Sunderam, PVM: A framework for parallel distributed computing, Concurrency: Practice and Experience , vol. 2(4), pp. 315-339, Dec. 1990.
  15. C.L. Viles, M.J. Lewis, A.J. Ferrari, A. Nguyen-Tuong, and A.S. Grimshaw, "Enabling Flexibility in the Legion Run-Time Library," Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA'97), pp. 265-74, Las Vegas, Nevada, June 30 - July 2, 1997.
  16. W.A. Wulf, C. Wang, D. Kiezle, "A New Model of Security for Distributed Systems," University of Virginia CS Technical Report CS-95-34 , August, 1995.

Index

A

add a new host 23

add a new vault 25

add objects on new hosts 29

architectures available in Legion 51

AuthenticationObject 7

B

bootstrap host 22

bootstrap vault 22

BootstrapHost 11

BootstrapVault 11

C

commands

legion_activate_instances 60

legion_activate_object 59

legion_add_class_mapping 60

legion_bfs 63

legion_cat 20, 60

legion_class_host_list 61

legion_class_vault_list 62

legion_classof 62

legion_context_add 19, 60

legion_context_create 18, 60

legion_context_list 15, 16, 41, 60

legion_context_lookup 17, 60

legion_context_remove 19, 60

legion_cp 20, 60

legion_create_class 61

legion_create_implementation 60

legion_create_object 29, 51, 59

legion_create_object_r 59

legion_create_stat_tree 62

legion_create_user 9, 62

legion_deactivate_instances 60

legion_deactivate_object 59

legion_destroy_instances 60

legion_destroy_object 59

legion_direct_output 60

legion_get_acl 62

legion_get_candidate_hosts 62

legion_get_candidate_placements 62

legion_get_candidate_vaults 62

legion_get_host 60

legion_get_implicit_params 62

legion_get_interface 59

legion_get_vault 60

legion_host_stats 59, 62

legion_host_vault_list 62, 65

legion_import_tree 20, 61

legion_init_arch 60

legion_initialize 61

legion_instance_host_list 62

legion_java 31, 63

legion_list_attributes 59

legion_list_instances 41, 59

legion_list_objects 62

legion_ln 18, 19, 61

legion_login 8, 9, 62

legion_ls 16, 61

legion_mpi_debug 63

legion_mpi_register 43, 44, 63

legion_mpi_run 44, 63

legion_mplc 63

legion_mv 19, 61

legion_passwd 9, 62

legion_ping 59

legion_print_config 61

legion_pvm_register 42, 63

legion_register_program 49, 63

legion_register_runnable 50, 63

legion_rm 19, 61

legion_run 51, 63

legion_set_acl 9, 62

legion_set_context 18, 61

legion_set_context.sh 18

legion_set_host 60

legion_set_implicit_params 9, 62

legion_set_tty 51, 61

legion_set_vault 60

legion_setup_state 61

legion_shutdown 61

legion_shutdown_class 61

legion_starthost 23, 61, 64

legion_startup 61

legion_startvault 25, 61, 65

legion_tty_redirect 61

legion_tty_unredirect 61

legion_tty_watch 51, 61

legion_update_attributes 59

legion_vault_host_list 62, 65

legion_wellknown_class 62

context space

about 12

assign context name to a LOID 19

change the working context 18

commands for working in 15

copy a Legion file object 20

copy a local file to a Legion file object 20

import local Unix directory tree 20

list a sub-context 15

look up a LOID 17

make a new sub-context 18

multiple context names for one object 18

naming objects 14

newly installed 10

organization 13

remove context name 19

rename an object 19

same name in different contexts 19

view a file object's contents 20

F

Fortran applications 39

G

GUI

about 31

Context Manager 31

icons 34

menus 36

on Windows95 32

restart 33

shut down 33

using 33

H

host 22

host object 22

hosts and vaults pairing 27, 65

host-vault compatibility 27

I

instance placement on hosts and vaults 29

L

LOID 40

about 12

and context names 12

M

MPI

applications 39

debugging support 48

functions 48

interface 43

library 43

makefile 43

MPI functions supported in Legion 48

O

OPA 64

OPR 64

P

PVM

applications 39, 40, 41

legion_pvm_register 41

Legion-PVM library 40

Legion-PVM tasks 40

library 40

makefile 41

PVM-specific context 41

Tids 40

Tid-to-LOID mapping 40

R

remote program execution

about 49

compatible programs 50

converting C/C++ programs to runnable objects 52

non-compatible program 49

running 51

setting command-line arguments 52

setting input and output Legion files 51

setting input and output local files 52

specifying program's platform 51

RSA public key 13

T

tty objects

creating a tty object 51

setting a tty object 51

U

user id

change password 9

change user information 8

create new 8

implicit parameters 9

V

vault 22

vault object 22


1. It is actually not always necessary to have a Legion system running in order to use Legion: some Legion hosts can run in "consumer mode." Consumer mode hosts do not require the full set of Legion system binaries to be installed and running; a subset of binaries can access a Legion system that runs on different remote hosts, and can potentially even use that system to execute parts of Legion programs. A consumer mode host cannot itself be used to carry out parts of Legion programs unless those programs are started directly on that host by mechanisms outside of Legion (for example, from a shell running on the host's operating system).

2. Legion uses a the Unix-style slash ("/") in the default context space path names, so the examples here will follow that style. However, it is possible to change this to the DOS-style backslash ("\") via the primitive context manipulation routines provided by the Legion library.

3. The home context being discussed here should not be confused with the sub-context called home , one of the four basic sub-contexts provided in a new Legion system.

4. There is another context that represents the user's shell, but it does not appear in object path names nor does it affect the user's movements in context space.

5. It is important to note that this default path service provided by the Legion library is completely replaceable. For example, if a user wished to manipulate context spaces via DOS-style "\" delimited path names, the service could easily be implemented using the primitive context manipulation routines provided by the Legion library.

6. Full path syntax can be used as well as relative path syntax in all Legion commands.

7. In this and some of the following examples the entire LOID is not reproduced, in the interests of space and readability.

8. An alternative procedure, using command-line utilities, is explained in the Reference Manual on See Starting extra hosts and vaults from the command line . If possible, we recommend using the rsh procedure explained here, since it is faster and easier.

9. If you use the -h flag in this way, be sure to double-check the order in which the host objects are listed in your context.

10. Note that this file should not include either the <program class> parameter or the [< arg1 >, < arg2 >, ... < argn >] arguments: those should be listed on the command line.