2.10 Program support

legion_link 
     [-CC <compiler>] [-Fortran] [-FC <compiler>] [-pvm] [-mpi] 
     [-L<library path>] [-l<library>] [-v] [-o <output file>] 
     [-bfs] <object file list> [-debug] [-help]

This command links together a set of object files and libraries to produce an executable program and automatically binds the produced executables to the Legion libraries. It is similar to the Unix ld command. It can be used with object files and libraries created from C, C++, or Fortran source code.

If any Fortran object files are linked, the -Fortran flag must be included. If you specify a Fortran compiler with the -FC flag, the -Fortran flag is automatically implied.

The following options are available with this command:

-CC <compiler>

Select a C++ compiler to perform linkage. The default value is based on the default compiler used by the Legion system on your platform.

-Fortran

This flag must be included if any Fortran object files are linked.

-FC <compiler>

Select a Fortran compiler. This flag implies the -Fortran flag, but the two can be used together. Currently supported <compiler> options are:

fort

Compaq Fortran compiler (Alpha Linux only)

g77

Gnu g77

pgf77

Portland Group pgf77 compiler (Intel Linux)

pgf90

Portland Group pgf90 compiler (Intel Linux)

-pvm

Link the produced executable to the Legion PVM compatibility library.

-mpi

Link the produced executable to the Legion MPI compatibility library.

-l<library path>

Include the <library path> in the set of directories searched for libraries.

-l<library>

Link against the specified library.

-v

Provide a verbose output as the command is running.

-o <output path>

Specify the local path of the resulting program. The default is a.out.

-bfs

Link the produced executable to the Legion Basic Fortran Support (BFS) library.

-debug

Catch and print Legion exceptions.

-help

Print command syntax and exit.

legion_mpi_debug 
     [-q] {[-c] <instance context path>} [-help]

A utility program that allows the user to examine the state of MPI objects and print out their message queues. This is a handy way to debug a deadlock. There are a few limitations: if an MPI object doesn't respond, the command will hang and not go on to query additional objects. An MPI object can only respond when it enters the MPI library; if it is in an endless computational loop, it will never reply. Output goes to stdout.

The following option is available with this command:

-q

List the contents of the queues.

-help

Print command syntax and exit.

legion_mpi_register 
     <class name> <binary local path name> <platform type> [-help]

MPI implementation generally require that MPI executables reside in a given place on a disk. Legion's implementation objects have a similar requirement. This command registers the executables of different architectures for use with Legion's MPI. It creates MPI-specific contexts, a class object, and an implementation for the program, and registers the name in Legion context space. If Legion security has been enabled the newly created contexts will be placed in the user's /home/<user_name>/mpi context except if the user is admin or not logged in. If Legion security has not been enabled, the newly created contexts will be placed in the /mpi context.

The command can be executed several times, if you have compiled your program on several architecture.

The following option is supported:

-help

Print command syntax and exit.

legion_mpi_run 
     {-f <options file> [<flags>]} |
  {-n <number of hosts> [<flags>] <command> [<arguments>]} [-help]

Starts an MPI program. The <command> argument requires the full context path for the class (the class is created through legion_mpi_register: if you have not previously registered the program, you'll need to do so before you can run it). Parameters used for this command are:

-f <options file>

Allows users to start multiple MPI applications via an options file. All applications must be registered with Legion and have a common MPI_COMM_WORLD.

-n <number of processes>

Specify the number of hosts on which the program will run.

Supported <flags> are1:

-v

Verbose option (up to four can be specified for increased detail).

-h <host set context path>

Specify the set of hosts on which the program will run. The context path should not resolve to a particular host but to a context containing the names of one or more hosts. The default setting is the system's default placement: i.e., the system will pick a compatible host and try to run the object. If it fails the system will try another compatible host.

<host context name>

Run the first process (i.e., process zero) on this node.

-d <Unix path name>

Specify that all children change to the specified directory before they begin to run.

-hf <specification file context path>

Use a specification file from context space for scheduling objects from the current execution. The file must contain a list of hosts and the number of Legion objects that can be scheduled on those hosts.

You can create the file with legion_make_schedule or by hand (see below).

All of the objects from this Legion MPI run will be created using vector creates.

-A

Pass the files in the -std*/-STD* parameters to all nodes. The default setting gives these files only to node 0.

-a

Pass the files in the -in/-out/-IN/-OUT parameters to all nodes. The default setting gives these files only to node 0.

Files in the -out/-OUT parameters will be copied to files that use the following conventions: <name>. <mpi object id>, where <name> is either the <context path> or <local file name> specified in the -out/-OUT parameters.

-in <context path>

Context path of a Legion basic file object whose contents should be copied into the remote program's current working directory before execution begins. The local file will have the same name as the basic file object. This flag can be used multiple times to indicate multiple input files.

-out <context path>

Context path of a Legion basic file object whose contents should be copied out of the remote program's current working directory after the program terminates. The new local file will have the same context name as the original file object (unless the -a flag is used, in which case the name will be <context path>.<mpi object id>). This flag can be used multiple times.

The output files will be copied out when the program calls MPI_Finalize or MPI_Abort. Output files that are not found in the program's current working directory will be copied but will appear as empty files. If the program crashes, the files will be left in the program's current working directory.

-stdin <context path>

Map stdin to the Legion file object named in <context path>.

-stdout <context path>

Map stdout to the Legion file object named in <context path>. The local object will have the same context name as the original file object unless the -A flag is used, in which case the name will be <context path>.<mpi object id>.

-stderr <context path>

Map stderr to the Legion file object named in <context path>. The local object will have the same context name as the original file object (<context path>.<unique id>) unless the -A flag is used, in which case the name will be <context path>.<mpi object id>.

-HF <specification file>

Use a specification file from local file space for scheduling objects from the current execution. The file must contain a list of hosts and the number of Legion objects that can be scheduled on those hosts.

You can create the file with legion_make_schedule or by hand (see below).

All of the objects from this Legion MPI run will be created using vector creates.

-IN <local file name>

Similar to the -in option, but operates on a file in the local execution environment of legion_mpi_run (i.e., the file named in <local file name>). This flag can be repeated.

-OUT <local file name>

Similar to the -out option, but operates on a file in the local execution environment of legion_mpi_run (i.e., the file named in <local file name>). This flag can be repeated.

-STDIN <local file name>

Map stdin to the local file named in <local file name>.

-STDOUT <local file name>

Map stdout to the local file named in <local file name>.

-STDERR <local file name>

Map stderr to the local file named in <local file name>.

-S

Print statistics at exit.

-p <PID context>

Specify a context path for PIDs. The default is /mpi/instances/ <program_name>, unless security is enabled. In that case the default is /home/<user_name>/mpi/ instances/<program_name>. If the context does not exist it will be created.

-D <variable_name>=<value>

Set the environment variable named in <variable_name> to a specified value on all MPI processes after they have called MPI_Init(). This option may be repeated multiple times to set additional environment variables.

-help

Print command syntax and exit.

If the -f flag is not used, the MPI application named in the legion_mpi_run command will be started with the given flags. An example, here running MPI program vdelay on two hosts, would be:

 $ legion_mpi_run -n 2 /mpi/programs/vdelay

Use legion_ls to examine the running objects of your application. The /mpi/instances/<program_name> context has an entry for each object in the specified application:

$ legion_ls /mpi/instances/vdelay

If you run multiple versions of an application simultaneously, you can use the -p flag to specify an alternate context in which to put the list of objects.

Specification file

If you use either the -HF or -hf flag you must name a specification file. You can use legion_make_schedule to produce this file or create it yourself. List one host and one integer indicating the number of objects the host can create (the default is 1) per line. List the host by its Legion context name. A host can be listed multiple times in one file, in which case the integer values accumulate. E.g.:

/hosts/BootstrapHost	5
/hosts/slowMachine1
/hosts/bigPBSqueue 	100
/hosts/slowMachine2 	1

Options file

The options file contains a list of all MPI applications that you wish to execute as well as any necessary arguments, as they would appear on the command line. All applications must have been registered (with legion_mpi_register) and use a common MPI_COMM_WORLD. The file must list one binary (and any arguments) per line. Each line can also contain one or more legion_mpi_run flags (but not -f). For example:

-n 2 /mpi/programs/mpitest
-n 3 /mpi/programs/mpitest_c

This would start a run of five instances (two of mpitest and three of mpitest_c).

If you use an options file, Legion will ignore the -n flag, program name, and any arguments given on the command line. Any <flags> will be treated as defaults and applied to all processes executed by this command (unless otherwise specified in the options file).

Fault tolerant mode

There is a set of special legion_mpi_run flags that can be used when running in a fault tolerant mode. These flags are used in specific combinations. You must use the -ft in all cases and you must use either -s or -R. The -g and -r flags are optional.

-ft

Turn on fault tolerance.

-s <checkpoint server>

Specifies the checkpoint server to use.

-R

Recovery mode.

-g <pingInterval>

Specify a ping interval. Default is 90 seconds.

-r <reconfigurationInterval>

Specify a period (in seconds) for detecting failure. If an object does not respond within the specified period, the application restarts from the last consistent checkpoint. Default value is 360 seconds.

Please see section 10.1.10 in the Basic User Manual for more information on running in fault tolerant mode.

legion_native_mpi_config_host [<wrapper>] [-debug] [-help]

This command sets native MPI properties on a host. The following options are supported:

<wrapper> Specify a wrapper script that locates mpirun on the host. The default specifies the legion_native_mpich_wrapper script, which is for an MPICH implementation.

-debug

Catch and print Legion exceptions.

-help

Print command syntax and exit.

legion_native_mpi_init [<architecture>] [-debug] [-help]

This command installs the legion_native_mpi_backend class in a native MPI host. The following options are supported:

<architecture>

Specify an architecture for which an implementation for this class can be registered.

-debug

Catch and print Legion exceptions.

-help

Print command syntax and exit.

legion_native_mpi_register 
     <class name> <binary path> <architecture> [-help]

Register a native MPI program with your Legion system. The example below registers /myMPIprograms/charmm (the binary path) as using a Linux architecture.

$ legion_native_ mpi_register charmm \ 
    /myMPIprograms/charmm linux

You can run register a program multiple times, perhaps with different architectures or different platforms. If you have not registered this program before, this will create a context in the current context path (the context's name will be the program's class name -- in this case, charmm) and registers the name in Legion context space.

The following option is supported:

-help

Print command syntax and exit.

legion_native_mpi_run 
     [-v] [-a <architecture>] [-h <host context path>]
     [-IN <local input file>] [-OUT <local result file>]
     [-in <Legion input file>] [-out <Legion result file>]
     [-n <nodes>] [-t <minutes>] [-legion] [-help] 
     <program context path> [<arg 1> <arg 2> ... <arg n>]

Start a native MPI program. The following parameters are used for this command:

-h <host context path>

Specify the host on which the program will run. The default setting is the system's default placement, which is to pick a compatible host and try to run the object. If the host fails, the system will try another compatible host.

-v

Verbose option.

-a <architecture>

Specify the architecture on which to run.

-IN <local input file>

Specify an input file that should be copied to the remote run from the local file system.

-OUT <local result file>

Specify a result file that should be copied back from the remote run to the local file system.

-in <Legion input file>

Specify an input file that should be copied to the remote run from the Legion context space.

-out <Legion result file>

Specify a result file that should be copied out from the remote run into Legion context space.

-n <nodes>

Specify the number of nodes for the remote MPI run.

-t <minutes>

Specify the amount of time requested for the remote MPI run. This option is only meaningful if the host selected to run the remote program enforces time limits for jobs.

-legion

Indicate whether the application makes Legion calls (see section 10.2.5 in the Basic User manual).

-help

Print command syntax and exit.

<arg1> <arg2> ... <argn>

Arguments to be passed to the remote MPI program.

legion_pvm_register 
     <class path name> <binary local path name> <platform type> [-help]

Registers a PVM task implementation. This setup is not necessary for tasks that will only be started from the command line (tasks that will not be "spawned"). Given the class named in <class path name>, the binary named in <binary local path register>, and the architecture type (currently Linux, Solaris, or SGI), this command creates an implementation object that can then be used by the Legion PVM Library.

Once you've registered an application, you can run it. If necessary, you can examine Legion context space with either

$ legion_ls /pvm

to list the Tids of running PVM tasks, or

$ legion_ls /pvm/tasks

to list the registered task classes. You can also use Legion class utilities to examine the class state (e.g., legion_list_instances).

To register a Linux binary named matmult in Legion, enter:

$ legion_pvm_register /pvm/tasks/matmult matmult linux

The following option is supported:

-help

Print command syntax and exit.

legion_register_program 
     <program class> <executable path> <legion arch> 
     [-debug] [-help]

This command allows uses to register an independent (i.e., not linked to the Legion libraries) executable program (specified by the <executable path> argument) and make the program available for use within the Legion system. If the program was not previously registered, a new class object the registered program will be associated with a Legion class object, named in <program class>, the class and the context path will be created. The registered program will execute only on hosts of the architecture specified in <legion arch>.

Programs registered through legion_register_program can be executed with the legion_run command. See also legion_register_runnable for information about registering Legion programs (below).

The following parameters are used with this command:

<program class>

The Legion context path name of the class with which the registered program should be associated.

<executable path>

The local file path of the executable program to register. This can be any program that could be run from the shell command prompt, including scripts, and binary executable generated by any programming language.

<legion arch>

The platform type on which the program should be executed.

The following options are supported:

-debug

Catch and print Legion exceptions.

-help

Print command syntax and exit.

For more information please see "Registering independent programs" in the Basic User Manual.

legion_register_runnable
     <program class> <executable path> <legion arch> 
     [-debug] [-help]

The legion_register_runnable command is similar to the legion_register_program command in that it allows programs to be registered for execution through the legion_run utility. However, whereas the legion_register_program tool is used to register independent programs, legion_register_runnable is used to register Legion-linked programs: programs that are linked against the Legion libraries and export the "runnable" object interface.

The following parameters are used with this command:

<program class>

The Legion context space path of the class with which the registered program should be associated.

<executable path>

The local file path of the executable program to register. This program should be a valid Legion object implementation that has been linked with the Legion library, and that exports the Legion "runnable" interface.

<legion arch>

The platform type on which the program should be executed.

The following options are supported:

-debug

Catch and print Legion exceptions.

-help

Print command syntax and exit.

For more information please see "Registering Legion-linked programs" in the Basic User Manual.

legion_run 
     [-w] [-v] [-a <architecture>]
     [-in <context path name>] [-out <context path name>] 
     [-IN <local file name>] [-OUT <local file name>] 
     [-f <options file>] <program class> 
     [-t <minutes>] [-n <nodes>] [<arg1> <arg2> ... <argn>] 
     [-debug] [-help]

The legion_run command executes a single instance of a program associated with the program class specified in the <program class> argument. The command will randomly select a host to execute the program (observing the restriction that only hosts with an acceptable architecture may be selected). Support for directing scheduling (e.g. selecting a given host for remote execution) will be added in future releases. Arbitrary command-line arguments may be specified for the remote program.

Note that any number of input and output files may be specified for a single execution of legion_run (i.e., the -in/-out and -IN/-OUT options can be repeated).

The following parameters are used with this command:

<program class>

Specifies the program class of which an instance should be executed. This program class should have been previously created with legion_register_program or legion_register_runnable (above).

The following optional parameters are supported:

-w

Specifies that the set tty object should be used. If no tty object has been set, the flag will be ignored.

-v

Run command in verbose mode.

-a

Allows users to specify what kind of architecture the program should be executed on.

-in <context path name>

Context path of a Legion file object whose contents should be copied into the remote program's current working directory before execution begins.The local file will have the same name as the <context path name> basename.

-out <context path name>

Context path of a Legion file object whose contents should be copied out of the remote program's current working directory after the program terminates. The local source file will have the same name as the <context path name> base-name. Note that output files are copied out regardless of the cause of program termination. Partial results will be available if the program crashes. Output files that are not found in the program's current working directory are skipped.

-IN <local file name>

Similar to the -in option, but operates on a file in the local execution environment of legion_run (i.e., the file named in <local file name>).

-OUT <local file name>

Similar to the -out option, but operates on a file in the local execution environment of legion_run (i.e., the file named in <local file name>).

-f <options file>

Allows users to specify options for running legion_run in a separate file rather than listing them on the command line. This is useful for programs that make extensive use of the -in/-out or -IN/-OUT options. The file can contain one or more of the following, delimited by spaces, tabs, or blank lines: -help, -w, -v, -a, -in, -out, -IN, -OUT, -f, -t, -n. The program class name and any arbitrary command-line arguments may not be included in this file. No other information should be included.

-t <minutes>

Specifies the amount of time (in minutes) needed to run the program. If this option is not used, the host will assign the job its default time block. This option is only meaningful if the host selected to run the remote program enforces time limits for jobs: otherwise this option is not required.

-n <nodes>

If the remote program is a native parallel job (e.g., a program written to use the local vendor MPI implementation), use this option to specify the number of nodes that should be allocated to the job. This option is not meaningful if the host selected to run the remote program is a uniprocessor or does not support multi-node allocations.

<arg1> <arg2>... <argn>

Allows users to provide arbitrary command-line arguments for the program.

-debug

Catch and print Legion exceptions.

-help

Print command syntax and exit.

For more information on this command please see "Running a Legion application" in the Basic User Manual.

legion_run_multi 
     [-v] {-n <number of processors>}
     [-s <schedule file name>] {-f <specification file name>} 
     <program class name> [<arg1> <arg2> ... <argn>] 
     [-debug] [-help]

This command runs a previously registered serial program, named in <program class name>, with all of the different input files, using a simple specification file, named in <specification file name>, to describe the names of the expected input and output files.

You must have already registered the program name and created the specification file. The specification file might look something like this:

IN in.dat in.dat.*
constant const.dat
OUT out.dat

Where IN identifies the line as a description of the input files (so it is a keyword): in.dat is the file name that the serial program expects for its input file. The final string, in.dat.*, is a pattern that the command will use to match all the different input files in the current directory that will be individually fed to different runs of the serial program. The constant keyword names a single input file that is fed to each run. OUT is a keyword indicating the output file specification, and out.dat is the name of the output file from the serial program. If in.dat.123 is an input file, the command will direct the output to out.dat.123.

The keywords' case determines the file's location:

CONSTANT/IN/OUT/stdout/stderr/stdin

local file space

constant/in/out

Legion context space2

In addition, you can use the stdout keyword to denote a file that will receive standard output from each serial run (i.e., you will end up with the same number of files as serial runs). Similarly, you can use stderr to denote a file that will receive standard error and stdin to denote a file that will provide standard input for the program.

stdout stdout.dat

If you do not use the stdout keyword, standard output will be sent to your Legion tty object (if you have not yet created a tty object, see legion_set_tty).

For example, with the following specification file:

IN input input.*
constant constant.dat
OUT output output.*
console console

If you begin with the following files:

input.1 input.2 constant.dat

then the following output files will be produced:

output.1 output.2 console.1 console.2

The following parameters are used with this command:

<program class name>

The program's Legion context path (created when the program was registered in Legion).

-n <number of processors>

The number of processors that should run the program.

-f <specification file name>

The specification file's Unix path name.

The following options are available with this command:

-s <schedule file name>

Specify a Legion schedule for running the program. The schedule contains a list of Legion host context names, followed by an integer specifying how many jobs can be run at once on a host (e.g., /hosts/Foo 3 to run up to three jobs on host object Foo).

-v

Provide a verbose output as the command is running.

<arg1> <arg2> ... <argn>

Allows users to provide arbitrary command-line arguments for the program.

-debug

Catch and print Legion exceptions.

-help

Print command syntax and exit.


1. There is a set of special flags for running in a fault tolerant mode listed on page 73. Back

2. Please note that the in and out keywords are not functioning correctly in version 1.6.x. This will be fixed in the near future, but we advise that you do not use them. Please contact us at legion-help@virginia.edu for more information. Back

Directory of Legion 1.6.4 Manuals
[Home] [General] [Documentation] [Software]
[Testbeds] [Et Cetera] [Map/Search]

Free JavaScripts provided by The JavaScript Source

legion@Virginia.edu
http://legion.virginia.edu/