Legion PVM cookbook Cookbook:
Running a PVM code in Legion with the fewest changes

PVM (Parallel Virtual Machine) is a software package designed to link up a heterogeneous collection of Unix and NT machines into a network that functions as a virtual single parallel computer. For more information, please see the web site, http://www.epm.ornl.gov/pvm/.

The current Legion implementation supports a core PVM interface, which includes functions for data marshaling, message passing, and task creation and control. Legion supports legacy PVM codes, and provides an enhanced PVM environment using Legion features such as security and placement services. For more information, please see the Basic User manual.

Table of Contents
Modifying the Program
Running this Legion-PVM Program
Limitations of this "fewest changes" approach
Modifying the program for the Typed Binary Interface
Other on-line tutorials & documentation
Click on the to move to the selected text.

Depending on how your system is set up, you may need to set up your access to your system before you can run Legion commands. This will probably involve running a command such as this: 
$ . ~legion/setup.sh
     or
$ source ~legion/setup.csh
The exact syntax will depend on what kind of shell you are using and where your Legion files are installed (i.e., the value of ~legion will depend on your individual Legion net). Consult your system administrator for more information.

The following style conventions are used in these tutorials:

Modifying the Program

Consider the following PVM program stub, which runs a binary named example: 

           program foo

           implicit none
           include 'fpvm3.h'
           integer info, mytid, myparent, numprocs, numret
           integer tids(0:MAXNCHILD)
           logical master
           integer i, j, k

c      Get numprocs somewhere

           call pvmfmytid( mytid )
           call pvmfmyparent( tids(0) )
           
           if ( tids(0) .eq. PVMNOPARENT ) then
               master = .true.
               call pvmfspawn( 'example', PVMDEFAULT, '*', 
    &                           numprocs, tids, numret)
           endif

           if ( master ) then
              open (10, file = 'input', status = 'old')
              read (10,*) i, j, k
              close (10)
           endif

           call do_work (i, j, k)

           if ( master ) then
              open (11, file = 'output', status = 'new' )
              write (11, *) i, j, k
              close (11)
           endif

           call pvmfexit(info)
           stop
           end
The subroutine do_work() represents the computational part of the program.

In order to run this program over Legion and get the benefit of remote I/O, it is necessary to insert extra I/O calls. The PVM calls do not change. Here is the result, in which the changes are in upper case:

           program foo

           implicit none
           include 'fpvm3.h'
           integer info, mytid, myparent, numprocs, numret
           integer tids(0:MAXNCHILD)
           logical master
           integer i, j, k
           CHARACTER*256 INPUT, OUTPUT

c      Get numprocs somewhere

           call pvmfmytid( mytid )
           call pvmfmyparent( tids(0) )
           
           if ( tids(0) .eq. PVMNOPARENT ) then
               master = .true.
               call pvmfspawn( 'example', PVMDEFAULT, '*', 
    &                           numprocs, tids, numret)
           endif

           if ( master ) then
              CALL LIOF_LEGION_TO_TEMPFILE ('input', INPUT, info)
              open (10, file = INPUT, status = 'old')
              read (10,*) i, j, k
              close (10)
           endif

           call do_work (i, j, k)

           if ( master ) then
              CALL LIOF_CREATE_TEMPFILE (OUTPUT, IERR)
              open (11, file = OUTPUT, status = 'new' )
              write (11, *) i, j, k
              close (11)
              CALL LIOF_TEMPFILE_TO_LEGION (OUTPUT, 'output', IERR)
           endif

           call pvmfexit(info)
           stop
           end
We added four lines and changed two.

              CHARACTER*256 INPUT, OUTPUT
allocates space where we will store the local names of files we will read and write remotely.

              call LIOF_LEGION_TO_TEMPFILE ('input', INPUT, info)
copies the file with the Legion filename input into some local file, storing the name of that local file into the variable INPUT.

              open (10, file = INPUT, status = 'old')
opens this local copy of the file. We may then do normal Fortran I/O on this local file.

              call LIOF_CREATE_TEMPFILE (OUTPUT, IERR)
creates the local output file, storing the name of the file in the variable OUTPUT.

              open (11, file = OUTPUT, status = 'new' )
opens this local copy of the file. We may now do normal Fortran I/O to this local file.

              call LIOF_TEMPFILE_TO_LEGION (OUTPUT, 'output', IERR)
copies the local file OUTPUT into the Legion file with the name output.

Running this Legion-PVM Program

In order to run this program under Legion PVM, we need to:

  1. Compiling and linking

    Assuming that our program is in the file example.f, we would compile and link using:

     % f77 -c example.f -I$LEGION/include/pvm3.3 \
  legion_link -Fortran -pvm -o example example.o
  2. Setting up access to the Legion system

    Before you issue Legion commands, you may need to execute an extra command:

    % . /home/appnet/setup.sh
    or 
    % source /home/appnet/setup.csh
    Also, before you run a Legion program, it is useful to have a Legion tty, which is an object which allows you to observe the output of your program.

    % legion_tty my_tty
    This creates a tty with the name my-tty in Legion space, and send the output from this tty object to your current terminal.

  3. Tell Legion where the executable and input file are:

    • executable 
         % legion_mpi_register ./example example $LEGION_ARCH
      This command registers the Unix file ./example as the PVM Class example. Legion will create a class object, named /pvm/tasks/example, in Legion space.

    • input file 
         % legion_cp -localsource ./input input
      This command creates a Legion FileObject with the name input, and copies the Unix file with the name ./input into it.

  4. Run the program

    Now you may run the program:

    % legion_run_pvm /pvm/tasks/example
    This command will run your program somewhere in the Legion system. Any output sent to the screen will be sent to your tty object.

  5. Copy the output file back to our workstation

    Finally, we might want to copy the output file named output in Legion space back to our local workstation.

    legion_cp -localdest output ./output
Limitations of this "fewest changes" approach

While this approach allows you to run PVM programs and transparently read and write files remotely, it does have one limitation: it does not support heterogeneous conversions of data. If you run this program on several machines which have different formats for an integer, such as Intel PCs (little-endian) and IBM RS/6000s (big-endian), the result of using unformatted I/O will be surprising. If you want to use such a heterogeneous system, you will have to either use formatted I/O (all files are text) or use the "typed binary" I/O calls instead of Fortran READ and WRITE statements. These "typed binary" I/O calls are discussed in "Buffered I/O Library, low impact interface," in the Legion Developer Manual.

Modifying the program for the Typed Binary Interface

Here's how the program would change if we used the typed binary interface:

           program foo

           implicit none
           include 'fpvm3.h'
           integer info, mytid, myparent, numprocs, numret
           integer tids(0:MAXNCHILD)
           logical master
           integer i, j, k
           integer fd, ierr

c      Get numprocs somewhere

           call pvmfmytid( mytid )
           call pvmfmyparent( tids(0) )
           
           if ( tids(0) .eq. PVMNOPARENT ) then
               master = .true.
               call pvmfspawn( 'example', PVMDEFAULT, '*', 
    &                           numprocs, tids, numret)
           endif

           if ( master ) then
              CALL LIOF_OPEN ('input', 0, FD)
              CALL LIOF_READ_INTS (FD, I, 1, IERR)
              CALL LIOF_READ_INTS (FD, J, 1, IERR)
              CALL LIOF_READ_INTS (FD, K, 1, IERR)
              CALL LIOF_CLOSE (FD, IERR)
           endif

           call do_work (i, j, k)

           if ( master ) then
              call LIOF_OPEN ('output', 0, FD)
              call LIOF_WRITE_INTS (FD, I, 1, IERR)
              call LIOF_WRITE_INTS (FD, J, 1, IERR)
              call LIOF_WRITE_INTS (FD, K, 1, IERR)
              call LIOF_CLOSE (FD, IERR)
           endif

           call pvmfexit(info)
           stop
           end
Running the program is the same as in the previous example.

Other relevant on-line documents:
Click on the to go to the page.
Logging in to a running Legion system
Introduction to Legion context space
Context-related commands
Legion tty objects
Running a PVM code in Legion
Running a Legion MPI code
Running native MPI code 		Quick list of all 1.7 Legion commands
Usage of all 1.7 Legion commands
FAQs for running programs in Legion
Starting a new Legion system
Legion security
Legion host and vault objects
Adding host and vault objects
Brief descriptions of all on-line tutorials

Last modified: Thu Jun 15 16:18:40 2000

 

[Home] [General] [Documentation] [Software]
[Testbeds] [Et Cetera] [Map/Search]

legion@Virginia.edu
http://legion.virginia.edu/