3.0 Starting a new system

3.1 Before you start

Before you start a new Legion system, you should consider what type of set-up will best suit your needs. Primary considerations are:

  • What kind of system do you need? How many machines do you anticipate using? Do you only use local hosts or do you also use remote machines? Possible configurations might include:
  • a. A single host system: one Legion machine with one or more host objects (Figure 1). This is the simplest system.

    Figure 1: Single host system

    b. A multihost system: multiple Legion hosts linked together and sharing local resources (Figure 2). This can include homogeneous or heterogeneous platforms, as well as non-Legion machines. The machines do not need to be in physical proximity.

    Figure 2: Multihost system

    c. A multidomain system: multiple Legion domains connected together and sharing each others' resources (Figure 3).

    Figure 3: Multidomain system

  • Will you be using Legion security? This is an important consideration, since a secure Legion system cannot be cleanly shut down and restarted. If you are using security, you should decide who will be the "admin," the system administrator. The system should be started up by the admin, since he or she will have special privileges on the core objects created in the new system. (These privileges do not extend to other users' objects, however.)
  • What kind of host objects will you be using? Options include:
  • a. Basic host object: it resides on its host and manages and guards its host's resources. This is the template for the other host objects.

    See Host and vault objects in the Basic User manual for discussion of basic host objects. Section 11.0 in this manual discusses host-vault pairings and adding new hosts.

    b. PCD host object: it resides on its host, manages and guards its host's resources, and uses a process control daemon to regulate ownership of all Legion processes executed on that host. If you use a PCD host as your bootstrap host, the start-up process will be slightly different.

    The daemon requires root privileges to start and to run. The PCD host object is useful if outside users will be running processes on your host, but can only be used if Legion security is enabled. Each user's processes will be tracked and accounted for. For more information, please see "Process control daemon host objects".

    c. Batch queue host object: it resides on its host, manages and guards its host's resources, and submits Legion jobs to the local queueing system.

    This is the best choice for hosts that use a queue management system, although the PCD host object is more secure and has better accounting. For more information, please see "Batch queue host objects".

    d. Virtual host object: it resides on a different host, represents and guards its host's resources, but does not run normal Legion objects. A virtual host cannot be used as a bootstrap host: it is added to an already running system.

    A virtual host object is used for running Legion jobs on unsupported platforms. The host object resides on a supported platform and runs native jobs with standard Legion tools on the target host machine. It can be used for scheduling, resource selection, and transparent execution on the target machine. For more information, please see "Virtual hosts".

      3.2 Set up the environment

      A properly set-up environment is crucial for working in the Legion system. The start-up process uses certain Legion-specific environment variables, which must be correctly set before starting applications and running command-line utility programs. You must set these variables each time you starting working in Legion. Without a properly set environment, programs cannot communicate with other objects in the system, and the program may terminate with an error, never return a value, or fail in a more spectacular fashion. If this occurs, try setting your environment properly and starting over.

      You must have /bin/ksh installed in your system. There are a number of Legion scripts that will look for ksh, and if it is not installed in your system you will get error messages.

      To prepare the environment in a newly started or restarted system, set the $LEGION and $LEGION_OPR variables and source the script that is appropriate for your shell.1

      (ksh or sh users)

      export LEGION=<Legion root dir path>
      export LEGION_OPR=<Legion OPR root dir path> 

      (csh users)

      setenv LEGION <Legion root dir path>
      setenv LEGION_OPR <Legion OPR root dir path>

      The environment must be properly set in each shell in which you plan to run Legion commands. Check to be sure that environment variables are properly set (we suggest $LEGION/../OPR for the OPR root directory path).

      If necessary, run the following.

      (ksh or sh users)

      . $LEGION/bin/legion_env.sh
      . $LEGION_OPR/legion_context_env.sh

      (csh users)

      source $LEGION/bin/legion_env.csh
      source $LEGION_OPR/legion_context_env.csh

      3.3 Starting a single host system

      If you have downloaded the source code, you must compile the system before you can start it (see section 2.5).

      There are three steps in this procedure: configure the system with legion_setup_state, start it with legion_startup, and, finally, initialize it with legion_initialize.

      3.3.1 Choose a bootstrap host

      You must first choose a bootstrap host. Start and shut down your system from here.2 The bootstrap host will have the following features:

      • The LegionClass object (the root of the Legion binding mechanism and the parent class of many metaclasses) will be placed here.
      • The $LEGION_OPR/LegionClass.config file will be created on this host and will contain the LegionClass object address, which must be globally known to all Legion objects. The file must be available when other objects are started in the system.

      If you choose a PCD host as your bootstrap, the start-up procedure is slightly different than for a basic host object.

      3.3.2 Set up and configure the system

      You must first set up the initial state for core Legion system objects. Legion system objects are persistent, and can save and restore their own state. Some of these objects must have their state initialized before they run for the first time. After the initial start-up, these objects will manage their own state and configuration, if the system is properly maintained.

      If you are booting on a PCD host, first run:

      $ LEGION_HOST_BIN=PCDUnixHost

      Enter the legion_setup_state command to configure the system:

      $ legion_setup_state

      This program will return your start-up host name, a port number for the LegionClass object, and a time. If you do not want to use the default settings, use the -i flag to run the command in an interactive mode. Your output will look something like this:

      $ legion_setup_state
      Creating OPR directory /home/xxx/OPR/.
      Saving LegionClass configuration file: 
      	/home/xxx/OPR/LegionClass.config
      LegionClass host name   = your.startup.host.name
      LegionClass port number = 7899
      LegionClass timestamp   = 898198093
      $
      

      The script creates the $LEGION_OPR directory and several sub-directories, populating them with initial states for several core system objects. The timestamp sets the starting time for the system: Legion objects use a timestamp to guarantee each object's unique identity. The current time is measured in seconds since January 1, 1970.

      3.3.3 Start the system

      The legion_startup script provides prompts asking whether or not to start each component. It's best to answer "yes to all" (Y). The verbose option allows you to see more detailed information, as the script works, about debugging. (This can be large amounts of information, so use this option only if you are searching for a problem.)

      To start the main core system objects, enter the legion_startup command.

      $ legion_startup

      Legion will start several classes on your host. Your output will look something like this

      $ legion_startup
      Starting meta-class object: LegionClass
      Continue (y=yes, Y=yes to all, n=no, N=no to all,
      	v=verbose, V=verbose all)? Y
      Starting meta-class object: BootstrapMetaClass
      Starting class object: BindingAgentClass
      Starting class object: CommandLineClass
      Starting class object: UnixHostClass
      Starting class object: UnixVaultClass
      Starting class object: UnixImplementationClass
      Starting class object: UnixImplementationCacheClass
      Starting class object: ContextClass
      Legion first-time system startup complete
      Run "legion_initialize"
      $
      

      This indicates that the major class objects were started: LegionClass is the top-level metaclass and is the parent of every other object in the system. BootstrapMetaClass is the class object for the bootstrap class objects, which are those class objects that have instances that need to get created in the initialization phase; those class objects are UnixVaultClass, UnixHostClass, UnixImplementationClass, and UnixImplementationCacheClass.

      The new class objects will start instances in the new system: BindingAgentClass parents binding objects, CommandLine- Class parents command-line objects, etc.

      3.3.4 Initialize the system

      Like legion_startup, the legion_initialize script will provide prompts asking whether or not to perform each task, and it is generally best to use the "yes to all" option (Y). To initialize Legion, enter

      $ legion_initialize

      The output shows the system creating and tagging the key ingredients of a new system. It is too long to reproduce in full here, but we'll look at some selected actions.

      The first line shows the system creating a bootstrap host object on your current host (a host object manages a host, so the bootstrap host object manages the bootstrap host).

      Creating host object BootstrapHostObject on 
      	"your.current.host.name"
      Continue (y=yes, Y=yes to all, n=no, N=no to all,
                    v=verbose, V=verbose all)? Y
      Configuring wellknown binaries for host
      	"1.01.07.0100..."
      

      A bootstrap vault object is automatically created on your current host (a vault object manages a vault, which stores Legion object's permanent states). This guarantees that the bootstrap host object has a compatible vault object. All host objects must be paired with at least one compatible vault object (i.e., a vault that it can "see" -- see "About host-vault pairs").

      Creating vault object BootstrapVaultObject on
      	"your.bootstrap.host.name"
      Setting BootstrapHost and BootstrapVault restrictions
      Added 1 host(s) to vault's compatibility set
      Added 1 vault(s) to host's compatibility set
      

      Implementation objects represent and manage the implementation cache (used to allow Legion processes to take place in different architectures) and context space.

      Creating an ImplementationCache
      Creating an implementation (ContextObject) for ContextClass
      Creating the root context object
      

      More implementation objects are created as the process creates new object classes.

      Adding "BootstrapHost" to the hosts context
      Adding the alias "your.bootstrap.host.name" for
      	BootstrapHost to the hosts context
      Adding "BootstrapVault" to the vaults context
      

      Two context names are added to the /hosts context (BootstrapHost and your.bootstrap.host.name): both names refer to the Bootstrap host object (see Figure 4) but only one is added to the /vaults context.

      Figure 4: Context paths for the bootstrap host and vault objects

      This object will manage a portion of the persistent storage mechanism for the Bootstrap host (please see section 7.0 in the Basic User Manual for a discussion of Legion hosts and vaults). The /impls context contains names of the default implementation objects (see Implementation model). These can all be viewed with context-related commands or the GUI once the system has been completely started.

      3.3.5 Set the environment

      Source the script that sets up environment variables for using the context space.

      ksh or sh users type:

      $ . $LEGION_OPR/legion_context_env.sh

      csh users type:

      $ source $LEGION_OPR/legion_context_env.csh

      You must source this script in each shell in which you plan to run Legion commands, including the shell in which you ran legion_initialize. Otherwise you will not be able to run any Legion utilities or commands.

      3.3.6 Choose your security setting

      If you wish enable Legion security, run the legion_init_security command. You'll have to decide now whether or not you want security, since the command will not run properly unless you run it immediately after initializing the system. If you don't wish to use it, just skip over this section. However, none of your processes will be protected.

      To use Legion security, run:

      $ legion_init_security

      You will be given a system administrator user id called admin. The administrator has root privileges in the system and can create new users, modify security settings, etc. He/she also has ownership of all currently existing objects in the new system but not any other objects (i.e., any objects that other users create).

      Several events take place when you run this command, and we will discuss each in turn.

      Creating the context "/users" to contain user-objects
      Creating context "users" in parent "/".
      New context LOID = "1.3622260c.05.08000000.000001fc..."
      

      The /users context holds Legion user ids, starting with admin:

      Creating the initial system-admin user object, "/users/admin"
      Please select a Legion password for "/users/admin":
      New Legion password: xxxx
      Retype password: xxxx
      1.3622260c.6b000000.01000000.000001fc0c...
      Please enter the Legion password for "/users/admin" to 
      	continue:
      Password: xxxx
      

      This admin is the system administrator's user id: anyone logging in as admin has root privileges. You will have to enter the password three times during the legion_init_security process and when logging in.

      Changing ownership of all objects to "/users/admin"
      1.3622260c.01..000001fc0cbe1846763f895a...
      1.3622260c.02..000001fc0b3b16eb8b2dde29...
      [...etc.]
      Changed ownership of 43 objects.
      

      Legion assigns ownership of all existing objects to admin. Objects created by other users will belong to those users.

      Creating initial ACLs files for all core objects in
      	/home/xxxx/OPR/init_acls
      Creating ACL for /class/AuthenticationObjectClass class
      Creating ACL for /class/BasicFileClass class
      [...etc.]
      

      The access control lists (ACLs) protect objects against unauthorized use. Only an object's creator can use the object, unless the creator specifies otherwise. The initial ACL files allow only the admin to use the core objects.

      Creating context "tmp" in parent "/".
      New context LOID = "1.362dabd6.05.09000000.000001fc..."
      [...etc.]
      Done creating acl files
      

      Several new context objects are created and placed in the root context and given ACLs.

      Setting ACL for /class/AuthenticationObjectClass class
      [...etc.]
      All acls set.
      $

      ACL files are then created for all previously existing objects.

      At this point, security has been enabled and is running. You must now log in as admin. Note that you need to include the /users path.

      $ legion_login /users/admin
      Password: xxxx
      $

      You are now ready to start working. For more information, see "About Legion security" and "Using security features".

      3.4 Starting a multihost system

      This is a two-part process. First, you have to have a running single host system, as laid out in section 3.3. Second, you add new host objects on the desired machines. Since you will be starting processes on the target hosts from the bootstrap host be sure that you can run rsh/ssh on the bootstrap host as well as on the target hosts from the bootstrap host without having to enter a password. You can set up a .rhosts file for rsh or an authorized_keys files for ssh to accomplish this (see the rsh and ssh man pages for more information).

      3.4.1 Set up

      You'll need to set the proper environment variable on the bootstrap host and the remote host(s) so that you can run Legion commands on a remote host using rsh or ssh.

      For sh, ksh, or bash:

      LEGION_RSH=<rsh|ssh>
      LEGION_RCP=<rcp|scp>
      export LEGION_RSH LEGION_RCP

      For csh:

      setenv LEGION_RSH <rsh|ssh>
      setenv LEGION_RCP <rcp|scp>

      Set these variables on the bootstrap host before you start the new system (i.e., before you run legion_startup). Please note that you only need to follow these steps on the bootstrap host: you will need to install the Legion binaries on any other machines that you add to your system, but you do not need to start more Legion systems.

      To add additional hosts and users, copy the $LEGION_OPR/setup.[sh|csh] scripts to a globally accessible location. Hosts can share an NFS-mounted Legion tree, but for best results you should place the OPRs on a local disk.

      3.4.2 Create a new vault

      If the new host will not be compatible with your existing vaults, create a new vault object with the legion_startvault command.

      $ legion_startvault -L <path_to_Legion> \
         -O <path_to_OPR> -U <user> -A <arch> <host_name>
      

      See section 11.4 for more information on new vaults.

      3.4.3 Create a new host

      Use the legion_starthost command to create a new host object on the desired host. (See section 11.3 for more information on new hosts.)

      $ legion_starthost -L <path_to_Legion> \
         -O <path_to_OPR> -U <user> -A <arch> <new_host_name> \ 
         /vaults/vault-<new_host_name>
      

      The same command, with the -B flag, will start a new PCD host object. (See section 12.1 for more information on adding PCD hosts.)

      $ legion_starthost -L <path_to_Legion> \
         -O <path_to_OPR> -U <user> -A <arch> -B PCDUnixHost \
         <new_host_name> /vaults/vault-<new_host_name>
      

      The same command, with the -B flag, will start a new batch queue host object. You'll need to update the host object's attributes to include the queue type. (See section 13.1 for more information on adding batch queue hosts).

      $ legion_starthost -L <path_to_Legion> \
         -O <path_to_OPR> -U <user> -A <arch> \
         -B BatchQueueHost <new_host_name> \
         /vaults/vault-<new_host_name>
      $ legion_update_attributes /hosts/<new_host_name> \
         -a "host_queue_type(`<Queue_type>')"

      Adding a virtual host is more complicated: see section 14.0.

      If the new host has a new architecture, you now need to add implementations of the core objects for the new architecture. Log in to the new machine and run the following:

      $ source <path_to_globally_visable_setup_script>/setup.[sh|csh]
      $ legion_login /users/admin 
      		[run legion_login only if you 
      		have enabled Legion security]
      $ legion_init_arch

      Repeat these steps for each additional host. We suggest that you customize these steps and write a script to simplify the process, especially if you need to bring up a big net.

      3.4.4 Adding new users to a secure net

      To add users to a secure net, run:

      $ legion_create_user <new_user_name>

      If you are working on a PCD host, follow up with these steps:

      $ for i in `legion_ls /hosts'
      $ do
      $ legion_add_host_account /hosts/$i <unix_id> \
        /users/<new_user_name>
      $ done

      You will need to wait an hour for new user account(s) to be available.

      3.4.5 Working in the new net

      Users can work in an insecure net by entering:

      $ source <path_to_globally_visible_setup_script>/setup.[sh|csh]

      User can work in a secure net by entering:

      $ source <path_to_globally_visible_setup_script>/setup.[sh|csh]
      $ legion_login /users/<user_name>
      $ legion_cd /home/<user_name>

      Please note that neither of these procedures will open a separate shell.

      3.5 Making a set-up script for users

      We suggest that Legion system administrators prepare a set-up script for users to source when starting work in Legion. You can run the legion_make_setup_script command to automatically generate a set-up script for your system. This script sets the environment variables for Legion users. The usage is:

      legion_make_setup_script
      [-o <script basename>]
      [-OPR <OPR dir name>]
      [-L <$LEGION dir name>]
      [-debug] [-help]

      Supported options are:

      -o <script basename>

      Specify the basename for the resulting setup scripts (default is $LEGION_OPR/setup). This command will generate two setup scripts, one for /bin/sh derivative users and one for csh-derivative users. The scripts will be named <script basename>.sh and <script basename>.csh, respectively.

      -OPR <OPR dir name>

      Specify the OPR directory name that will be set up when the resulting scripts are run. This directory will contain the user's local copy of LegionClass.config (default is Legion-OPR). The user's local version of the directory will be placed in the user's $HOME.

      -L <$LEGION dir name>

      pecify the value of $LEGION, which is the directory where the resulting scripts are run. The default is the current value of $LEGION.

      -debug

      Catch and print Legion exceptions.

      -help

      Print command syntax and exit.


      1.Bourne Shell is not directly supported by our implementation of Legion, due to the use of alias to implement some Legion commands (notably legion_set_context). Bash, however, is supported.

      2. Once fully operational, Legion does not automatically shut down and restart: the system is intended to stay up.

      Directory of Legion 1.6.3 Manuals
      [Home] [General] [Documentation] [Software]
      [Testbeds] [Et Cetera] [Map/Search]

      Free JavaScripts provided by The JavaScript Source

      legion@Virginia.edu
      http://legion.virginia.edu/