3.0 Starting a new system

3.1 Before you start

Before you start a new Legion system, consider what type of set-up will best suit your needs. Primary considerations include:

  • What kind of system do you need? How many machines do you anticipate using? Do you only use local hosts or do you also use remote machines? Possible configurations might include:
  • a. A single host system: one Legion machine with one or more host objects (Figure 1). This is the simplest system.

    Figure 1: Single host system

    b. A multihost system: multiple Legion hosts linked together and sharing local resources (Figure 2). This can include homogeneous or heterogeneous platforms, as well as non-Legion machines. The machines do not need to be in physical proximity.

    Figure 2: Multihost system

    c. A multidomain system: multiple Legion domains connected together and sharing each others' resources (Figure 3).

    Figure 3: Multidomain system

  • Will you be using Legion security? This is an important consideration, since a secure Legion system cannot be cleanly shut down and restarted. If you are using security, you should decide who will be the "admin," the system administrator. The system should be started up by the admin, since he or she will have special privileges on the core objects created in the new system. (These privileges do not extend to other users' objects, however.)
  • What kind of host objects will you be using? Options include:
  • a. Basic host object: it resides on its host and manages and guards its host's resources. This is the template for the other host objects.

    See Host and vault objects in the Basic User manual for discussion of basic host objects. In this manual, section 11.0 discusses host-vault pairings and adding new hosts.

    b. PCD host object: it resides on its host, manages and guards its host's resources, and uses a process control daemon to regulate ownership of all Legion processes executed on that host. If you use a PCD host as your bootstrap host, the start-up process will be slightly different.

    The daemon requires root privileges to start and to run. The PCD host object is useful if outside users will be running processes on your host, but can only be used if Legion security is enabled. Each user's processes will be tracked and accounted for. For more information, please see "Process control daemon host objects".

    c. Batch queue host object: it resides on its host, manages and guards its host's resources, and submits Legion jobs to the local queueing system.

    This is the best choice for hosts that use a queue management system, although the PCD host object is more secure and has better accounting. For more information, please see "Batch queue host objects".

    d. Virtual host object: it resides on a different host, represents and guards its host's resources, but does not run normal Legion objects. A virtual host cannot be used as a bootstrap host: it is added to an already running system.

    A virtual host object is used for running Legion jobs on unsupported platforms. The host object resides on a supported platform and runs native jobs with standard Legion tools on the target host machine. It can be used for scheduling, resource selection, and transparent execution on the target machine. For more information, please see "Virtual hosts".

3.2 Set up the environment

A properly set-up environment is crucial for working in the Legion system. The start-up process uses certain Legion-specific environment variables, which must be correctly set before starting applications and running command-line utility programs. You must set these variables each time you starting working in Legion. Without a properly set environment, programs cannot communicate with other objects in the system, and the program may terminate with an error, never return a value, or fail in a more spectacular fashion. If this occurs, try setting your environment properly and starting over.

You must have /bin/ksh installed in your system. There are a number of Legion scripts that will look for ksh, and if it is not installed in your system you will get error messages.

To prepare the environment in a newly started or restarted system, set the $LEGION and $LEGION_OPR variables and source the script that is appropriate for your shell.1

(ksh or sh users)

export LEGION=<Legion root dir path>
export LEGION_OPR=<Legion OPR root dir path> 

(csh users)

setenv LEGION <Legion root dir path>
setenv LEGION_OPR <Legion OPR root dir path>

The environment must be properly set in each shell in which you plan to run Legion commands. Check to be sure that environment variables are properly set (we suggest $LEGION/../OPR for the OPR root directory path).

If necessary, run the following.

(ksh or sh users)

. $LEGION/bin/legion_env.sh
. $LEGION_OPR/legion_context_env.sh

(csh users)

source $LEGION/bin/legion_env.csh
source $LEGION_OPR/legion_context_env.csh

3.3 Starting a single host system

There are three steps in this procedure: configure the system with legion_setup_state, start it with legion_startup, and, finally, initialize it with legion_initialize.

3.3.1 Choose a bootstrap host

You must first choose a bootstrap host. Start and shut down your system from here.2 The bootstrap host will have the following features:

  • The LegionClass object (the root of the Legion binding mechanism and the parent class of many metaclasses) will be placed here.
  • The $LEGION_OPR/LegionClass.config file will be created on this host and will contain the LegionClass object address, which must be globally known to all Legion objects. The file must be available when other objects are started in the system.

If you choose a PCD host as your bootstrap, the start-up procedure is slightly different than for a basic host object.

3.3.2 Set up and configure the system

You must first set up the initial state for core Legion system objects. Legion system objects are persistent, and can save and restore their own state. Some of these objects must have their state initialized before they run for the first time. After the initial start-up, these objects will manage their own state and configuration, if the system is properly maintained.

If you are booting on a PCD host, first run:

$ LEGION_HOST_BIN=PCDUnixHost

Enter the legion_setup_state command to configure the system:

$ legion_setup_state

This program will return your start-up host name, a port number for the LegionClass object, and a time. If you do not want to use the default settings, use the -i flag to run the command in an interactive mode. Your output will look something like this:

$ legion_setup_state
Creating OPR directory /home/xxx/OPR/.
Saving LegionClass configuration file: 
	/home/xxx/OPR/LegionClass.config
LegionClass host name   = your.startup.host.name
LegionClass port number = 7899
LegionClass timestamp   = 898198093
$

The script creates the $LEGION_OPR directory and several sub-directories, populating them with initial states for several core system objects. The timestamp sets the starting time for the system: Legion objects use a timestamp to guarantee each object's unique identity. The current time is measured in seconds since January 1, 1970.

3.3.3 Start the system

The legion_startup script provides prompts asking whether or not to start each component. It's best to answer "yes to all" (Y). The verbose option allows you to see more detailed information, as the script works, about debugging. (This can be large amounts of information, so use this option only if you are searching for a problem.)

To start the main core system objects, enter the legion_startup command.

$ legion_startup

Legion will start several classes on your host. Your output will look something like this

$ legion_startup
Starting meta-class object: LegionClass
Continue (y=yes, Y=yes to all, n=no, N=no to all,
	v=verbose, V=verbose all)? Y
Starting meta-class object: BootstrapMetaClass
Starting class object: BindingAgentClass
Starting class object: CommandLineClass
Starting class object: UnixHostClass
Starting class object: UnixVaultClass
Starting class object: UnixImplementationClass
Starting class object: UnixImplementationCacheClass
Starting class object: ContextClass
Legion first-time system startup complete
Run "legion_initialize"
$

This indicates that the major class objects were started: LegionClass is the top-level metaclass and is the parent of every other object in the system. BootstrapMetaClass is the class object for the bootstrap class objects, which are those class objects that have instances that need to get created in the initialization phase; those class objects are UnixVaultClass, UnixHostClass, UnixImplementationClass, and UnixImplementationCacheClass.

The new class objects will start instances in the new system: BindingAgentClass parents binding objects, CommandLine- Class parents command-line objects, etc.

3.3.4 Initialize the system

Like legion_startup, the legion_initialize script will provide prompts asking whether or not to perform each task, and it is generally best to use the "yes to all" option (Y). To initialize Legion, enter

$ legion_initialize

The output shows the system creating and tagging the key ingredients of a new system. It is too long to reproduce in full here, but we'll look at some selected actions.

The first line shows the system creating a bootstrap host object on your current host (a host object manages a host, so the bootstrap host object manages the bootstrap host).

Creating host object BootstrapHostObject on 
	"your.current.host.name"
Continue (y=yes, Y=yes to all, n=no, N=no to all,
	v=verbose, V=verbose all)? Y
Configuring wellknown binaries for host
	"1.01.07.0100..."

A bootstrap vault object is automatically created on your current host (a vault object manages a vault, which stores Legion object's permanent states). This guarantees that the bootstrap host object has a compatible vault object. All host objects must be paired with at least one compatible vault object (i.e., a vault that it can "see" -- see "About host-vault pairs".).

Creating vault object BootstrapVaultObject on
	"your.bootstrap.host.name"
Setting BootstrapHost and BootstrapVault restrictions
Added 1 host(s) to vault's compatibility set
Added 1 vault(s) to host's compatibility set

Implementation objects represent and manage the implementation cache (used to allow Legion processes to take place in different architectures) and context space.

Creating an ImplementationCache
Creating an implementation (ContextObject) for ContextClass
Creating the root context object

More implementation objects are created as the process creates new object classes.

Adding "BootstrapHost" to the hosts context
Adding the alias "your.bootstrap.host.name" for
	BootstrapHost to the hosts context
Adding "BootstrapVault" to the vaults context

Two context names are added to the /hosts context (BootstrapHost and your.bootstrap.host.name): both names refer to the Bootstrap host object (Figure 4) but only one is added to the /vaults context.

Figure 4: Context paths for the bootstrap host and vault objects

This object will manage a portion of the persistent storage mechanism for the Bootstrap host (please see section 6.0 in the Basic User Manual for a discussion of Legion hosts and vaults). The /impls context contains names of the default implementation objects (see Implementation model). These can all be viewed with context-related commands or the GUI once the system has been completely started.

3.3.5 Set the environment

Source the script that sets up environment variables for using the context space.

ksh or sh users type:

$ . $LEGION_OPR/legion_context_env.sh

csh users type:

$ source $LEGION_OPR/legion_context_env.csh

You must source this script in each shell in which you plan to run Legion commands, including the shell in which you ran legion_initialize. Otherwise you will not be able to run any Legion utilities or commands.

3.3.6 Choose your security setting

If you wish enable Legion security, run the legion_init_security command. You'll have to decide now whether or not you want security, since the command will not run properly unless you run it immediately after initializing the system. If you don't wish to use it, just skip over this section. However, none of your processes will be protected.

To use Legion security, run:

$ legion_init_security

You will be given a system administrator user id called admin. The administrator has root privileges in the system and can create new users, modify security settings, etc. He/she also has ownership of all currently existing objects in the new system but not any other objects (i.e., any objects that other users create).

Several events take place when you run this command, and we will discuss each in turn.

Creating the context "/users" to contain user-objects
Creating context "users" in parent "/".
New context LOID = "1.3622260c.05.08000000.000001fc..."

The /users context holds Legion user ids, starting with admin:

Creating the initial system-admin user object, "/users/admin"
Please select a Legion password for "/users/admin":
New Legion password: xxxx
Retype password: xxxx
1.3622260c.6b000000.01000000.000001fc0c...
Please enter the Legion password for "/users/admin" to 
	continue:
Password: xxxx

This admin is the system administrator's user id: anyone logging in as admin has root privileges. You will have to enter the password three times during the legion_init_security process and when logging in.

Changing ownership of all objects to "/users/admin"
1.3622260c.01..000001fc0cbe1846763f895a...
1.3622260c.02..000001fc0b3b16eb8b2dde29...
[...etc.]
Changed ownership of 43 objects.

Legion assigns ownership of all existing objects to admin. Objects created by other users will belong to those users.

Creating initial ACLs files for all core objects in
	/home/xxxx/OPR/init_acls
Creating ACL for /class/AuthenticationObjectClass class
Creating ACL for /class/BasicFileClass class
[...etc.]

The access control lists (ACLs) protect objects against unauthorized use. Only an object's creator can use the object, unless the creator specifies otherwise. The initial ACL files allow only the admin to use the core objects.

Creating context "tmp" in parent "/".
New context LOID = "1.362dabd6.05.09000000.000001fc..."
[...etc.]
Done creating acl files

Several new context objects are created and placed in the root context and given ACLs.

Setting ACL for /class/AuthenticationObjectClass class
[...etc.]
All acls set.
$

ACL files are then created for all previously existing objects.

At this point, security has been enabled and is running. You must now log in as admin. Note that you need to include the /users path.

$ legion_login /users/admin
Password: xxxx
$

You are now ready to start working. For more information, see "About Legion security" and "Using security features".

3.4 Starting a multihost system

This is a two-part process. First, you have to have a running single host system, as laid out in section 3.3. Second, you add new host objects on the desired machines. Since you will be starting processes on the target hosts from the bootstrap host be sure that you can run rsh/ssh on the bootstrap host as well as on the target hosts from the bootstrap host without having to enter a password. You can set up a .rhosts file for rsh or an authorized_keys files for ssh to accomplish this (see the rsh and ssh man pages for more information).

3.4.1 Set up

You'll need to set the proper environment variable on the bootstrap host and the remote host(s) so that you can run Legion commands on a remote host using rsh or ssh.

For sh, ksh, or bash:

LEGION_RSH=<rsh|ssh>
LEGION_RCP=<rcp|scp>
export LEGION_RSH LEGION_RCP

For csh:

setenv LEGION_RSH <rsh|ssh>
setenv LEGION_RCP <rcp|scp>

Set these variables on the bootstrap host before you start the new system (i.e., before you run legion_startup). Please note that you only need to follow these steps on the bootstrap host: you will need to install the Legion binaries on any other machines that you add to your system, but you do not need to start more Legion systems.

To add additional hosts and users, copy the $LEGION_OPR/setup.[sh|csh] scripts to a globally accessible location. Hosts can share an NFS-mounted Legion tree, but for best results you should place the OPRs on a local disk.

3.4.2 Create a new vault

If the new host will not be compatible with your existing vaults, create a new vault object with the legion_startvault command.

$ legion_startvault -L <path_to_Legion> \
   -O <path_to_OPR> -U <user> -A <arch> <host_name>

See section 11.4 for more information on new vaults.

3.4.3 Create a new host

Use the legion_starthost command to create a new host object on the desired host. (See section 11.3 for more information on new hosts.)

$ legion_starthost -L <path_to_Legion> \
   -O <path_to_OPR> -U <user> -A <arch> <new_host_name> \ 
   /vaults/vault-<new_host_name>

The same command, with the -B flag, will start a new PCD host object. (See section 12.1 for more information on adding PCD hosts.)

$ legion_starthost -L <path_to_Legion> \
   -O <path_to_OPR> -U <user> -A <arch> -B PCDUnixHost \
   <new_host_name> /vaults/vault-<new_host_name>

The same command, with the -B flag, will start a new batch queue host object. You'll need to update the host object's attributes to include the queue type. (See section 13.1 for more information on adding batch queue hosts).

$ legion_starthost -L <path_to_Legion> \
   -O <path_to_OPR> -U <user> -A <arch> \
   -B BatchQueueHost <new_host_name> \
   /vaults/vault-<new_host_name>
$ legion_update_attributes /hosts/<new_host_name> \
   -a "host_queue_type(`<Queue_type>')"

Adding a virtual host is more complicated: see section 14.0.

If the new host has a new architecture, you now need to add implementations of the core objects for the new architecture. Log in to the new machine and run the following:

$ source <path_to_globally_visable_setup_script>/setup.[sh|csh]
$ legion_login /users/admin 
		[run legion_login only if you 
		have enabled Legion security]
$ legion_init_arch

Repeat these steps for each additional host. We suggest that you customize these steps and write a script to simplify the process, especially if you need to bring up a big net.

3.4.4 Adding new users to a secure net

Only admin can add users to a secure net. Run the legion_create_user command with the new user's name:

$ legion_create_user <user_name>

If you are working on a PCD host, follow up with these steps:

$ for i in `legion_ls /hosts'
$ do
$ legion_add_host_account /hosts/$i <unix_id> \
  /users/<new_user_name>
$ done

You will need to wait an hour for new user account(s) to be available.

3.4.5 Working in the new net

Users can work in an insecure net by entering:

$ source <path_to_globally_visible_setup_script</setup.[sh|csh]

User can work in a secure net by entering:

$ source <path_to_globally_visible_setup_script>/setup.[sh|csh]
$ legion_login /users/<user_name>
$ legion_cd /home/<user_name>

Please note that neither of these procedures will open a separate shell.

3.5 Making a set-up script for users

We suggest that Legion system administrators prepare a set-up script for users to source when starting work in Legion. You can run the legion_make_setup_script command to automatically generate a set-up script for your system. This script sets the environment variables for Legion users. The usage is:

legion_make_setup_script
    [-o <script basename>]
    [-OPR <OPR dir name>]
    [-L <$LEGION dir name>]
    [-debug] [-help]

Supported options are:

-o <script basename>

Specify the basename for the resulting setup scripts (default is $LEGION_OPR/setup). This command will generate two setup scripts, one for /bin/sh derivative users and one for csh-derivative users. The scripts will be named <script basename>.sh and <script basename>.csh, respectively.

-OPR <OPR dir name>

Specify the OPR directory name that will be set up when the resulting scripts are run. This directory will contain the user's local copy of LegionClass.config (default is Legion-OPR). The user's local version of the directory will be placed in the user's $HOME.

-L <$LEGION dir name>

Specify the value of $LEGION, which is the directory where the resulting scripts are run. The default is the current value of $LEGION.

-debug

Catch and print Legion exceptions.

-help

Print command syntax and exit.


1. Bourne Shell is not directly supported by our implementation of Legion, due to the use of alias to implement some Legion commands (notably legion_set_context). Bash, however, is supported. Back

2. Once fully operational, Legion does not automatically shut down and restart: the system is intended to stay up. Back

Directory of Legion 1.6.4 Manuals
[Home] [General] [Documentation] [Software]
[Testbeds] [Et Cetera] [Map/Search]

Free JavaScripts provided by The JavaScript Source

legion@Virginia.edu
http://legion.virginia.edu/