The standard Legion host object creates objects using the process creation interface of the underlying operating system. However, some systems require using a queue management system to take full advantage of local resources. For example, some parallel computers contain a small number of "interactive" nodes, which can be accessed through normal means, and a large number of "compute" nodes, which can only be reached by submitting jobs to a local queue management system.
To make use of hosts that are managed by local queuing systems, Legion provides a modified host object implementation called the BatchQueueHost. BatchQueueHost objects submit jobs to the local queuing system, instead of using the standard process creation interface of the underlying operating system.
Please see section 3.4.3 for more on starting new hosts.
A BatchQueueHost can be used with a variety of queue systems (LoadLeveler, Codine, PBS, and NQS are the currently supported queue types). You can specify what type of local queue a given BatchQueueHost object by editing the host object's host_queue_type attribute. For example, if you want your new BatchQueueHost object to use the local "LoadLeveler" queue, you would run legion_update_attributes and add the "LoadLeveler" attribute:
Currently, each BatchQueueHost can use only one queue type at a time (i.e., if multiple local queuing systems are available, they can not all be used by the same BatchQueueHost -- an individual BatchQueueHost would need to be started to represent each queue). Typically, though, individual machines are managed by a single queue.
By default every Legion class contains a desired_host_property attribute specifying that it be run on an interactive host. You can use the legion_list_attributes command to check this particular attribute:
This signals the scheduler that the class's instances should not run on BatchQueueHosts. This is based on the conservative assumption that any class can run on interactive hosts, but not all classes can run on batch hosts.
If you are having trouble creating objects on a BatchQueueHost, there are several points of possible trouble. First be sure that you've removed the problem class's interactive desired_host_property (see above). If you still have trouble, you may have a misconfigured host object. Check the following points to be sure that your host object is set up correctly.
You can get a better idea of whether or not the local scripts are being called and what they're doing by looking in the log file maintained by the scripts (look in $LEGION_OPR/Legion-BatchLog). You'll find this log on the host where the BatchQueueHost object is running. If the logs indicate that the scripts are never called there may be a scheduling problem.
There is also a six minute delay after you add a new host to the system before which it will not be selected for scheduling, so you may need to wait a few minutes before you can test a new batch queue host.