One is always a long way from solving a problem
until one actually has the answer.

-- Stephen Hawking

Appendix E

Hierarchical Autonomous Agents

We demonstrate how designers can apply UNIFY to achieve effective Multi-Representation Modelling (MRM) in hierarchical autonomous agents (HAA) [Was98b]. Hierarchical autonomous agents employ multiple models to achieve a goal. Examples of the models are a planning model that selects actions that an agent can perform, and a perception-action model that senses and changes an agent's surroundings. A HAA may execute multiple models jointly.

The HAA model we considered is part of a research project undertaken by the Vision group at the University of Virginia. The agent, Marcus, has been programmed to construct complex arrangements such as archways from basic building blocks. Marcus is a hierarchical autonomous agent that has two models -- a planner model and a perception-action (PA) model. Typically, the planner maintains long-term or abstract representation, whereas the PA system maintains immediate and detailed representation. Each model may have its own representation of the world in which Marcus operates. Accordingly, each model may represent building blocks, partially-completed arrangements, obstacles, doors and pathways by a number of relevant attributes such as position, orientation and colour. Marcus considers relationships among blocks that are stacked or placed next to each other as an arrangement. We construct an MRE for Marcus and show how to maintain consistency within this MRE when concurrent interactions occur.

The jointly-executing models in Marcus are a planner model and a PA model. We determine the attributes in the planner and PA representations and construct a Multiple Representation Entity (MRE) for Marcus, as shown in Figure 75. Next, we capture the relationships among attributes using an Attribute Dependency Graph (ADG) and select mapping functions to maintain consistency in the Marcus MRE. Finally, we select policies for resolving the effects of concurrent interactions.

In §E.1, we list steps for incorporating UNIFY in Marcus. We demonstrate each step in subsequent sections. In §E.2, we construct an MRE. In §E.3 and §E.4, we construct an ADG and select mapping functions for attribute dependencies in the MRE. In §E.5 and §E.6, we determine and resolve the effects of concurrent interactions. In §E.7, we construct a CE and IR for the MRE.

E.1 Steps

The steps for incorporating UNIFY in Marcus are:

  1. 1. Construct an MRE from Planner and PA Representations
  2. 2. Construct an ADG for the MRE
  3. 3. Select Mapping Functions for Dependencies in the ADG
  4. 4. Determine the Effects of Interactions
  5. 5. Resolve the Effects of Concurrent Interactions
  6. 6. Construct a Consistency Enforcer and an Interaction Resolver

E.2 Construct an MRE from Planner and PA Representations

In order to construct an MRE for Marcus, we must determine what constitutes the representations of the planner and PA models. In HAAs, the representation of a planner or PA consists of attributes that capture properties of objects that are that are important for the current goal. In order to achieve a goal, an agent decomposes the goal into tasks and sub-tasks. Different models in a hierarchical agent architecture view an agent's tasks at different levels of abstraction. Deciding what tasks an agent must execute will be based on the agent's goals and capabilities. Marcus's goal is to build an archway out of coloured blocks scattered throughout a room. This goal can be refined to the planner tasks of build-tower and span-towers-with-block . The PA model accomplishes these tasks by executing tasks such as goto-block , pick-up-block , put-down-block , stack-block-on-block and span-blocks-with-block .

After the tasks and sub-tasks have been identified, the representation for each model can be constructed by identifying the objects relevant to the agent's tasks. These objects are parts of the environment that are affected by a task. For example, in the build-tower task, tower is a relevant object. Likewise, in the stack-block-on-block task , two block s are the relevant objects. A model represents objects relevant to its current tasks. Attributes are properties of these objects described using traditional data structures. During the execution of a task, the representation for a model may consist of many attributes for each object in the task. Given Marcus goal of constructing an archway from blocks, we list the attributes of objects represented by the PA and planner (Table 38).

Attributes of planner and PA (Marcus)

Entity

Object

Attributes

Planner

Tower1

T1.position, T1.orientation, T1.height, T1.width, T1.stacked, ...

Tower2

T2.position, T2.orientation, T2.height, T2.width, T2.stacked, ...

Arch

A.position, A.orientation, A.height, A.width, A.connected, ...

PA

Block1

B1.position, B1.orientation, B1.height, B1.width, B1.colour, ...

Block2

B2.position, B2.orientation, B2.height, B2.width, B2.colour, ...

Block3

B3.position, B3.orientation, B3.height, B3.width, B3.colour, ...

Block4

B4.position, B4.orientation, B4.height, B4.width, B4.colour, ...

Block5

B5.position, B5.orientation, B5.height, B5.width, B5.colour, ...

Wasson addresses how tasks can be decomposed and representation identified for jointly-executing models in HAAs [Was99].

E.3 Construct an ADG for the MRE

We construct an Attribute Relationship Table (ART) for the attributes in the planner and PA representations. We construct an example ART for our MRE (Table 39); in practice designers provide ARTs for their applications. The specification of the relationship may be accomplished formally; in Table 39, we present informal specifications in the last column.

Attribute Relationship Table for Marcus MRE

Dependency

Type

Specification

B1.position → T1.position

Cumulative

The positions of blocks determine the position of a tower and vice versa .

B2.position → T1.position

Cumulative

T1.position → B1.position

Distributive

T1.position → B2.position

Distributive

B1.orientation → T1.orientation

Cumulative

The orientations of blocks determine the orientation of a tower and vice versa .

B2.orientation → T1.orientation

Cumulative

T1.orientation → B1.orientation

Distributive

T1.orientation → B2.orientation

Distributive

B1.height → T1.height

Cumulative

The heights of blocks determine the orientation of a tower and vice versa .

B2.height → T1.height

Cumulative

T1.height → B1.height

Distributive

T1.height → B2.height

Distributive

T1.height → T1.stacked

Modelling

A tower with indeterminate height or orientation is not stacked.

T1.orientation → T1.stacked

Modelling

T1.position → A.position

Cumulative

The positions of towers determine the position of an arch and vice versa .

T2.position → A.position

Cumulative

A.position → T1.position

Distributive

A.position → T2.position

Distributive

A.width → A.stacked

Modelling

A tower with indeterminate width or orientation is not connected.

A.orientation → A.stacked

Modelling

...

We construct an ADG for the Marcus MRE. From Table 38, we determine the nodes in the ADG. From the ART in Table 39, we determine the arcs in the ADG. The ADG is shown in Figure 76. The interaction dependencies to some attributes exist because interactions with the environment may change those attributes.

The ADG in Figure 76 is constructed by Marcus as it progresses towards its goal. Initially, the ADG in Marcus consists of only nodes corresponding to the attributes at all representation levels. As Marcus stacks blocks to construct a tower or spans towers to construct an arch, it adds arcs to its ADG. If a previously-stacked tower falls apart, a CE in Marcus can detect that a relationship among the constituent blocks of the tower no longer holds. Subsequently, the CE changes the values of attributes in the ADG to denote that the tower is no longer stacked. At a future time, the planner model in Marcus can attempt to reconstruct the tower by stacking blocks.

E.4 Select Mapping Functions for Dependencies in the ADG

We select mapping functions to translate attributes among concurrent representations within the Marcus MRE. Recall from Chapter 6 that mapping functions must translate values or changes in values of attributes from one to another. Additionally, it is desirable that mapping functions complete their translations in a time-bound manner, and that they be composable and reversible.

We show mapping functions for some dependencies in Table 40. The mapping functions are presented as pseudo-code. Error-checking has been omitted for brevity. Pseudo-code in the second column of Table 40 implements specifications in the last column of Table 39. The position of a tower is identical to the position of its lowermost block. Conversely, the position of the lowermost block in a tower is identical to the position of the tower. If the positions of other blocks are desired, then appropriate dependencies must be specified, for example, dependencies between the height of the lowermost block and the position of the block immediately above it. If either the height or the orientation of a tower is invalid, the tower is not stacked. The orientation of a tower can become invalid if its constituent blocks are oriented differently. A similar condition may be specified for the height of a tower. Similar mapping functions for other dependencies can be constructed. Mapping functions such as those shown in Table 40 translate values or changes in values of attributes.

Mapping Functions for Marcus MRE

Dependency

Mapping Function

B1.position → T1.position

T1.position ← fp (B1.position, B2.position)

fp : T1.position ← B1.position

B2.position → T1.position

T1.position → B1.position

(B1.position, B2.position) ← gp (T1.position)

gp : B1.position ← T1.position

T1.position → B2.position

B1.orientation → T1.orientation

T1.orientation ← fo (B1.orientation, B2.orientation)

fo : T1.orientation ← B1.orientation

B2.orientation → T1.orientation

T1.orientation → B1.orientation

(B1.orientation, B2.orientation) ← go (T1.orientation)

go : B1.orientation ← B2.orientation ← T1.orientation

T1.orientation → B2.orientation

T1.orientation → T1.stacked

T1.stacked ← p 1(T1.orientation)

p 1: if (T1.orientation = invalid) T1.stacked = false

T1.height → T1.stacked

T1.stacked ← p 2(T1.height)

p 2: if (T1.height = invalid) T1.stacked = false

...

The mapping functions shown in Table 40 are composable and reversible. Moreover, since they are simple in construction, we expect that they will complete in a time-bound manner, thus ensuring that the Marcus MRE is consistent at all observation times. When an interaction changes the value of any attribute, mapping functions propagate the change in the attribute to dependent attributes. For example, if an interaction changes the PA-level attribute, B1.orientation, the mapping function fo changes the dependent planner-level attribute, T1.orientation. Subsequently, the mapping function go changes the PA-level attribute, B2.orientation. Since fo and go are composable, the change to B1.orientation eventually propagates to B2.orientation. Since fo and go are reversible, B1.orientation does not change again as a result of the same interaction.

When an interaction occurs, traversing the ADG in Figure 76 and applying the mapping functions in Table 40 ensures that the Marcus MRE is consistent at all observation times. Next, we determine and resolve the effects of concurrent interactions.

E.5 Determine the Effects of Interactions

We determine the effects of interactions that Marcus can send and receive. Marcus can interact with its environment only at the PA level. The planner model in Marcus does not interact with the environment apart from initially receiving a goal and finally reporting on the success or failure in achieving the goal. However, the planner interacts with the PA level in order to specify sub-tasks. In Table 41, we list the interactions that the planner and PA levels can send and receive. In the first column, we list the name of an interaction. In the second and third columns, we indicate the sender and receiver of an interaction. A sender or receiver that is not "planner" or "PA", is external to Marcus and is part of Marcus' environment. In the fourth column, we list the attributes affected by the interaction directly, i.e., we list the set affects for the interaction. We do not list the set affects + for the interaction because this set changes as Marcus adds arcs to its ADG while progressing towards its goal. Finally, we indicate the type of the interaction.

We augment each interaction with its type (see Chapter 7): Type 0 (certain responses), Type 1 (uncertain responses), Type 2 (certain requests), and Type 3 (uncertain requests). Assigning a type requires information about the semantics of an interaction. In Marcus, PA-level interactions are assumed to be certain. For example, the PA model in Marcus is assumed to be able to sense the position of an object correctly. Likewise, if the PA model requests the underlying hardware to move Marcus, the hardware will not fail to do so. Hence, interactions between the PA model and the processor are Type 0 or Type 2. In contrast, planner-level interactions are assumed to be uncertain. For example, Marcus may not be able to pick up a block as per the planner's request. Hence, interactions between the planner model and the PA model are Type 1 or Type 3. Classifying the interactions in this manner reflects the design philosophy of "trusting sensors more than memory".

Interactions sent and received by the Marcus MRE

Interaction

Sender

Receiver

affects

Type

SenseObject( X )

PA

processor

 

2

UpdateObject( X )

processor

PA

X .position, X .orientation, X .height, X .width, ...

0

Move

PA

processor

 

2

Swivel

PA

processor

 

2

Turn

PA

processor

 

2

MoveStatus

processor

PA

 

0

SwivelStatus

processor

PA

 

0

TurnStatus

processor

PA

 

0

PickObject( X )

planner

PA

X .position, X .orientation

3

PutDownObject( X )

planner

PA

X .position, X .orientation

3

StackObject( X , Y )

planner

PA

Y .position, Y .orientation

3

SpanObject( X , Y , Z )

planner

PA

Z .position, Z .orientation

3

GoThroughDoor

planner

PA

 

3

Travel

planner

PA

 

3

ActionStatus

PA

planner

 

1

Any subset of the interactions in Table 41 may occur concurrently. Next, we show how to resolve the effects of concurrent interactions.

E.6 Resolve the Effects of Concurrent Interactions

The effects of concurrent interactions can be resolved by implementing application-specific polices. In practice, a designer selects policies specific to the application. We select example policies, shown in Table 42. A designer specifies policies in a Concurrent Interactions Table (CIT) for resolving the effects of concurrent interactions. The CIT consists of sets of concurrent interactions with dependent effects, policies for resolving them and conditions under which the policies are applicable. Concurrent interactions that are independent of one another can be resolved by serialization and are not specified in the CIT. Some interactions may be independent because they affect disjoint sets of attributes. Other interactions may be independent because their effects are applied in different time-steps, for example, interactions sent and received by an entity. Yet other interactions are independent because they are request-response pairs. Policies must be specified in the CIT for only the remaining interactions. An Interaction Resolver for the Marcus MRE applies the policies in the CIT only if the effects of concurrent interactions conflict. If concurrent interactions do not conflict, they may be serialized.

Concurrent Interactions Table for Marcus MRE

Concurrent Interactions

Condition

Policy

Type 0, Type 1

 

Ignore Type 1

Type 2, Type 3

 

Delay Type 3

Any Interaction

Ignored or Delayed

Ignored or Delayed entirely, i.e., no partial effects permitted

E.7 Construct a Consistency Enforcer and an Interaction Resolver

A Consistency Enforcer (CE) and an Interaction Resolver (IR) for an MRE maintain consistency and resolve concurrent interactions respectively. A CE consists of an ADG and mapping functions, whereas an IR consists of policies for resolving concurrent interactions. Figure 77 shows an MRE for Marcus. The MRE can interact at multiple representation levels -- the planner and PA levels -- concurrently. Moreover, the concurrent representations within the MRE are consistent at all observation times.

A CE consists of an ADG and application-specific mapping functions. We presented an ADG for Marcus in Figure 76 and mapping functions in Table 40. In Figure 34 (see Chapter 6), we presented an algorithm for implementing a CE. In §6.3, we discussed how to traverse an ADG and apply mapping functions to keep an MRE internally consistent.

An IR consists of application-specific policies for resolving the effects of concurrent interactions. For the Marcus MRE, we presented policies for resolving concurrent interactions in Table 42. In Figure 47 (see Chapter 7), we presented an algorithm for implementing an IR. In §7.5, we presented a taxonomy for classifying interactions. Using this taxonomy, we presented policies for resolving the effects of concurrent interactions.

A CE and an IR ensure that an MRE is internally consistent when concurrent interactions occur. During a time-step, a number of concurrent interactions may occur. The IR determines the type of each interaction. Next, the IR applies the effect of each interaction as if the interaction occurred in isolation. In order to do so, the IR permits the interactions to take effect one at a time. When an interaction changes an attribute, the CE traverses an ADG and translates changes to dependent attributes by invoking the appropriate mapping functions. The CE maintains a list of changes for each attribute as a result of computing the effects of each interaction. Subsequently, the CE applies the effects of all the interactions on each attribute. The CE queries the IR about policies to resolve the effects of dependent concurrent interactions whenever the CE detects conflicts in the list of changes for an entity. If the IR contains a policy for resolving conflicting changes, the CE applies the changes accordingly; otherwise, the CE assumes the changes are independent and applies them in an arbitrary order. When the changes to all attributes have been applied, the MRE is internally consistent.