AI Magazine Volume 10 Number 4 (1989) (© AAAI)
Articles
The
Advanced
Architectures
Project
James Rice
26
AI MAGAZINE
Articles
The Advanced Archipublications
are
The Advanced Architectures Project at Stanford
University's Knowledge Systems Laboratory
available from KSL,
tectures Project (AAP)
seeks to gain higher performance for expert
and many have also
has been running for
system applications through the design of new,
been published elsea number of years at
innovative software and hardware architectures.
where. This article,
Stanford University’s
This research concentrates particularly on the
then, is more of a
Knowledge Systems
use of parallel machines to gain speedup and the
concept or a research
Laboratory (KSL). The
design of the software to exploit emergent paralin progress article
project is large and
lel hardware architectures. This article describes
than one presenting
has a number of
the project and details its goals and the work
scientific goals.
components that
performed in the pursuance of these goals. A
have been documentbrief description is given of each of the project
components, and a complete bibliography appears
ed at length. These
Project Goals
of the publications produced for the project.
components have
never been drawn
The project’s primatogether in one docry goal is to find
ument; thus, this article describes the project
ways to increase the performance of expert
and gives a taste of the individual subprojects
systems through the use of the new, emerthat have kept the project members so busy for
gent, parallel hardware designs.
so long. A large number of publications have
The number of possible implementation
emerged from the project, so a full bibliograstrategies for such a project is huge. One only
phy of the work appears for the reader who
has to look at the large number of different
wants to follow up on any intriguing topics.
hardware designs that are emerging and at
AAP straddles a number of research areas
the number of different problem-solving
and, thus, does not fall easily into any one
methods to see how combinatorial the probsphere of interest. A certain amount of work
lem would be if the project tried to investigate
has been done on the parallelizing of expert
all the reasonable and plausible architectural
systems, most notably by Gupta (1986).
combinations. It was decided, therefore, that
Similarly, some machines on the market
the project members could learn a great deal
resemble in some respects the machines that
by simply making a commitment to one, or
project members have been designing, most
at least a small number of, different options
notably the Ametek machine. This article
at each point in the system’s makeup. Thus, it
does not in any significant way relate the
was decided that the project would take a
work of AAP to the work in other fields; for
“vertical slice” through the space of possible
such comparative studies, you can refer to the
solutions. Clearly, the members did not
bibliography. The reader should keep in mind,
intend to investigate any options that seemed
however, that I know of no projects that have
nonuseful. Although they could not prove
been comparable to AAP, and so it is often
they had the best design to meet their goals,
hard to find a reasonable point of comparithey knew from the outset their design would
son. For instance, considerable effort is being
at least be a plausible architecture for a future
spent on concurrent image recognition on
computational environment.
massively parallel machines such as the ConThe project members viewed the task of
nection Machine™, 2but this research has had
implementing concurrent expert systems as
one that was split into a number of implelittle influence on the work of this project.
mentation layers. If they could achieve
At this point, I should make a brief disspeedup at each one of these layers, then they
claimer. The subject matter for AAP is comcould hope for a substantial overall perforplex and not in the mainstream of experience
mance improvement compared to existing AI
for members of the AI community. Thus, the
systems. The model of the layers into which
AAP project members are concerned that the
the project could be split appears in figure 1.
conclusions they have drawn from their work
It was originally anticipated that the needs
might be misinterpreted, trivialized, or overof the applications would drive the developgeneralized. It is unrealistic in such a short
ment of the problem-solving frameworks and
article to try to do justice to the full lessons
the needs of the problem-solving frameworks
that can be learned from the project’s extenwould drive the development of the knowlsive quantitative experiments. For this reason,
edge retrieval layer and so on until eventually
I do not attempt to draw any great concluthe hardware would be designed under the
sions here. The reader is strongly encouraged
constraints passed down from above (figure
to read any of the 50-odd papers that are
1). In practice, however, this process did not
cited here to gain a deeper understanding of
occur. Because of the difficulty of finding and
the work and the lessons learned. All these
1
. . . it was
decided
that the
project would
take a
“vertical
slice” through
the space of
possible
solutions.
0738-4602/89/$3.50 ©1989 AAAI
WINTER 1989
27
Articles
Applications
Problem-Solving
Frameworks
Knowledge Retrieval
Applications
Problem-Solving
Frameworks
Resource Management
Resource Management
Programming Languages
Programming Languages
Hardware
Operating Systems
Figure 2. Layers Actually Tackled in the Project.
Hardware
Resource management appears in small type because it
is a recent addition, and most of the work has been
done without the help of this layer.
Figure 1. The Layers of System Implementation
Through Which Project Members Hoped to
Achieve Computational Speedup.
mounting an application suitable to the project’s needs and the early availability of personnel interested in the hardware design
aspect, hardware design proceeded more
rapidly than the other layers. Thus, the
designs were more hardware driven than
application driven. Such a result is not necessarily bad because an entirely top-down
design process could have easily resulted in
low-level system requirements that could not
have been implemented.
The levels of abstraction actually implemented differed significantly from those
shown in figure 1. Figure 2 gives a more realistic representation of what the project members actually did, as opposed to what they
intended to do.
KSL staff members have considerably more
expertise in software than in hardware. Project members decided early not to build any
hardware; many other research groups are
better at hardware development. Therefore,
they decided to simulate their hardware,
allowing them to modify the software and
hardware designs easily and extract the maximum insight with the minimum effort.
The remainder of this article is split into
sections that reflect the major layers shown
in figure 2. In each of these sections, the
work of the relevant subprojects is discussed.
Because of the project’s bottom-up emphasis,
the components are discussed in bottom-up
order. This format reduces the number of for-
28
AI MAGAZINE
ward references made because a discussion of
the higher layers inevitably has to refer to the
substrates on which they are implemented.
Personnel
This project has employed a large number of
people over the years. It seems appropriate to
name them all here because otherwise they
might only appear as authors referenced in
the bibliography: Ed Feigenbaum, Bob Engelmore, Penny Nii, Bruce Delagi, Harold Brown,
Hiroshi Okuno, John Delaney, Byron Davies,
Hirotoshi Maegawa, Nelleke Aiello, James
Rice, Nakul Saraiya, Sayuri Nishimura, Eric
Schoen, Greg Byrd, Max Hailperin, Russel
Nakano, Masafumi Minami, Chris Rogers,
Alan Noble, Jean-Christophe Bandini, Manu
Thapar, Pandu Nayak, Djuki Muliawan, Jerry
Yan, and Sam Hahn.
Hardware
As mentioned earlier, hardware design led the
way in AAP. In this section, I discuss a little
bit of the motivation for the hardware designs
and briefly describe the current generation of
hardware designs that project members are
working on and the simulator they are using.
Simple and Helios
The hub of all the work done on AAP has
been the circuit simulator; everything else is
Articles
Figure 3. The Simple System Provides a Tool Kit from Which to Build Circuits to Be Simulated,
a Collection of Probes to Connect to the Circuit, and a Set of Instruments to Connect to the Probes.
built on it. This simulator is called Simple. It
is an event-driven simulator devised to allow
the user to design and specialize digital circuits in a simple and modular way using a circuit design tool called Helios. A sophisticated
set of instrument tools allows the user to
design and specialize simulated probes that
can be connected to the circuit while it is running. This setup allows a number of instruments to be connected to the probes that
permit the user to see the behavior of the circuit as it operates without interfering with
the system’s behavior. Project members like to
view this model as a laboratory workbench
equipped with collections of instruments,
probes, and circuit-building components from
which the user can build systems and on
which the user can perform quantitative
experiments (figure 3) (Delagi et al. 1986, 1987).
It was found early on that simulations of
the sort project members wanted to do would
be computationally expensive. An attempt
was made, therefore, to parallelize the simula-
tor itself in an attempt to reduce the time
taken for the simulations, which often
exceeded one day. This attempt resulted in
AIDE, a distributed version of Simple (Saraiya
1986). Unfortunately, project members were
unable to achieve any speedup at all for their
simulations, largely because of the communications bandwidth necessary and the latency
associated with communicating between the
multiple Symbolics machines used by way of
Ethernet. Also, the simulator was event driven
and required frequent synchronization on the
event queue, which serialized the processing.
CARE
The Simple simulator was used to design and
build what project members refer to as the
CARE machine and simulation system
(Delagi et al. 1988) (figure 4). The CARE
machine is the simulated machine on which
all the experiments mentioned in the following paragraphs were performed. The
3
WINTER 1989
29
Articles
on their simulated machines. Therefore, they
traded accuracy in their processor simulation—assuming that the processing elements
will behave much like existing Lisp machine
processors—in favor of greater realism in
terms of the system’s performance under the
load of real programs.
A number of system design aspects have
not been addressed in detail, and the simulations do not take these into account. Perhaps
most significant is the fact that memory
usage, code distribution, and garbage collection are not simulated.
The CARE/Simple simulator system is perhaps the most valuable tangible product of
the project. It is now being used in a number
of research departments, both corporate and
academic, outside Stanford. Like all AAP software, it is in the public domain. CARE/Simple
will soon be running under Common Lisp,
CLUE, and X11 on a number of different
platforms.
Operating Systems
and Languages
Figure 4. An Example Piece of Instrumentation from the CARE System.
This figure shows the lengths of the task queues on the different processors plotted
against time.
machine’s design has a few key features that
are worthy of note:
• Dynamic cut-through routing to optimize
network throughput (Byrd, Nakano, and
Delagi 1987a)
• Toroidal topology (Byrd and Delagi 1987)
• Nonblocking message sending to encourage
pipeline processing
• Communications network with alternative
paths between points to reduce communications problems resulting from busy communication paths
• A separate communications controller to
support operating system functions and
implement the nonblocking send functionality mentioned previously
The work on the CARE subproject has
focused mainly on the design of interprocessor communication networks, as is appropriate. Thus, project members have been able to
ignore the instruction-level behavior of the
processors themselves. The application programs that the members run are merely timed
as they run between the points at which code
fragments cause communication between
processors. Being able to avoid doing registerlevel simulation of the processors themselves
has allowed project members to execute
much more complex and realistic programs
30
AI MAGAZINE
A considerable amount of effort has been
spent working at the operating system level
of abstraction. Curiously, the project members have written no operating systems. The
CARE machine itself features a dual processor
for each processing element. This arrangement allows much of the work of the operating system, particularly interprocessor
communication, to be done by a dedicated
processor in parallel with the execution of
user code. The behavior of this communications processor is coded directly into the simulated hardware. Work in this area has been
done on concurrent object-oriented systems,
concurrent Lisp dialects, programming
models, and resource allocation.
CAREL
CAREL (Davies 1986) was one of the first programs written to run on the CARE simulated
machine. It was an early attempt to find a
Lisp language interface to the distributedmemory hardware provided by CARE. It took
as its basis Scheme (Abelson and Sussman
1983) and QLisp (Gabriel and McCarthy
1984) and included primitives to allow
remote function calls and remote CONSing.
It was quickly found that because of the cost
of process creation, it was desirable to make
the best use of any processes which were
spawned. Therefore, there was a need to store
application-dependent data in nonephemeral
spawned processes.
Articles
Work in this area has been done on concurrent objectoriented systems, concurrent Lisp dialects, programming
models, and resource allocation.
State of this type was implemented in
CAREL as writable closure variables. These
process closures could be used as elements in
pipeline computations or as representations
of mutable communicating program objects,
for instance, to represent real-world objects
with state. State, as encapsulated in communicating objects, and the idea of pipeline parallelism have been pivotal in the design of
the other systems developed in the project.
The CAREL project was used primarily as a
feasibility study and was soon discontinued.
CAOS
The first implementation of the Elint application, described later, was made without the
benefit of any problem-solving framework. It
was anticipated that the application could
easily be mounted almost directly on the
CARE machine and that some experiments
could be run quickly, which would allow project members to learn some important lessons
early in the project.
To mount the application, a distributed
object-oriented system was implemented; at
the time, the CARE system did not come with
its own preferred object system. The system
was called CAOS (concurrent asynchronous
object-oriented system) (Schoen 1986). It was
implemented using the Flavors system supported by the project’s Lisp machines. It had a
number of key features.
First, each CAOS object was potentially a
multiprocess object, although executing on a
single processor, with at least one stack group
associated with each CAOS object.
Second, CAOS objects were intentionally
large grained. Because it was anticipated that
the communications network would be the
resource most competed for, the programmer
was encouraged to perform a lot of computation to reduce the number or size of the messages sent.
Third, message passing was used as the
metaphor for communication in the language
extensions provided by CAOS.
Finally, a large number of different message-sending primitives were defined, including nonblocking sends that did not require a
reply from the message target, sends which
returned futures to the values returned by the
targets, and send operations that immediately
blocked to wait for a reply from their targets.
The CAOS system proved to be too expensive to use for future experiments. Contrary
to the intuition of the project members, the
communications network proved to be the
least loaded of the CARE machine’s resources
during our experiments on CAOS. The computational expense of supporting its complex
object model caused the granularity of the
resulting computations to be too large.
LAMINA
Lamina (Delagi, Saraiya, and Byrd 1986) is the
object system that was designed after the
lessons learned from the CAOS experiments.
It was originally intended to provide a small,
lightweight layer on top of the CARE machine
so that distributed object-oriented programs
could be efficiently implemented. A significant part of the motivation for Lamina’s
design was the desire to reduce the overhead
suffered by the CAOS system in terms of associating large stack groups with each of the
CAOS objects. Lamina introduced the idea of
objects with restartable, rather than resumable, code segments, which do not require
stacks to preserve their state when they are
not running. Since its first appearance,
Lamina has been extensively developed, and
although still small and lightweight, it now
provides a platform for the development of
computational models for functional and
shared-variable, as well as object-oriented,
programming.
Lamina has been used to implement a
number of programs, both for direct implementations of the two real-time expert systems being investigated (see Applications)—
AirTrac and Elint—and for a number of
numeric programs. Lamina is now the preferred core programming system for the CARE
machine. Applications in Lamina have consistently shown the highest performance of all
programs running on the CARE machine.
Interprocessor and Interprocess
Communication
Different mechanisms for interprocessor and
interprocess communication have been
WINTER 1989
31
Articles
?W2@@@@@@@@6K?
O.M?g?I4@6K
O20YhfI46X
W20M
I/X?
?O.M
?V/X
?O20Y?
V/X?
?W20M?
?V/X
O.M?
V/X?
O20YgO2@@@@@6K?hf?V/X
W20MgW20Mf?I46X?hfV/X?
7<h7<h?I/Khf?V/X
?J5?g?J5?heV46KhfV/K?
W.Y?gW.Y?hfI46Xhe?V46X?hg
7HeW&f7H
I/X?he?I/Xhg
?J5?e75f@?
?V/XhfV/X?hf
?7H??J@Hf@?
V/X?he?V/Xhf
J5eW&@Lf@?
?V/XhfN1hf
?W.Ye7>@)X??J5?
V/X?he?3L?he
?7H?e@@R')X?7H?
?V/Xhe?V/Xhe
?3L??J@@LN@)X@
V/X?heV/X?h
?N1??7<I/X@V'@
?V/Xhe?V/Xh
3L?@eV'@LV'
V/X?heV/X?g
V/T5e?V'1
?V/Xhe?N1?g
?N@HfV'
N1hf3Lg
??
@?
?3L?heN1g
?N1?he?3L?f
?O2@@@@@@@@@@@@6K?
3Lhe?N1?f
O2@@@@@@@@@@@@@@@@@@@@@6X?
??
N1hf@?f
?O2@@@@@@@@@@@@@@?@@@@@@@@@@)K
?@hf3Lf
W2@@@@@@@@@@@@@@@@@?@@@@@@@@@@@@6K
??
?@hfN1f
?O&@@@@@@@@@@@@@@@@@@?@@@@@@@@@@@@@@6X
?@hf?3L?e
?O2@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@)X?
??
J5hf?N1?e
?W2@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@)X
?
?W.Y
@?e
O&@@@@@@@@@@@@@@@@@@@@@@@@@@@?@@@@@@@@@@@@@)X?
W.Y?hf?J5?e
O2@@@@@@@@@@@@@@@@@@@@?@@@@@@@@X@@@@@@@@@@@@@@)X
??
?W.Y
?7H?e
W2@@@@@@@@@@@@@@@@@@@@@@?@@@@@@@V@@@@@@@?@@@@@@@@)X?hg?
W.Y?
J5f
?W&@@@@@@@@@@@X@@@@@@@@@@5?@@@@@@@?@@@@@@@?@@@@@@@@@)Xhg?
7H
?W.Yf
W&@@@@@@@@@@@V@@@@@@@@@@@HJ@@@@@@@?@@@@@@@?@@@@@@@@@@1hg?
?J5?
W.Y?f
?W&@@@@@@@@@@@@X@@@@@@@@@@@?7@?@@@@@X@@@@@@@?@@V'@@@@@@@hg?
W.Y?
7Hg
?7@@@@@@@@@@@@V@@@@@@@@@@@@?@@?@@@@V@@@@@@@@@@@?V@@@@@@@L?hf?
7H
?J5?g
J@@@@@@@@@@@@@?@@@@@@@@?@@@W@@?@@@@?@@@@@@@@@@@@@@@@@@@@1?hf?
?J5?
?7H?g
?O&@@@@@@@@@@@@@?@@@@@@@@?@?@(Y@?@@@@?@@@@@@@@@@@@@@@@@@@@@?hf?
?7H?
J5h
?W2@@@@@@@@@@@@@@@?@@@@@@@@?@?@H?@?@@@@?@@@@@@@?@@@@@@@@@@@@@Lhf?
J5
?W.Yh
?7@@@@@?@@@@@@@@@@?@@@@@@@@?@?3L?@?@@@@?@@@@@@@?@@@@@@@@@@@?@1hf?
?W.Y
?7H?h
?3@@@@@?@@@@@@@@@@?@@@@@@@@?@LN1?@?@@@@?@@V4@@@?@@?@@@@@@@@?@@L?he?
?7H?
J5he
?V@@@@@?@@@@@@@@@@?@@@@@@@@?@)X@?@?@@@@?@@L?W@@?@@?@@@?@@@@@@@1?he?
J5
7Hhe
?W2@@@@@@?@@@@@@@@@@@@@@@@@@@?@@@@?@?@@@@?@@1?7@@?@@?@@@?@@@@@@@@Lhe?
?W.Y
?J5?he
?7@@@@@@@?@@@@@@@@@@?@@@@@@@@?@@@@?@?@@@@?@@@?3@@?@@?@@@?@@@@@@@@1he?
W.Y?
W.Y?he
?@@@@@@@@?@@@@@@@@@@?@@@@@@@@?@@@@?@?@@@@?@@@?N@@?@@?@@@?3@@@@@@@@he?
?W.Y
?W.Yhf
J@@@@@@@@?@@@@@@@?@@?3@@@@@@@?@@@@?@?@@@@?@@@??@@X@@?@@@?N@@@@@@@@he?
?*U?
W.Y?hf
?W&@@@@@@@@X@@@@@@@?@@?N@@@@@?@@@@@@?@?@@@@?@Y@??3>@@@?@@@e@@@@@@@@he?
?N1?hf?W.Yhg
W&@@@@@@@@V@@@@?@@@?@@e@@@@@?@@@@@@?@?@@@@?e@??S@@@@?@@@?J@@@X@@@@he?
@?hfW.Y?hg
7@@@@@@@@@?@@@@?@@@?@@L?@@@@@?@@@@@@?@?@@?@?@?e?7@@@@?@@@?7@@V@@@@@L?h?
3Lhe?W.Y
?J@@@@@@@@@@?@@@@?@@@?@@1?@@@@@?@@@@@@?@?@5?@?@?e?@@@@@?@@@?@@@X@@@@@1?h?
N1heW.Y?
?7@@@@@@@@@@X@@@@?@@@?@@@?3@@@@?@@@@@@?@?@H?@?@?e?@@@@@?@@@?@@V@@@@@@@?h?
?@h?W.Y
?@@@@@@@@@@V@@@@@?@@@?@@@?N@@@@?@@?@@@?@?@?J@@@?eJ@@@@@?@@@?@@?@@@@@@@?h?
?3L?gW.Y?
J@@@@@@@@@@?@@@@@?@?e@@@?J@@@@?@@?@@@?@e?7Y@@?e7@@@@@?@@@?@@?@@@@@@@?h?
?N1?f?W.Y
7@@@@@@@@@@?@@@@@X@@6X@@5?7@@@@?@@?@@@?@?@?@W@@?@?@@@@@@?@@@@@@?@@@@@@@Lh?
3LfW.Y?
?J@@@@@@@@@@@?@@@@V@@@@@@@H?@@@@@?@@?@@@?@?@?@@Y@?@?@@@@@@?@@@@@@?@@@@@@@1h?
N1e?W.Y
?7@@@@@@@@@@@?@@@@?3@@@@@@e@@@@@?@@?@@5?@@@?@@?@?@?@@@@@@?@@@@@@?@@@@@@@@h?
?@eW.Y?
?@@@@@@@@@@@@?@@@@?V@@@@@@e@@@@@?@@X@@Y?@@@?@@?@?e@@@@@@L??@@@@?@@@@@@@@h?
?3=O.Y
?@@@@@@@@?@@@W@@@@@@@@@@@@e@@@@@?@V@@@@@@@@?@5?@?e@@@@@@)K?@@@@?@@@@@@@@h?
?N@(Y?
J@@@@@@@@?@@@@U?W@@@@@@@@@e@@@@@?@?@@@@@@@@?3U?@?@@@@@@@@@@@@@@@W@@@@@@@@L?g?
(Y
7@@@@@@@@?@@@@)?&@@@@@@@@@e@@@@@?@?@@@@@@@@?S1?e@@@@@@@@@@@@@@@@Y@@@@@@@1?g?
@@@@@@@@@?@@@@@@@@@@@@@W@@L?@@@@@?@?@@@@@@@5?7@?e@@@@@@@@@@@?@@@@?@@@@@@@@?g?
3@@@@@@@@?@@@@@@@@@@@@@@<B1?@@@@@?@?@@@@@@@U?@5?e@@@@@@@@@@@?@@@@?@@@@@@@@?g?
?@@@@@@@@?@@@@@?@@@@@@@@e@?@@@@@?@?@@@@@@V1?3U?e@@@@@?@@@@@?@@@@?@@@@@@@@?g?
?W&@@@@@@@@?@@@@@?@@@@@@@@e@?@@W@@?@?3@@@@@?@?N)?2@@@@@@?@@@@@?@@@@?@@@@@@@@Lg?
?7@@@@@@@@@?@@@@@?@@@@@@@@e@?@@@>5?@?N@@@@@?@e@@@@@@@@@?@@@@@@@@@@?@@?@@@@@1g?
?3@@@@@@@@@?@@@@@?@@?@@@@@e@?@@@@U?@??@@@@@?@L?@@@@@@@?@?@@@@@@@@@@?@@?@@@@@@g?
@@@@@@@@@?@@@@@?@@?3@@@@e@?@@@@1?@?J@@@@@?@1?@@@@@@@?@?@@@@@@@@@@?@@?3@@@@@g?
?7@@@?@@@@@@@@@@@?@@?N@@@@e@?@@@@@?@?7Y@@@@@@@?@@@@W@@?@?@@@@@@@@@@@@@LN@@@@@g?
?@@@@?@@@@@@@@@@@?@@e@@@@e@@@@@@@W5?@?@@@@@@@?@@@@@Y@?@?@@@@@@@@@@@@@)X@@@@@g?
?@@@@?@@@@@@@@@@@?@@e@@@@e@@@@@@@@H?@?@@@@@@@?@@@@@W5?@?@@@@@@W@@@@@@@@@@@@@g?
?@@@@@@@@@?W@@@@@?@@L?@@@@e@@@@@@@@L?@?@@@@@@@?@@W@@@H?@?@@@@@@@Y@@@@@@@@@@@@L?f?
J@@@@@@@@@W&@@@?e@@)X@@@@e@@@@@@@V)X@?@@@@@@@?@@@Y@@e@?@@@@?@@?@@@@@@@(Y@@@1?f?
7@@@@@@@@@@Y@@@?@?@@V@@@@@?J@@@@@@@?@@@?@@@@@@@X@@@?@@f@@@@X@@?@@@@@@@H?@@@@?f?
3@@@@@@@@@@?@@@?@?@@?@@@@@?7@@@@@@@?@@@?@@@@@@V@@@@W@@f@@@V@@@?@@@@@@@??@@@@?f?
?@@@@@@@@@@?@@@@@?@@?@@@@@?@@@@@@@@?@@@?@@@@@@X@@@@@Y@f@@@?@@@W@@@@@@@??@@@@?f?
7@@@@W@@@@@@@@@@@?@@?@@@@@?@@@@@@@@?@@@?@@@@@V@@@@@@?@f3@@?@@@@Y@@@@@@??@@@@?f?
@@@@@@Y@@@@@@@@@@?@@?@@@@@?@@@@@@@@?@@@?@@@@@?@@@@@@?@f?@@?@@@@W@@@@@@L?@@@@?f?
@@@@@@?@?@@@@?@@@?@@?@@@@@?@@@@@@@@?@@@?@@@@@?@@@@@@@@f7@@?@@@@@R'@@@V1?@@@@?f?
@@@@@@?@?@@@@?@@@?@5?@@@@@W@@@@@@@@?@@@?@@@@@?@@@@@@@@?@e@@@?@@@@@?N@@5?@?@@@@?f?
@@@@@@W@T@@@@?@@@?3U?@@@@@@Y@@@@@@@?@@@?@@@@@?@@@@@@@5?@e@@@?@@@@@??@@HJ@?@@@@?f?
?J@@@@@@@R@>@@@?@@@?S1?@@@@@@?@@@@@@@?@@@?@@@@@?@@@@@@@H?@e@@@?@@@@@L?@@?7@?@@@@?f?
?@@@@@@@@?@@Y@@?@@5?7@?@@@@@@?@@@@@@@?@@@?@@@@@?@@@@@@@?J@e@@@?@@@@@1?@@?@@?@@@@?f?
?N@@@@@@@?@@@@@?@@U?@@?@@@@@@W@@@@@@@?@@@?@@@@@?@@@@@@@?7@e@@@?@@@@@@?@@?@@?@@@@?f?
@@@@@@@?@@@@@X@V)X@@?@@@@@@@Y@@@@@@?@@@?@@@@@?@@@@@@@?@@e@@@?3@@@@@?3@?3@?@@@@?f?
@@@@@@@?@@@@V@@?@@@@?@@@@@@@?@@@@@@?@@@?@@@@@?@@@@@@@?@@e@@@?V@@@@@e@?N@?@@@@?f?
@@@@@@@X@@@@X@5?3@@@?@@@@@@@?@@@@@@?@@@?@@@@@X@@@@@@@?@@e@@@@@@@@@@?7@?J5?@@@@Lf?
3@@@@@V@@@@V@@eN@@@?@?@@@@@?@@@@@@?@@@?@@@@V@@@@@@@@?@@e@@@@@@?@@@?@@?7U?@@@@1f?
N@@@@@?@@@@?@@)X?@@@X@?@@@@@?@@@@@@?@@@?@@@@?@@@@@@@@?@@e@@@@@@?@@5?@@?@)X@@@@@f?
?@@@@@@@@@@?@@@1?@@>@@X@@@@@?3@@@@@?@@@?@@@@X@@@@@@@@?@@e@@@@@@?@@H?@@?@@@@@@@@f?
?@@@@@@@@@@?@@@@?3>@<B@@@@@@?N@@X@@@@@@?@@@>@@@?@@@@@X@@e@@@@@@?@5??@@?@@@@@@@@f?
J@@@@@@@@@@?@@@@?N@@e@@@@@@?J@V@@@@@?e@@>@@@5?@@@@>@@@e@@@@@@?@H??@@?@@@@@@@@f?
7@@@@@@@@@@?@@@@e@@?J@@@@@@?7@X@@@@@?e?S@@@@U?@@@>@@@@e@@@@@@?@e?@@?@@@@@@@@f?
@@@@@@@@@@@?@@@5f?7@@@@@@?@V'@@@@@?eW&@@@V/?@?S@@@@@e3@@@@@?@L??@@?@@@@@@@@L?e?
@@@@@@@@@@@?@@@Uf?@@@@@@@?@?N@@@@@?e7@?@@?e@W&@@@@@eN@@@@@?@1??@@?@@@@@@@@1?e?
W2@?
??
@@@@@X@@@@@?@@V1fJ@@@@@@@?@e@?g@@?3@?@?@@@@@?@@?@X@@@@@?@@?J@@@@@@@@@@@@?e?
7R'L
@@@@>@@@@@@?@5?@e/T&@@@@@@@?@e@?@@@@e@@?V'X@?3@@@@?@@?@@@@@@@?@@?7@@@@@@@@@@@@?e?
@LV/X?
??
@@@>@@@@@@@@@HJ5eN@Y@@@@@@@?@e@?3X@@e@@eN@@?N@@@@?@@?@@@@@@@X@5?@@@@@@@@@@@@@?e?
@@6?-T@1?N1?
??
@@>@@@@@@@@@@?7HeJ5?@@@@@@@?@e@?N@@@@@@@T&X@@??@@@@?@@?@@@@@@V@@H?@@@@@@@@@@@@@?e?
@?@@@0R'e@?
3>@@@@(Y@@@@@X@Le7U?@?@@@@@g?@@@@@@V@@@@@??@@X@?@@X@@@@@@?@@?J@@@@@@@@@@@@@?e?
@@@Xg@L
S@@@@@H?@?@@V@@1e@1?3X@@@@@?W&?e?@@@@@@?@@@@@L?@V@@?3>@@@@@@@?@@?7@@@@@@?@@@@@5?e?
S)K?f31
??
7@@@@@e@?@@?@@@e@@?N@@@@@@?7@W2@@@@@@@@?@@@@@)T@X@@?S@@@@@@@@X@5?@@@@@@@?@@@@@H?e?
@@@@@>@6X?eN@L?
?
@@@@@@e@?@@X@@5e@@L?@@@@@@?@@@>@@@@@@@@?@@@@@@@R@@5?&@@@@@@@>@@H?@@@@@@@?@@@@@f?
3@eI'@V)Xe?@)X
@@@@@@e@?3>@@@He?S)X@@@@@@?@@@@Y@@@@@@@?@@@@@@@?@@Y?N@@@@@@>@@@??@@@@@@@?@@@@@f?
V4@6KS@@@)fB1
??
@@@@@@?J5?S@@@@?e?*>@@@@@@@?@@@@?@@@@@@@?@@@@@@@?@@@@?@@@@@>@@@@L?@@@@@@@?@@@@@f?
?@@@<?g?@
??
@@@@@@?.YW&@@@@?e?S@@W@@@@@?@@@@?@@@@@@@?@@@@@@@?@@@@?@@@@V@@@@@1?@@@@@@@X@@@V@f?
?@@@@0Y@L?g?3L?
@@@@@@eO&@@@@@?eW&@@@>@@e?@@@@?@@@@@@@?@@@@@@@?@@@@W@@@@?@@@@@@?@@@?@@>@@@5g?
?@@Xe?@)Xg?V/X
@@@@@@?@@@@@@X@?e7@@@@@>5eJ@@@@?@@@@@@@?@@@@@@@X@@@@@Y@@@?@@@@@@?@@@?3>@@@@H?W&?e?
I4@@@@@1hV/X?
??
@@@@@@?@@@@@V'@?e@@@@@@@Ue7@@@@?@@@?@@@?@@?@@@>@@@@@@?@@@?@@@@@@?@@@?S@@@@@L?7@?e?
?W@@h?N)X
@@@@@@@@@@@@9S5?e@@@@@@@1e@@@@@?@@@?@@5?@@W@@>@@@@@@@W@@@?@@@@@@?@@@W&@@@@@1?@@?e?
@@<?he31
??
@@@@@@@@@@@V@@H?e@@@@@@@@e@@@X@?3X@?@@U?@@@8S@@@@@@@@(R'@?3@@@@@?@@@@@@@@@@@?3@?e?
3@?O2@6XgV'L?
?W2(hg?
@@@@@@@@@@@X@@L??J@@@@@@@@?J@@V@@?S@@W@>@@@@@W&@@@@@@@@H?S5?N@@@@@?@@@@@@@@@@@?N@?e?
V4@@0?')X?f?N)X
O&(Yhg?
@@@@@@@@@@>@@@@??7@@@@@@@@?7@@?@@W&@@@R@>@@@@@@@@@@@@@@=O&U??@@@@@?@@@@@@@@@@@e@?e?
V'1?g31
W2@0Y?hg?
@@@@@@@@@>@@@?eJ@@@@@@@@@?@@@?@@@@@@@@@@R4@@@@@@@@@X?S@@@)X?@@@@@?@@@@@@@@@@@e@?e?
?V'LgN@
?O&0M?@@@@@@@@>@@@@??W&@@@@@@@@@@@@@?@@@@@@@@@@?eW@@@@@@@)?&@@@@1?@@@@@?@@@?@@@@@@@e@?e?
N)X?f?3L?he?W2(M?
??
@@@?@@@>@@@@@??*@@@@@@@@@@@@@@?3@@@@@@@@@?O2@UI'@@@@@@@@@W@@X@@@@@?@@@?@@@X@@@e@?e?
?31?f?N1?heW&0Y
??
@@@@@@V@@@@@@??N@@@@@@@@@@@@@@LV'@@@@@@@@@@@@)XV4@@@@@@@@0MS@@@@@@?@@@?@@V@@@@?J@?e?
?N@Lg@Lh?O.M
@@@@@@X@@@@@@?e@@@@@?@@@@@@@V)XN@@@@@@@@?@@@@)KheW&@@@@@@@@@@?@@X@@@@?7@?e?
3)X?f3)X?f?W2(Y?
??
@@@@@V@@@@@@@?@?@@@@@?@@@@@@@?@1?@@@@@@@@?@@@@@@@@@@@@@@6T&@@@@@@@@@@@eB@@@@@?@@Le?
V'1?fV'1?fW&(Y
@@@@@X@@@@@@@T@X@@@@@?@@@@@@@@@@W(Y@@@@@@?@@@@@@@@@@@@@@V@@@@@@@@@@@@@e?@@@@@?@@1e?
?N@?f?V'=e?O&(Y?
O&e?
@@@@>@@@@@@@V@R@@@@@5?@@@@@@@@@@@H?@@@@@@?@@@@@@@@@@@@@@?@@@X@@@@@@@@@e?@@@@@?@@@e?
3LgV'6?2@0Y
?O2@@@e?
@@@>@@@@@@@@?@?@@@@@U?@@@@@@@?@@@??@@@@@5?@@@@@@@@@@@@@@?@?B@@@@@@@@@@?/X@@@@@?@@@e?
N1g?V4@0M
?O2@0Mf?
@?S@@@@@@@@@?@?@@@@V1?@@@@@@@@@@@??3@@@@H?@@@@@@@@@@@@@@?@?J@@@@@@@@@5?N@@@@@@?@@@e?
?@L?
?W2(M?g?
@?7@@?@@@@@@?@?@@@@?@?@@@@@@@@@@@??N@@@@e@@@@@@?@@@@@@@e?7@@@@@@@@@He@@@@@@?@@@e?
?31?
O&0Yh?
@?@@@?@@@@@@e?@@@@?@?@@@@@?@@@@@Le@@@@e@@@@@@?@@@@@@@L??@@@@@@@@@@?e@@@@@@?@@@e?
?N@L
O20Mhe?
@@@@@?@@@@@@e?@@@@@@?@@@@@?@@@@@)X?g@@@@@@?@@@@@@@1??@@@@@@@@?@?e@@@@@@@@@@e?
3)X?
O20Mhf?
@@@@@?@@@@@@e?@@@?@@?@@@@@?@@@@@@)?@@@@?J@@@@@@?@@@@@@@@??@@@@@@@@?@L?J@@@@@@@@@@e?
V')X
O2@@0Mhg?
?J@@@@@?@@@@@@e?@@@?@@?@@@@@?@@@@@@HJ@@@5?7@@@@@@?@@@@@@@@L?@@@@@@W5?@1?7@@@@@@@@@@e?
?V')X?hfO2@0M?
??
?7@@@@@@@@@@@@e?@@@?@@?@@@@@?@@@@@@?7@@@U?@@@@@@@?@@@@@@@V1?@@@@?@@H?@@?@@@@@@?@@@@e?
V'1?heW2(M ?3@@@@@@@@@X@@e?@@@?@@?@@@@@?@@@@@@W@@@>1?@@@@@@@?@@?@@@5?@?@@@@X@@e@@@@@@@@@?@@@@e?
?V'=h?O&0Y?
?N@@@@@@@@V@@@?@?@@@@@@?@@@@@@@@@@@@@Y@V@@?@@@@@@@?@@W@@@H?@?3X?B@@@e@@@?@@@@@W@@@@e?
V'6XfO2@0M?
??
@V'@@@@@X@@@?@?@@@@@@?@@@@@@@@@@@@@?@?@@?@@@@@@@?@@@Y@@?J@?S1?J@@@e@@@?@@@@@@Y@@@e?
?V')?2@@0M
??
@LN@@@@V@@@@@@W@@@@@@?@@@@@@@@V'@@@W5?@@?@@@@@?@?@@@?@@?7@?7@?7@@@e@@@?@@@@@@?@@@e?
V4@0M?
@1?@@@@?@@X@@@@Y@@@@@W@@@@@@@@e@@@@H?@@W@@@@@?@?@@@W@@?@@?@@?@@@@e@@@?@@@@@@?@@@e?
??
@@?@@@@??V@@@@@W@@@@@@Y@@@@@@@?7@@@@e@@@>@@@@?@?@@@@Y@?@@?@@?@?@@e@@@?@@@@@@@@@@e?
@@?@@@@@@@@@@@@@R4@@@@W@@@@@@@?@@@@@e@@@@>@@@?@?@@@@W@X@@@@@@@?@@e@@@?@@@@@@@@@@e?
?
@@@@@@@@@@@@@@@@e?@@@@Y@@@@X@@@@@@@L?3@@@@>@@X@?3@@@@R@@@@@@@@?@@e@@@W@@@@@@@@@@e?
@@@@@@@@@@@@@@@@@@@@@@@?@@@>@@@@@@@@1?N@@@@@<B@@?N@@@@?@@@@@@@@?@@?J@@@@Y@@@@@@@@@e?
??
@@@@@@?@@@@@@@@@@@@?@@@?@@>@@@@@@@@@@L?@@@@@?J@@?J@@@@?@@@@@@@@?@@W&@@@@?@@@@@@@@@e?
@@@@@@?@@@@@@@@@?@@?@@@?3>@@@@@@@@@@@)X@@@@@?7@@?7@@@@?@@@@@@@@?@@@@@@@@?@@@@@@@@@e?
??
@@@@@@?@@@?@@@@@?@@?@@@?V'@?W@@@@@@@@V@@@@@@?@@@?@@@@@?@@@@@@@@?@@@@@@@@?@@@?@@@@@e?
@@@@@@?@@@?3@@@@@@@?@@@??N@?7@@@@@@@@?@@@@@@?@@@@@@@@@W@@@@@@@@?@@@@@@@@?@@@?@@@@@e?
@@@@@@?@@@LV4@@@@@@?@@@@6X@?@@@@@@@@@?@@@@@@@@@@@@@@@@@Y@@@@@@@?@@@@@@@@?@@@?@@@@@e?
@@@@@@?@@@)Ke?@@@@?@@@@@@@?@@@@@@@@@@@@@@@@@@@@@@@@@@@W@@@@@@@?@@@@@@@@?@@@?@@@@@e?
?J@@@@@@?@@@@@@@@@@@@?@@@@@@@W@@@@@@@@@@@@@@@@?@@@@@@@@@@@Y@@@@@@@@@@@@?@@W@@@?@e@@e?
?7@@@@@@@@@@@@@@@@@@@?@@@@@@@@Y@@@@@@@@@@@@@@@?@@@@@@@@@@@?@@@@@@@@@@@@?@@@>@@?@?J@@e?
?
?3@@@@@@@@@@@@@@@@@@@W@@@@@@@@@@@@@@@@@@@@@@@@?@@@@@@@@@@@?@@@@@@@@@@@@W@@@@>5?3T&@@e?
?
?N@@@@@@@@@@@@@@@@@@@@Y@@@@@@@@@@@@@@@(MI4@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@Y@@@@U?N@@@@e?
?@6Xg?
@@@@@@@@@@@@@@@@?I4@?3@@@@@@@@@@@@@(Y?e?@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@)e3@@5e?
?@he?N@1g?
3@@@?@@@@@@@@@@5f?V'@@@@@@@@@@@(Yf?3@@@@@@@@@@@@@@@(M?I'@@@@@@@@@@@0M?eV40Ye?
?@hf3@g?
N@@@@@@@@@@@@@0YgV'@@@@@@@@@(Y?f?V40?'@@@@@(M?I40Ye?V'@@@@@@@@0M?he?
?@hf?@g?
?3@@@@@@@@@W(Mh?V'@@@@@@@0Yhe?V'@@@@HheV'@@@@@0M?hf?
?@hf7@g?
?N@@@@@@@@@0Y?heV'@@@@
V'@@5?he?V4@0M
??
@@@@@@@0M?hf?V'@@5 ?@hf@@g?
?V40Y?
?@hf@@g?
@@@@
V40Y ?@hf@@g?
3@@5
???
?@L?he@@g?
V40Y
?@1?he3@L?f?
?
?3@?heV'1?f?
?N@?he?N@?f?
@?hf@Lf?
@?hf@1f?
@Lhf3@f?
31hfN@f?
N@hf?@f?
?@hfJ@f?
J@hf7@L?e?
@@hf@@1?e?
N@hf@?@?e?
?@hf@?@?e?
?@hf@@5?e?
?@hf@@H?e?
?@L?he@@f?
?@1?he@@f?
?@@?he@@f?
?3@@(?h@@f?
?N@(Y?h@@L?e?
@Hhe3>1?e?
@?heN@@?e?
@?he?@@?e?
@?he?@@?e?
@?he?@@?e?
@?he?@@?e?
@?he?@@?e?
@?he?@@?e?
@?he?@@?e?
@?he?@@?e?
@Lhe?@@?e?
31he?3@?e?
N@he?N@Le?
?@hf31e?
?@hfN@e?
?@hf?@e?
?3L?he?@e?
?N1?he?@e?
@?he?@e?
@?he?@e?
@?hg?
@?hg?
?J@?hg?
W&5?he?@e?
?O2@@@@@@6K?he?O&0Y?he?3L??
?@@@@?fI4@@@@@@6T2@@@@0M?hf?S1??
J(MW@?hfI40M
?75??
7HW&@L?'6X
?@e?
@?7(R/?S@)X?
?31??
@?@HeW&<I/?
@??
@?@?@?75
?7@??
3X@?@?@H
J@@??
N@5?e@?
?O&<e?
?@H??J@?
'@5?e?
?3L??@@?
?O2@@?V+Y?e?
?V/Khf?O@?@@@6T2@@@@@@@0M?h?
V4@@@@@@@@@@@@@@@?@@@@0M
?
?
?
?
?
?
10T
?
?
?
?
?
@@
?
@6X?@@L?
?
@@1?@V1?
?
@@@?@?@?@?@?
?
@?@?@?3@5?@?
?
3@@T@LN@Y?@?@6X?
?
?@V'>)T@@@@W@@1?
?
'@?N@@@<I'@@<?@?
?
S5e3X@??V4@?J@?
?
?W2@@UeV'@?f?75?
?
?7UI4)e?V'?f?@
?
?@)Xhf?3)X
?
I/K?he?V')X?
?
?V4@@6K?hV4)K
?
I4@@heI'6K
?
I'=?h?V4@6X
?
?V4@6Xh?I')K?
?
I/X?hV4@6K?
?
?V/Khe?I4@6Khg?
V'6Xhe?I4@6Xhf?
?O2@@@@@@@@@@@@@@@6K
?V4)K?he?I')X?he?
O2@@@@@@@@@@@@@@@@@@@@@@@@@@6K
O2@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@6X? ?I'6K?heV')Xhe?
V4@6K?h?V')X?h?
O2@@@@@@@@@@X?@@@@@@@@X?e@@@@@@@@@@)K
?I4@6XhV4)Xh?
O2@@@@@@X?e?V@@@@@@@@XV@@@@@@@@@@@@@@@@6X
?I')X?hB)X?g?
O2@@@@@@XV@@@@@@@@@@@X?V@@@?W@@@@V'@@@@@@@@)X?
V4)Xh?3)Xg?
?@@@@@@@@>@@@@@@@@@@0MS@@@@@@W&@@@@?S@@@@?@@@@)X
B)X?g?V'1g?
?@@@@@@@V@@@@@@@@@@??O&@X@@@@@@@@@@@@U@@@?@@@@@)X?
?31?h?@g?
W2@@@@@@@@?@@@@@@@@@@@@@UV@@@@@@@@@@@@V@@@@?@@@@@@)X
@?h75g?
?O&@@@@@@@@@@@@@@@@@@@@@@>@@@@@?@(M?W@@@X@@@@?3@@@@@@)X?
?75?h@Hg?
?W2@@@@@@@@@@@@@@@@@@@@@@?V@@@@@@X@H?O&@@V'@@@@??@@@@@@@)X
J@H?h@?g?
?7@@@@@@@Xe@@@@?@@@@@@@@@@@@@@@>@@W2@@@@?V@@@@?7@@@@@@@@)X?
?W&5h?J5?g?
J@@@@@@@>@@@@@@@?@@@@@@@@@@@@@?S@@@@@@@@@@@@@@@?@@@@@@@@@@)XW&(Yh?7H?g?
?W&@@@@@XV@@@@@@e?@@@@@@@@@@@@@W&@@@@@@?@@@@@@T@X@@@@@@@?W@@)K?hg
7(Y?h?@h?
W&@@@@@V@@@@@@@@T-X@@@@@@XW@@@@@@@@@@@@@?@@@@@V@R@@@@@@@@T&@@@@6X?hf
?J(YheJ5h?
?W&@@@@@@X@@@@@@@>@R@@@@@XS@@U?@@@@@@@@@@@?@@@@@X@@@@@@@@@>@@@@@@@)Xhf
W.Y?h?W&Hh?
W&@@@@@XV@@@@@@XV@@X@@@@>@@8S@@@@@@X@@@@@@X@@@XV@@@@@@@@XV@@@@@@@@@)X?he
?W&Hhe?75?h?
7@@@(MS@@@@@@XV@@@>@@@@V@@5?7@@@X?;@@@@@XV@@@V@@@@@@@@@V@@@@@@@@@@@@)Xhe
?75?heJ(Y?h?
?J@@@@YO&@@@@?V@@@0R@@@@@@@(YJ@@?V@6X@@@@>@@@@@?@@@@@X@@5?@@@@@@@X?@@@@)X?h
J(Y?he7Hhe?
O&@@@@@@@@@@@@@@@?e@@@@@@(YW&@@@@@>@@eV@@@@@@@@@@@V@@(YJ@@@@@XV@@@@@@@1?h
?W&Hhe?J@?he?
W2@@@@@@@@X?@@@@@@@@@@@@@@@@UO&@@@@@>@@@W2@@@@X@@@@@@@X@(YW&@@@@>@@@@@@@@@@Lh
?75?he?75?he?
?W&@@@@@@@XV@@@@@@@@@@@@@@@@@V@@@@@@@>@@@@@@@@@>@@@@@@@>@@UO&@@@0R'@@@@@@@@@@)X?g
J@H?he?@H?he?
W&@@@@@@@>@@@@@@@@@@@@@@X@@@@X@@@@@@>@Y@@@@@@?S@@@@@@@V@@V'@@@@XeN@@@@@@@?W@@1?g
?W&5hf?@hf?
?W&@@@@@@@>@@@@@@@@@@?e?V@@@@V@@@@@?S@@X@@@@@@T&@@@@@@@?@@??@@@>@6K?@@@@@@@T&@@@Lg
?7(YhfJ5hf?
?7@@@@@@@>@@@@@X@@@@@@@@@@@@@@X@@@@@W&@>@@@@@@V@@@@@@@@@?@5?7@@V@@@@@@@@@@@>@@@@@1g
?(Y?hf7Hhf?
J@@@@@@XV@@@@@V@@@@@@@@@@@@@@V@@@@@@@@>@@@@@@@@@@@@@@X@@@@HJ@@@?@@@@@@@@@@>@@@@@@@L?f
?J5?hf?
?W&@@@(MS@@@@@@@X@@@@@@?@@@@@@5?@@@@@@@V@@@@@@X@@@@@@@V@@@@@T&@@@@@@@@@@@X?S@@@@@@@@1?f
?7H?hf?
W&@@@@YO&@@@@@XV@@@@@@@X@@@@@(YJ@@@@@@@X@@@@@>@@@@@X@@X@@@?V@@@@@@@@@@@XI/T&@@@@@@@@@Lf
J5hg?
7@@@@@@@@@@@@V@@@@@@@@>@@@@@@H?7@@@@@(R@@@@@V@@@@@>@(R@@@@@@@@X@@@@@@@>1?S@@@@@@@@@@@1f
?W&Hhg?
?J@@@@@@@@@@@@@?@@@@@@@>@@@@@@5?J@@@@@@U?@@@@@X@@(MS@@U?@@@@@@XV@@@@@@@>@@W&@@@@@@@@@@@@f
?75?hg?
W&@@@@@@@@@@@@@X@@@@@XV@@@@@@(YW&@@@(MS@@@@@@V@@(YW&@V@@@@@@@>@@@@@@@(R@@@@@@@@@@@@@@@@@L?e
J(Y?hg?
7@@@@@@@@@@@@(R@@@@XV@@@@@@@(YW&@@@@HW&@@@@@@?@@UO&@@?@@@@@@>@@@@@@?@HJ@@@@@@@0Y@@@@@?@@1?e
7H
?
?J@@@@@@@@@@@@@HJ@@@>@@@@@@(?'U?7@@@@@T&@@@@@@5?3>@@@@5?@@@@@V@@@@@@@X@W&@@@@@@?e@@@@@@@@@?e
?J@?
?
W&@@@@X?@@X@@@@?7@@>@@@@@@(Y?S)X@@@@@V@@@@X@@(Y?N@@@@@Y?@@@@@X@@@@@@V@@@@@@@@@@W2@@@@@@@@@@?e
?75?
?
7@@@@>@@@V@@@@@X@@>@@@@@@(YW2@R@@@@@@X@@@>@@@YO-X@@@@@@@@@@?V@@@@@@@X@@@@@@@@@@@@@@@@@@@@@@?e
?@H?
?
?J@@@@>@@@@X@@@@>@@>@@@@@@@UO&@@X@@@@@V@@@>@@@@@@>@@@@@@@@@@@@@@@@@@@V@@@@@@@@@@@@@@@@@@@@@@@?e
J@
?
W&@@@>@@@@>@@@@>@(R@@@@@(MS@@@@V@@@@@@@@@V@@@@@XV@@@@@@@@@@@@X@@@@@@@?@@@@@@@@@@@@@@@@@@@@@@@?e
@@
?
7@@@V@@(MS@YeV@@Y?@@@@(YW&@@@@@@@@@@@@@@T@T@@>@@@@@@@@@@@@@V@@@@@@@@?@@@@@@@X@@@@@@@@@@@@@@@?e
?
?J@@@@?@@YO&@@@@@@@@@@@@(YW&@@@@X@@@@@@@@@>@R@YV@@@@X@@@@@@@@@X@@@@@@@@@@@@@@@V@@@@@@?@@@@@@@@@?e
?
?7@@@@@@@@@@@@@@@@@@@@@@UO&@@@@>@@@X@@@@XV@@X@@@@@@V@@@@@@@@@>@@@@@@@@@@@@@@@@X@@@@@@X@@@@@@@@@?e
?
J@@@@@@@@?@@@@@@X?@@@@@V@@@@@@V@@XV@@@@V@@@V@@@@@@@T@X@@@@@@>@@@@@@@@@@@@@@@@>@@@@@@V@@@@@@@@@@?e
7@@@@@X@@X@@@@@V@@@@@@@X@@@@@5?(R@@@@@@?@@@X@@@@(MS@>@@@@@@>@@@@@@@@@@@X@@@@>@@@@@@@X@e?@@@@@@?e
?J@@@@@V@@>@@@@@@X@@@@@@>@@@@@@He?@@@@@@?@@>@@@@(YW&R@@@@@@V@@X@@@@@@@@V@@@@>@@@@@@(R@@W2@@@@@@@?e
W&@@@@@@@>@@@@@@>@@@@@XV@@@@@@5?@@@@@@@@@X@V'@@@(YW&@X@@@@@@T@>@@@@@@@@@X@@@>@@@@@@@H?@@@@@@@@@@@?e
7@@@@@@@V@@@@@XV@@@@(R@@@@@@@(YJ@@@@@@@@V@5??@@@U?7@V@@@X@@>@R@@@@@@X@@>@@@V@@@@@X@@e@@@@@@@@@@@?e
@@@@@@@@T@@@@>@@@@@(YJ@@@@@@@H?7@@@@@@@@?3U?7@(R/X@@X@@>@@>@@X@@@@@>@@>@@@@@@@@@>@@@@@@@@@@@@@@@5?e
?W2@@@@@@@@>@>@XV@@@@@@YO&@@@@@@@??@@@@@@@@@?V/T@@U?V@@>@@V@@>@@>@@@@@>@@>@@@@@@@@@V@@@@@@@@@@@@@@@@H?e
?7@@@@@?@@V'?@>@@@@@@@@@@@@X@@@@@@@@@@@@@@@@eS@UV@@@@V@@@T@>@@V@@@@@V4@>@@@@@@?@@@?@@@@@@@@@@@@@@@@f
?@@@@@@@@5?S@R@@@@@@@@@@@@V@@@@@@@@@@@@@@@@@L?*>@@@@@@X@@V'>@@@X@@@@@??V@@@@@@@X@@@?@@@@@@@@@@@@@@@5f
?@@@@@@@@U?7@@@@@@@@@@@@@@X@@@@@@@@@@@@@@@@>,?S@@@@@@V@@@LN@@@>@@@@Xe@@@@@@@@>@@@@?@@@@@@@@@@@@@@@Hf
W2@@@@@@@@S)X@@@@@@@@@@@@@@>@@@@@@@@@@@@@@@@>(YW&@@@@@@@@@>)X@@V@@@@>@6X@@@@@@@>@@@@@@@@@@@@@@@@@@@@@?f
?O&@@@@@@?@@@R@@@@@@@@X@@@@@>@@@@@@@@X@@@@@@@V@HW&@@@@@@@@@>@>@@@X@@@V@@>@@@@@@@V@@@@@@@@@@@@@@@@@@@@@@?f
?W2@@@@@@@@@@@@X@@@@@@@>@@@@@>@@@@@@@@V@@@X@@@@@5?&@X@@@@@@@V@R@@(R@@@@T@>@@@@@@?@?@@@@@@@@@@@@@?@@@@@@@5?f
?7@@@@@@@@@@@@>@@@@@@XV@@@@@V@@@@@@@@@@@@V@@@@@@U??V@@@@@@@@T@X@@HJ@@@>@>@@@@@X@?@?@@@@@@@@@@@@@?@@@@@@@H?f
?@@@@@@(Y@@@?V@@@@@@>@@@@@@@?@@@@@@@@@@@5?@@@@@>@@@@@@@@@@@>@R@@@W&@@>@R@@@@@>@@T@X@@@@@@@@@@@@@?@@@@@@@g
J@@@@@@H?@@@@@@@@@XV@@@@@@X??3@@@@@@?@@(YJ@@@@>@@@@@@@?@@@V@@X@@@@@@>@@X@@@@>@@>@>@@@@@@@@@@@@@@@@@@@@@@g
7@@@@@@?J@@@@@@@@>@@@@@@@>@6KV@@@X@@@@(YW&@@@V@@@@@@@@X@@@X@S@@@@@@V@@V@@@@V@@>@R@@@@@@@@@@@@@@@@@@@@@@@g
?J@@@@@@@T&@@@@@@@>@@@@@@@V@@@@@@@>@@?@@HW&@@@@@@@@@@@@>@@@>@@@Y@@@@@X@@X@@@@X@V@@X@@@@@X@@@@@@@@@@X@@@@@5g
W&@@@@@@V@@@@@@@@V@@@@@@@@X@@@@@@>@@@?@@T&@@@@@@@?@@@@>@@@>@@@@@@@@@V@@V@@@XV@@X@V@@@@@>@@@@@@@@@@>@@@@@@Hg
?W&@@@@@@@T@@X@@@@5?@@@@@@0R@@@@@XV@@@@??S@@@@@@@@@X@@@V@@@V@@@@@@@X@@?@@@@@>@@@V@@X@@(MS@@@@@@@@@?S@@@@@@@?g
W&@@@@@@(R'UV@@@@@Y?@@@@@??:@@@@>@@@@@@L?*@@@@@@@@>@@@@?@@@X@@@@@@>@@@X@@@@V@@@@T@>@@0YW&@@@@@X@@@T&@@@@@@@?g
?O&@@@@@@@U?S@@@@@@@@@@@@@@T2@R4@>@@@@@XI/e@@@@@@@>@@@@@?@@>@@@@@@>@@@>@@@@@?@@@>@R@@e?7@@@@@V@@@V@@@@@@@@@?g
?W2@@@@@@@@>)?&@@@@@@@@@@@@@>@@@eV@@@@@@)X?W&@@@@@@>@@@@@@?@V@@@@@XV@@@V@@@@@@X@@V@@X@@@@?@@@X@@X@@@?@@@@@@@@@?g
W&@@@@@@@@>@@@@@@@@@@@@@@@@V@@@5?@@@@@@@@@)T&@@@@@@V@@@@@@@T@X@@@@>@@@@@?@@@@@V@@@X@V@@@@@?@@V@@V@@@@?@@@@@@@@@?g
7@@@@@@@@>@@@@@@@@@@@@@@@@@X@@(YJ@@@X@@@@@V@@@@@@@5?@@@@@@>@>@@@@>@@@@@@@@@@@@X@@>@@?@@@@@?@@?@@?@@@@@@@@@@@@@@?g
?J@@@@@@@@V@@@@@@@@@X@@@@@@@>@@(YW&@@>@@@@@@X@@@@@@(YJ@@@@@>@R@@@(R@@@@@W@X@@@@V@@>@@@X@@@@@?@@?@@?@@@@@@@@@@@@@@?g
W&@@@@@@@@X@@@@@@@@>@@@X@@@V@@@H?&@@>@@@@@XV@@@@@@@HW&@@@@V@@?@@@U?@@@@@@>@@@@@?@V@@@V@@@@@@X@@?@@?@@@@@@@@@@@@@5?g
?W&@@@@@@X?V@@@@@@@(R'U@V@@@5?3@@=?S@>@@@@@>@@@@@@@@@T&@@@@@?@@T@@>@@@@@@@>@@@@@@@@X@@@X@@@@@V@@@?@@X@@@@@@?@@@@@@H?g
W&@@@@(MS@@@@@@@@@0Y?V@@?@@(Y?S@>@@@>@@@@@>@@@@@@@X?V@@@@@@@@@V'UV@@@@@@@>@@@@@@@@>@@@V@@@@@@?3@@X@V@@@@@@@X@@@@@5h
?W&@@@@@e7@@@@@@@(M?W2@@@@@@U?@@>@@@V@@@@XV@@@@@@XV@@@@@@@@@@@@?S@@@@@@@XV@@@@@@@@V@@@@?@@@@@@LN@V@@?3@@@@@>@@@@@@Hh
W&@@@@@@)X@@@@@@@@Y?O&@@@@@@V/X@V@@@@X@@@V@@@@@@@>@@@@@X@@@@@(M??7@@X@@@>@@@@@@@@@@X@@@@?3@@@@@)X@?@@?V@@@?S@@@@@@5?h
?W&@@@@@@@>@@@X@@@@@@@@@@@@@@@LN@@X@@@>@@@@?@@@@@@>@@@@@>@@@@@@He?@@>@@@V@@@@@@@@X@V@@@@@?V@@@@@@@@?@@@@@@@T&@@@@@(Y?h
?7@@@@@@@>@@XV@@@@@@@@@@@@@@X;)X@V@@@>@@@@@?@@@@@>@@@@XV@@@@@@@W2@@@V@@@@X@@@@@@XV@@?@@@@@@@@@@@@@@@W@X@@@@>@@@@@@@Hhe
J@@@@@@@V'UV@@@@@@@@@@@@@(MS)X@@@?@@>@@@@@@@@@@XV@@@XV@@@@@@@@@@@@@@X@@@V@@@@@XV@@@@@@@@@@@@@@@@@@@@@R@@@@V@@@@@@@5?he
?W&@@@@@@@?S@@@@@@@@@@@@@@@U?7@@@@@@(R@@@@@@@@@@>@@@@>@@@@@X@@@@@@@@@>@@@@X@@@@>@@@@@X@@@@@@@@@@@?@@@@@@@@@@@@@@@@@(Y?he
W&@@@@@@@@T&@@@@@@@@@@@@@@>)X@@X@@@@HJ@@@@@@@@(R@@@?V@@@@?V@@@@@X@@@>@@@@V@@@@V@@@@@V@@@@@@@@@@@@?@@@@@@@@@@@@@@@@@Hhf
?W&@@@@@@@@V@@@@@@@@@@@@@X?V@@@@V@@@@5?7@@@@@@@(YJ@@@@@@@X@@@@@@@>@@@>@@@@@?@@@@?@@@@@X@@@@@@X@@@@@X@@@@@@@@@@@@@@@@5?hf
W&@@@@?@@W@@@@@@@@@@@@@XV@@@@@@@X@@@0YJ@@@@@@@0YW&@@@?eV@@@@@X?V@@@V@@@@@@?@@@@@@@@XV@@@@@@V@@@@@V@@@@@@@@@@@@@@@@(Y?hf
7@@@@@X@@@Y@@@@@@@@@(MS@@@@@@@@V@@@X?W&@@@@@@X?O&@@@@@@@@@@@@>@@@@@@@@@@@@@@@@@@@@@V@@@@@@@@X@@@@@X@@@@@@@@@@@@@@@@Hhg
?J@@@@@V@@@@@@@@@@@@@(YW&@@@@@@@@X@(R1?7@@@@@(R@@@@@@@@@@@@@@@>@@@@@@@@@@@@@@@@@@@@@@?@@@@@@@>@@@@@V@@@@@@?@@@@@@@@@5?hg
@6X?hf??
?7@@@@@?3@@??@@@@@@@@HW&@@@@@@@@V@@Y?@?@@@@@@H?@@@@@@@@X@@@@@V@@@@@@@@@X@@@@@@@@@@@@@?@@@@@?S@@@@@@X@@@@@@?@@@@@@@@@H?hg ?J(R1?hf?
?3@@@@@??@@@@@@@@@@@@?7@@@@@@@@@?@@@@@?@@@@@5?J@@@@@@@>@@@@@@?@@@@@@@@>@@@@@@X@@@@@@@X@@@@@T&@@@@@V@@@@@@@@@@@@@@@@5
W.YJ@?hf?
?V4@0Me7@@@@@@@@@@@@?@@@@@@@@@@@@@@@@?@@@@@HW&@@@@@@>@@@@@@@@@@@@@@@>@@@@@@V@@@@@T@>@@@@@>@@@X@@@X@@@@@@@@@@@@@@@@H
7H?7@T&?2@@?g?
@@@@@@@@@@@@@?@@@@@@@@@@@@@@@@X@@@@@W&@@@@@@>@@@@@@@@@@@@@@@>@@@@@@@X@@@@>@R@@@@@V@@@V@@@V@@@@@@@@@@@@@@@@5?
@??(R4@@@?@?g?
@@@@@@@X@@@@@@@@@@@@@@@@@@@@@>@@@@@@@@@@@@@>@@@@@@@@X@@@@@@>@@@@@@@>@@@@V@@?@@@@@@@@@X@@@?@@@@@@@?@@@@@@@(Y?
?J@?f?W@@@?g?
@@@@@@>@@@@@@??@@@@@X@@@@@@@>@@@@@@@@@@@@@>@@@@@@@@>@@@@@XV@@@@@@@>@@@@@@@@?@@@@@@@@V@@@@?@@@@@@@?@@@@@@@H
?75?fO&U?h?
@@@@@>@@@@@@@?J@@@@>@@@@@@@>@@@@@@@@@@@@@V@@@@@@@@>@@@@@>@@@@@@@@V@@@@@@@@@X@@@@@@@@X@@@@?@@@@@@@X@@@@@@@?
J@H?eW2@>@@@@@?f?
@@eV@@@@@@@@W&@@@V@@@@@@XV@@@@@@@X@@@@@@?@@@@@@@>@@@@@>@@@@@@@@@X@@@@?W@@V@@@@@@X?V@@@@@@@@@@@@>@@@@@@@5?
?W&@e?W&Y@(M??@5?f?
3@@@@@@@@@@@@@@@@@X@@@@@>@@@@@@@@>@@@@@@@@@@@@@@>@@@X?V@@@@@@@@@V@@@@@W&@@?@@@@@>@@@@@@@@@@@@@@>@@@@@@@(Y?
?
?7<?e?&@@@UO2@0Y?f?
N@@@@@@@@@@@@@@@@>@@@@XV@@@@@@@@V@@@@@@@@@@@@@@>@@?V@@@@@@@@@@@@@@@@@@@@@@?@@@@>@@@@@@@@@@@@@@>@@@@@@@@H
?
?@hB@@@h?
?@@@@@@@@@@@@@@(R@@@@>@@@@@@@@@@X@@@@?W@@@@@@XV@@@@@@@@@@@@@X@@X@@@@@@@@@@?@@XV@@@@@@@@@@@@@@V@@@@@@@@5?
?
J5hJ@V4@@@@f?
?3@@@@@@@@@@@@@HJ@@@V@@@@@@@@@@>@@@@@T&U@@@@>@@@@@@@@@@@@@@>@@>@@@@@@@@@@@@@>@@@@@@@@@@@@@@@@X@@@@@@@(Y?
?O2@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@?g?
?W.Yg?W&@e?W@@f?
?V4@0M?I@Me@@@W&@@@X@@@@@@@@@>@@@@@>@R@@X@V4@@@@@@@@@@@@@V@@>@@@@@@@@@@@@?V@@@@@@@@@@@@@@@@>@@@@@@@(Y
?W2@0M
?
W.Y?g?7@@@@@0M?f?
@@@@@@@>@@@@@@@(MS@@@@@V@@T@>@5?e@@@@@@@@@@@@T@>@@@@@@@@@@@@@@@@@@@@@X@@@@@@@@>@@@@@@@(Y?
?7(M
?
?W&Hh?@@Xhe?
@@@@@@V@@@@@@@@HW&@@@@@X@V'>@@HW2@@@@@@@@@@@@>@R@@@@@@@@@@@@@@@@@@@@@>@@@X@@@@V@@@@@@@(Y
?@H?
?
?75?heB@@?h?
@@@@@@?@@@@@@@@?7Y@@@@>@@LN@@@T&@@@@@X@@@@@@V@5?@@@@@@@@@@@@@@@@@@@XV@@(R@@@@@?@@@@@@(Y?
?@
?
?'6X
J(Y?f?W2@6K?@5?h?
@@@@@@X@@@@@@@@?@@@@@>@@V/X@(R@@@@@@V@@@@@@@@@Y?@@@@@@@X?@@@@@@@@@V@@@@U?@@@@@@@@@@@(Y
?@
?
?V')K?
?W&HgW&(?4@@0Y?h?
@@@@@>@@@@@@@@@X@@@@V@@@?S@(YJ@@@@@@?@@@@@@@@@@@@@@@@XV@@@@@@@@@@@X@@@V@@@@@@@@@@@@(Y?
?3L?
?
V4@6X?
?75?g7(Y?hg?
3@@@V@@@@@@@@@V@@@@@?@@@?&@U?7@@@@@@X@@@@@@@@@@@@@@0R@@@@@@@@@@@@>@(?'X@@@@@@@@@@@(Y
?N1?
?
I4)K
?@H?f?J(Y
N@@@?@@@@@@@@@X@@@@@@@@5eV1?@@@@@@V@@@@@@@?@@@@@@X??@@@@@@@@@@@>@@U?V@@@@@@@@@@@0Y?
@?
?
I'6XhfJ5gW&H?
??
?@@@?@@@@@@@@>@@@@@@@@(YW2@@?@@@@@@X@@@@@@@X@@@@@V@@@@@@@@@@@@@>@@>@@@@@@@@@@@@(M?
@?
?
?V4)X?he7Hg75
?@@@?@@@@@@@V@@@@@@@@@HW&@@@?@@@@@>@@@@@@@>@@@@@@@@@@@@@@@@@@@V@(R@@@@@@?@@@@@(Y
@L
?
?I/Kh?J@?f?J@H
??
?3@@@@0Y@@@@?@@@@@@@@@T&@@@@?@@@@V@@@@@@@>@@@@@@@@@@@@@@@X@@@@?(Y?@@@@@@X@@@@0Y?
@1
?
V'6XgW&5?fW&5?
??
?V4@0Me@@@@?@@@@@@@?S@@@@@@W@@@@X@@@@@XS@@@X@@@@@@@@@@@>@@@@@L??J@@@@?S@@@(M?
@@
?O2@@@6K
?
?V')X?f7(Y?f7(Y?
@@@@?@@@@@@@W&@@@@@@@Y@@>@@@@@>@@U?V@@@@@@@@@@@V@@@@@@)KO&@@@@T&@@(Y
@5hf?W2@@@@@@@0M?I4@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@?e?
?@K?
V')Ke?C(Yg@H
?
@@@@@@@@@@@@@@@@@@?W@?3>@@@@@>@UV@@@@@@@@@@@@@@X@@@@@@>@@@@@@>@@@(Y?
@Hhf?7(M
?
?@@@6K
?V4@6?2(Y?f?J5?
3@@@@@@@@?@@@@@@@@T&@LN@@@@@S@>@@@@@@@@@@@@X@(R@@@@@?S@@@@@@>@@@(Y
@?hfJ@H?
?
?I4@6K
?I4@0Yg?7H?
??
V4@@@@@@5?@@@@@@@V@@@)X@@@@@@>@@@@X@@@@@@@>@(YJ@@@@@T&@@@@@>@@@(Y?
@?hf@@
?
I'6X
J@
?I'@@(Y?3@@@@@@X@@@V@@@@@@V@@@@V@@@@@@@V@@HW&@@@@>@@@@@@>@@@@H
@?hfN@
?
?V4)K?
75
??
V40YeV'@@@@V@@@@?@@X@@@X@@@@X@@@@@X@@@5?7@@X?V@@@@@XV@@@@5?
@?hf?@
?
?I46K?
?J@H
??
?N@@@@?@@@5??V@@?S@@@@>@@@@@>@@@@Y?@@>@@@@@@@>@@@@@(Y?
@?hf?@
?
?I46K?
W&5?
3@@@@@@@H?@@@@?7@@@>@@@@@V@@@@@@@@>@@@@@@@V@@@@@(Y
@?hf?@
?
?I4@@6K?
?W&(Y?
?
N@@@@@@5?J@@@@?@@@V@@@@@@@@@@@@@@V@@@@@@@@X@@@@(Y?
@?hf?@
?
I4@6K?hfW&(Y
?
?3@@@@@UO&@@@@?@@@X@@@@@@@@@@@@@@X@@@@@@@>@@@@@H
@?hf?@
?
?I'6X?he7(Y?
??
?
?V4@@?V@@@@@@@@@@V@@@@@X?@@@@@@@>@@@@@@@V@@@@@5?
@?hf?@
?
V4)Kh?C(Y
??
?W2@
??
?I'@@@@@@@@@@@@X@@@@>@@@@@@@@V@@@@@@X@?@@@@(Y?
@?hf?@
?
I4@6K?e?W2(Y?
?7@Hhe?@
N@@@@@@@@@@?S@@@@V@@@@@@@@@?@@@@@>@@@@@@@H
@?hf?@L?
?
?I4@@6?&(Y
?@5?he?@
?3@@@@@@@@@?7@@@@?3@@@@@@@@@@@@@V@@@@@@@5?
@?hfJ@,?
?
I4@0Y?
??
?@hf?@
??
?N@@@@@@@X@?@@@@@??@@@@@@@@X@@@@X@@@@@@(Y?
@?hf7(Y?
?
?
?@1?he?@
@@@@@@>@@?@@@?e7@@@@@@@>@@@@>@@@@@@(Y??
@?hf@?
? @@@@@>@@@@@@@?/X@@@@@@@>@@@@>@@@@@@(Y?
?@@?he?@
??
@?hf@1
?
?@@?he?@
@@@@V@@@@@@@@?N@@@@@@@V@@@@V@@@@@@@H ??
@?hf@@
? @@@@X@@@@@@@@?J@@@@@@5?3@@@?@@@@@@5?
?@@?he?@
??
@?hf@@
?
?@@?heJ@
@@@>@@@(Y@@@@T&@@@@@@H?V@@@?@@@@@(Y? ??
??
@?hf@@
?
J@5?he7@
@@V@@@@H?@@@>@@@@@@@@W2@@@@@@@@@(Y
@?gW.f@@
7(Y?he@5
? 3@@@@@5??@@V@@@@@@@@@@@@@@@@@@@(Y?
?
@Hhf@H
??
@?g7Hf@5
? V'@@@@H??3@@@@@@@@@@@@@@@@@@@@(Y
?J@?hf@?
@?g@?f@H
?
?7@?hf@?
?
@Lf?J@?f@?
? ?V'@@5e?N@@@@@@@@@@@@@@@@@@@(Y?
V40Yf3@@@@@@@@@@@@@@@@@(Y
?@5?he?J@?
?
31fO&5?)Xe@?
?
V4@0?'@@@@@@@@@@@(Y?
?@H?he?75?
?
N@e'@@(Y?@)?J@?
?
?V'@@@@@@@@@0Y
?@hf?@H?
?
?@eV40Y?J@H?75?
?
V4@@@@@@(M
?@L?he?@
?
?@gW&@=C(Y?
?
I4@0Y?
J@1?he?@L?
?
?@f?O&@>@(Y
?
7@@?he?@@?
?
?@?@@@@@@>@(Y?
?
@?@?he?@H?
?
?3L?fB@(Y
?
@?@?he?@
?
?S1?fJ@H?
?
3@@?he?@
?
?&@=e?O&5
?
N@@?he?@
?
I4@@@@0Y
?
?@@?heJ@
?
?
?@@?he7@
?
?
?@@?he@@
?
?
?@@?h'@@5
?
J@@?hV'@H
?
7>5?h?N@?
@@H?he@?
??
@@hf@?
?
@@hf@?
?
@@hf@?
?
@@hf@?
?
@@hf@?
?
@@hf@?
?
@@hf@?
?
@@hf@?
??
@@he?J@?
@5he?75?
?
?J@Hhe?@H?
?75?he?@
??
?@H?he?@
?@hf?@
??
?@hfJ5
?
?@hf7H
?
?@hf@?
?
?@hf@?
@?
??
@?
?
@L
?
?@hf3)X?
?
J5hfV4)KhfO2@@@@@@6Kf?
7U
I4@@@@6T2@@@@@@0M?f@@@@e?
31
?I40M?hf@XI'L??
?@
?W2(?J@)XN1??
75
W&@U?.R'1?@??
@?
.MB)X??N@?@??
@1
?31?@?@?@??
@@L?
?N@?@?@W5??
?B)K
@?e3@H??
3@(?
@LeN@e?
V+Y?@@6K
@@eJ5e?
I4@@@@@@@6T2@@@?@Khf?O.Ye?
?I4@@@@?@@@@@@@@@@@@@@@0Y?e?
?
?
?
?
?
Figure 5. Load Balancing.
extensively investigated. For distributedmemory machines, project members believe
that the efficient distribution of work for
large applications is crucially linked to the
efficient implementation of multicast communication (Byrd, Nakano, and Delagi 1987;
Byrd, Saraiya, and Delagi 1988). Although the
principal emphasis has been on the development of distributed-memory hardware, the
fact that the CARE simulator can also simulate shared-memory machines has allowed
project members to investigate the relative
performance of these two distinct classes of
machines and the relative performance and
appropriateness of shared-variable and message-passing, object-oriented programming
models (Byrd and Delagi 1988). Current work
focuses on the design of hardware that might
provide efficient support for both the sharedvariable and the message-passing programming models (Byrd 1989).
Load Balancing
Project members have started examining
load-balancing problems (figure 5) within the
context of the AAP vertical slice (Hailperin
1988). In particular, this work focuses on a
load-balancing method intended to migrate
Lamina objects in large (thousands of processing elements) CARE multicomputers to
improve the performance of soft–real-time
32
AI MAGAZINE
signal-interpretation systems such as Elint and
AirTrac (see Applications).
Without load balanc8T
ing, only a lightly loaded
multicomputer
that
dynamically creates processes can, in general,
achieve real-time performance. The work focuses
on how to achieve global
load balancing, which
would be an attractive
solution to this problem
because it would allow
the effective use of massively parallel ensemble
architectures for larger
soft–real-time problems.
The challenge is to
replace quick global communication, which is
impractical in a massively parallel system, with
statistical techniques. In
this vein, a novel
approach to decentralized load balancing is
being investigated based
on statistical time-series analysis. Each processing element estimates the systemwide
average load using information about past
loads of individual sites and attempts to
equal this average. This estimation process is
practical because the soft–real-time systems
in which the project is interested naturally
exhibit loads that are periodic (in a statistical
sense akin to seasonality in econometrics).
A load-balancing system for Lamina/CARE
was designed using this load-characterization
technique. It has been implemented, and
experiments with it in the context of ELINT
and AIRTRAC have begun.
Concurrent and
High-Performance Lisp
To understand the behavior of Lisp on
shared-memory machines, work was done on
the QLisp system (Okuno and Gupta 1987).
Although this work was not directly used by
other parts of the project, it involved examining some of the constraints on parallelizing
production systems by studying the OPS5
language.
In the search for higher-performance symbolic computation, work was also done on
the development of high-performance Lisp
interpreters (Okuno, Osato, and Takeuchi
1987). This work was also not directly used in
Articles
the project because all the code
used in the project’s experiments
has been compiled.
Problem-Solving
Frameworks
Blackboard
Knowledge Base
KS
Node
@??@
3L?@
V/T@L?
?V'>1?
N@@L
?3>1
?V'@
N@L?
?31?
?S@L
W&R/X?
?W&@?V/X
?7@@eV/X?
J@@@e?V/X
7@@@L?eV/X?
@@@@1?e?V/K
@@@@@LfV46Khg
@@@@@1gI'6Khf
@@@@@@g?V4@6Khe
@@@@@@he@@@@@@@@@?e
@@@@@@g?W2@0Mhe
@@@@@5gO&0Mhf
3@@@@HfW20Mhg
N@@@5?e?O.M
?3@(Y??W20Y?
?V'UeW.M?
V/X?7H
?V/T5?
S(Y?
?W&H
?75?
?@H?
?@
Rule
?W26K?hf
?*US@6K?he
?V4@UI'@he
?B1?V'L?h
3L?N1?h
V1e@?h
W2@@@@@@@@@@@@e@@@@@@@@@@
*U
V4@@@@@@@@
@Y
@@@@@?hf
@?
@@@@@?hf
@?
@@@@hg
3XfO2@@@@@@@@@@
V4@@@@0Mhe
Update event
Rule
One of the key layers in the AAP
Node
Node
strategy was problem-solving
KS
frameworks. Faced with a large
number of different problemRule
solving models, the project comNode
Node
Node
mitted itself early to the
blackboard problem-solving
Rule
model (Engelmore and Morgan
1988). This choice was not
Scheduler
entirely arbitrary. The blackboard
Events
metaphor was successfully
applied in the area of real-time
signal processing (Nii et al.
1982), the selected problem
domain for AAP. Also, it was
anticipated that the blackboard
metaphor would help project
members extract parallelism
Figure 6. The Cage Architecture.
from the application in the way
Update events are perceived by the scheduling component and collected in a global event queue. The
that the problems were formulatscheduler selects the knowledge sources that are interested in any given event and can execute them in
ed because the metaphor has a
parallel. In turn, these knowledge sources inspect the blackboard and perform updates that are seen
model of asynchrony built into
by the scheduler.
it. For reasons detailed in Rice
(1988c), the blackboard model
turned out to not be as parallel as
was hoped, but project members
blackboard model, by its nature, provides a
still know of no better one for
certain amount of parallelism. Therefore, it
concurrent execution.
should be possible to exploit this parallelism
The development of problem-solving
without any major redesign or rethinking of
frameworks took two distinct courses. First
the problem-solving model. Cage is, therewas the development of a fairly conservative
fore, an implementation, that is designed to
concurrent implementation of an existing
allow the concurrent execution of a blackblackboard system to run on existing sharedboard system through the concurrent execumemory machines. This system is described
tion of the knowledge sources and rules in
in the next subection. Second was the need to
the application (figure 6).
rethink the blackboard metaphor from
At the outset, it was not known how diffiscratch in the hope of achieving really high
cult it would be to program such a system
performance on distributed-memory multiand how much performance could be expectprocessors, such as the CARE machine. The
ed. However, project members thought that
result was the Poligon system. Three generasuch an architecture might well be suitable
tions of papers were produced describing profor the current generation of multiprocessors,
ject strategy, the Cage and Poligon systems as
which mostly have a shared-memory design.
they evolved, and the experimental results
Blackboard systems are typically implemented
produced by these systems (Nii 1986; Nii,
using a central, shared database to represent
Aiello, and Rice 1988a, 1988b).
the blackboard. The match between the
shared blackboard and the shared-memory
Cage
resource seemed to be worth investigating.
The Cage system was implemented first on a
Cage (Concurrent AGE) (Aiello 1986) is a
simple emulator, which emulated the funcreimplementation of the AGE (Nii and Aiello
tionality of a QLisp implementation without
1979) blackboard system developed by the
paying the costs of detailed simulation. It was
Heuristic Programming Project at Stanford.
later ported to run on the CARE simulator
The central idea behind Cage is that the
?C
?W2@@@@@
W.M?e?I
7Hg
@?g
@?g
@?g
@?g
3Lg
V/K?f
?V4@@@@@
?W26K?hf
?*US@6K?he
?V4@UI'@he
?B1?V'L?h
3L?N1?h
V1e@?h
W2@@@@@@@@@@@@e@@@@@@@@@@
*U
V4@@@@@@@@
@Y
@@@@@?hf
@?
@@@@@?hf
@?
@@@@hg
3XfO2@@@@@@@@@@
V4@@@@0Mhe
@?g?@hg
@?g?@hg
@?g?@hg
@?g?@hg
@?g?@hg
@?g?@hg
@?g?@hg
@?g?@hg
@?gC@@@@@@@@@@@f
@?W2@@@@@Ug?W@@L?e
@W&@X?@@@)X?f?7@@1?e
@@(R@@@X?@1?f?@@@@?e
@@Y?@@@1?3@?f?@@@@?e
@@@@@@@@?N@?f?@@@@?e
@??I'@@@?J@?f?@@@5?e
@?eN@e?75?f?3@@H?e
@?e?@e?@Y?f?V@5f
@@@@@@@@@@@@@@@@@@0Yf
@?e@?hg
@?e@?hg
?J5?e@Lhg
?*U?e3)X?hf
?N@6X?N@1?hf
@@1??@@Lhf
3@@?eB1hf
V'@Le?3L?he
?V'1e?V/Xhe
N@L?eV/T-X?g
?3)Xe?S(R/Xg
?V')X?W.Y?V/X?f
V')T.Ye?V/Xf
?N@(Y?fV/X?e
?J(Yg?S)Xe
?*U?gW&@)X?
?V/?g&@@@)?
@??@
3L?@
V/T@L?
?V'>1?
N@@L
?3>1
?V'@
N@L?
?31?
?S@L
W&R/X?
?W&@?V/X
?7@@eV/X?
J@@@e?V/X
7@@@L?eV/X?
@@@@1?e?V/K
@@@@@LfV46Khg
@@@@@1gI'6Khf
@@@@@@g?V4@6Khe
@@@@@@he@@@@@@@@@?e
@@@@@@g?W2@0Mhe
@@@@@5gO&0Mhf
3@@@@HfW20Mhg
N@@@5?e?O.M
?3@(Y??W20Y?
?V'UeW.M?
V/X?7H
?V/T5?
S(Y?
?W&H
?75?
?@H?
?@
WINTER 1989
33
Articles
Rules
Node
?
?
?
?@e@?
?
?3L?@?
?V/T@L
??
V'>1
?
?N@@L?
?
3>1?
?
V'@?
?
?N@L
?
31
?
S@L?
?
?W&R/X
?
W&@?V/X?
7@@??V/X
??
?J@@@?eV/X?
?
?7@@@Le?V/X
?
?@@@@1fV/K?hg?
?@@@@@L?e?V46K?hf?
?@@@@@1?f?I'6K?he?
?@@@@@@?gV4@6K?h?
?@@@@@@?h?@@@@@@@@@e?
?@@@@@@?gW2@0M?h?
?@@@@@5?f?O&0M?he?
?3@@@@H?e?W20M?hf?
?N@@@5fO.M?hg?
3@(YeW20Y
?
V'U??W.M
?
?V/X?7H?
?
V/T5
?
?S(Y
?
W&H?
?
75
?
@H
@?
??
?
?
Pipe
Update
Poligon
Rule
Invocation
Rules
Node
Rules
Node
??
??
?@e@?
?3L?@?
?V/T@L
???
V'>1
?N@@L?
??
3>1?
V'@?
??
?N@L
31
??
S@L?
?W&R/X
??
W&@?V/X?
7@@??V/X
?J@@@?eV/X?
???
?7@@@Le?V/X
?@@@@1fV/K?hg?
?@@@@@L?e?V46K?hf?
?@@@@@1?f?I'6K?he?
?@@@@@@?gV4@6K?h?
?@@@@@@?h?@@@@@@@@@e?
?@@@@@@?gW2@0M?h?
?@@@@@5?f?O&0M?he?
?3@@@@H?e?W20M?hf?
?N@@@5fO.M?hg?
3@(YeW20Y
?
V'U??W.M
?V/X?7H?
???
V/T5
?S(Y
?
W&H?
75
??
@H
@?
???
?
??
??
?@e@?
?3L?@?
?V/T@L
???
V'>1
?N@@L?
3>1?
???
V'@?
?N@L
??
31
S@L?
??
?W&R/X
W&@?V/X?
??
7@@??V/X
?J@@@?eV/X?
??
?7@@@Le?V/X
?@@@@1fV/K?hg?
?@@@@@L?e?V46K?hf?
?@@@@@1?f?I'6K?he?
?@@@@@@?gV4@6K?h?
?@@@@@@?h?@@@@@@@@@e?
?@@@@@@?gW2@0M?h?
?@@@@@5?f?O&0M?he?
?3@@@@H?e?W20M?hf?
?N@@@5fO.M?hg?
3@(YeW20Y
?
V'U??W.M
?V/X?7H?
???
V/T5
?S(Y
?
W&H?
75
??
@H
@?
???
?
Update
Update
Rule
Invocation
Figure 7. The Poligon Architecture.
Updates on the blackboard are observed by rules that watch specific slots of blackboard nodes. These rules can fire in parallel, causing further updates to the same or
other nodes. This flow of updates from one node to another implicitly forms pipes,
which increase the parallelism realizable by the system.
using an implementation of QLisp built on
top of the Lamina shared-variable programming interface (Saraiya 1988).
The Elint application, discussed later, was
mounted on the Cage system, and experiments
were performed on it. These experiments are
detailed in Aiello (1988) and Rice and Aiello
(1989). The Cage system showed that blackboard programs can, indeed, be run in parallel
in a relatively simple manner. The performance
of Cage, however, is restricted by a number of
factors (Nii, Aiello, and Rice 1988b): (1) its
implementation, which was not highly tuned;
(2) its architecture, which exhibits significant
contention for global shared resources such
as the event queue; (3) the QLisp substrate on
which it is built; and (4) the shared-memory
hardware on which it runs.
Thus, the Cage architecture is viable for
existing shared-memory hardware systems,
but because of the close link between the
Cage programming model and its underlying
hardware, project members do not anticipate
that future concurrent expert system tools
will be built like Cage. Instead, they believe
34
AI MAGAZINE
that the trend of multiprocessor design is
moving away from shared-memory machines
toward distributed-memory designs because
of their greater ability to scale. Software
design is likely to track this trend.
The expectation is that for reasons of simplicity, performance and cost, the next generation
of multiprocessors is likely to be distributedmemory machines. This expectation required
rethinking the blackboard model so that it
could be mounted on such machines in a
manner likely to deliver good performance.
Poligon (Rice 1986a, 1986b, 1989) was developed in an attempt to address these needs. It
took the view that processors were going to
be cheap and plentiful; thus, if necessary, it
was quite acceptable to allocate one processor
or more to each node on the blackboard.
First, the serializing, centralized control
mechanism of conventional blackboard
systems was discarded. Distributing the nodes
of the blackboard over the processor network
allowed the knowledge base to be spread over
the blackboard as well; this arrangement
eliminated any performance bottleneck as a
result of the costs of communication between
the knowledge base and the blackboard. The
simplest available rule-invocation mechanism
was selected to maximize performance; rules
were directly attached to slots of the nodes
on the blackboard. Modifying a slot resulted
in invoking the rule attached to it. Rule invocations were spun off into different processes
on different processors for execution; thus,
the length of the critical sections on the processors holding blackboard nodes was minimized, and multiple, simultaneous rule
invocations for the same modified blackboard
object were allowed (figure 7).
In practice, these mechanisms did, indeed,
result in good performance, but they also
resulted in significant problems. Lots of
uncontrolled asynchronous processes all
reading and writing in a shared database are
bound to cause problems when it comes to
getting a coherent or correct answer. Extra
mechanisms had to be implemented, which
allowed the blackboard nodes to have goals
and the ability to evaluate their own performance with respect to the overall system
goal. These mechanisms allowed the blackboard nodes to have the final decision about
whether to perform any modification operation attempted by a rule. The result was a sort
of distributed hill-climbing behavior. Nodes
iterated toward a good solution.
These mechanisms did not come without
Articles
associated costs in terms of granularity.
Although the Poligon system delivers high
performance when compared to other blackboard systems such as AGE, it significantly
lacks the performance provided by an application written directly in Lamina. Therefore,
Poligon provides a fairly general concurrent
implementation of the blackboard problemsolving model with all the accompanying
advantages of abstraction and modularity. It
does so, however, at a price. A detailed
description of Poligon’s design and implementation can be found in Rice (1989), which
also describes how Poligon’s performance
could be improved by superior compilation if
it were to be turned into a production-quality
system.
The Elint application, described in the next
section, was implemented in the Cage, Poligon,
and Lamina systems. The results of these
experiments are reported in Rice (1988b); Rice
and Aiello (1989); Nii, Aiello, and Rice
(1988b); and Nii and Rice (1989). Another
application called ParAble, (Bandini 1989)
implemented using the Poligon framework, is
also described in the next section.
Blackboard
nodes
Clusters
Emitters
??
O2@@@@@@@6K?hg?
?O2@@@@@@Xe?I4@6Khf?
?W2@@@@@@@@1gI46Xhe?
O&@@@@@@@@@@e?@fI/K?h?
W2@@@@@@@@h?)X??V'6X?g?
?W&@@@@@@W@@@@@@f?@)?eV')Xg?
?7@@@@@@@@Y@@@@@he?V'1g?
J@@@@@@@@@@@@??@hfV'L?f?
?W&@@@@@@@@?@@@@@@@@he?V/Xf?
?7@@@@@@@@@@@@?@@@g?/X?fN1f?
J@@@@@@@@@@@@@@@@@g?N1?f?3L?e?
7@@@@@@@@@@@@?@@@5e@?f3Lf?N1?e?
@@@@@@@@@@@@@@@?@?f@?eV/e?)X?@?e?
?J@@@@@@@@@@@@@@@?@1f3Lg?31?3Le?
?7@@@@@@@@@@@@@@@@@5fV/g?N@?N1e?
?@@@@@@@@@@@@@@@@@@?
@??@e?
?@@@@@@@@@@@@@@@@@@1
?@e?
?@@@@@@@@@@@@@@@@@@@
?@e?
?@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@e?
?@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@e?
?@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@e?
?@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@e?
?3@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@5e?
?N@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@He?
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@?e?
3@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@5?e?
N@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@H?e?
?3@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@5f?
?V'@@@@@@@@@@@@@@@@@@@@@@@@@@@@@(Yf?
N@@@@@@@@@@@@@@@@@@@@@@@@@@@@@H?f?
?3@@@@@@@@@@@@@@@@@@@@@@@@@@@5g?
?V'@@@@@@@@@@@@@@@@@@@@@@@@@(Yg?
V4@@@@@@@@@@@@@@@@@@@@@@@0Y?g?
I'@@@@@@@@@@@@@@@@@@@(M?h?
?V4@@@@@@@@@@@@@@@@@0Yhe?
?I4@@@@@@@@@@@@@0Mhf?
I4@@@@@@@0M?hg??
?
Observations
Input Data
Figure 8. The Elint Application.
Sensor data are abstracted into hypothetical radar emitters, which are tracked as clusters of emitters.
Applications
As mentioned in the introduction, the project
members expected at the outset that AAP
would be application driven. In the search for
an application domain, which would need
significant speedup for expert systems to be
fielded and yet held a certain obvious potential for concurrent execution, they picked the
field of real-time signal understanding. Existing blackboard systems, such as HASP/SIAP
(Nii et al. 1982) and Tricero (Williams, Brown,
and Barnes 1984) showed that the blackboard
problem-solving model was appropriate for
this domain and that the performance deliverable using the existing blackboard tools was
entirely inadequate to field such systems.
What was needed, therefore, was a problem
complex enough to give us a reasonable
model of a real system and yet simple enough
to prevent project members from expending
too much effort on the mechanics of its
implementation. The unavailability of a satisfactory application at the start of the project
caused it to become somewhat more hardware driven than originally expected. It was
eventually decided that project members
would focus on a problem called Elint, a
system for understanding passive radar signals.
This application is described in the next section.
After a fair bit of experimentation, it was
determined that the ability to exploit parallelism was being constrained by the problem
being used; it was not sufficiently complex. In
the search for a more knowledge-rich and
computationally intensive application, project members developed AirTrac (described
later). Work was also done in areas other than
real-time signal understanding; ParAble
(described later), a system for fault finding in
particle accelerator beamlines, was developed
using the Poligon framework. A number of
numeric or seminumeric programs were also
developed during the hardware-related
experiments.
Elint
Elint is a soft–real-time system for interpreting passive radar signals. Data are collected
from a number of receiving stations and integrated to allow the system to track radar-emitting aircraft as they pass through the
monitored airspace. The data are abstracted
into hypothetical radar-emitting platforms. In
turn, these platforms are collected into clusters of emitters, which might represent a
number of planes or a single plane using multiple radar systems, as is often the case with
modern military aircraft (figure 8). Elint was
first implemented using the CAOS system. It
was originally thought that this work would
WINTER 1989
35
Articles
. . . the project members expected at the outset that
AAP (Advanced Architectures Project) would be
application driven.
take only a couple of months. However, the
complete task—implementation, experimentation, and analysis of results—took 18
months. Project members learned early that it
is by no means a trivial matter to reimplement an existing serial application in a parallel environment. These initial experiments
are detailed in Brown, Schoen, and Delagi
(1986).
Since the CAOS implementation, Elint has
been implemented three times, using Lamina
(Delagi and Saraiya 1988; Saraiya 1989) and
the Cage (Aiello 1988) and Poligon (Rice
1988b; Nii, Aiello, and Rice 1988b) frameworks, and a number of experiments have
been performed.
AirTrac
The development of the Elint application
showed that the amount of parallelism that
could be demonstrated was much more
dependent on the application than had been
anticipated. Project members had hoped that
by extracting parallelism at the different levels
of the system’s implementation hierarchy,
they could gain significant speedup. However, they were unable to demonstrate speedup
in this way. They were able to demonstrate
that their experiments showed poor speedup
largely because the application itself had run
out of parallelism.
What project members needed was an
application that would really stretch the
hardware and software they were developing
in a realistic manner. As a result, the AirTrac
application was developed (Delaney 1986).
The AirTrac problem domain sounds superficially like that of Elint. It was a system for
interpreting radar data, although in this case,
the radar systems being modeled were active
not passive. Unlike Elint, AirTrac was
designed to go much further than simply
tracking aircraft and finding likely threats.
The scenario for AirTrac was the detection
of smugglers flying across a border. The problem faced by existing radar users is that a
large number of legitimate aircraft travel in
the same airspace as smugglers. Smugglers
can take advantage of variations in terrain to
find areas of poor or no radar reception. They
also resort to other evasive tactics.
36
AI MAGAZINE
The system was designed in a number of
layers so that different implementation
efforts could be decoupled. The first subsystem implemented was called the data-association component (Nakano and Minami 1987)
and is the subsystem that most closely
matches the Elint application. Initially, this
component was to be implemented using the
Poligon framework. However, it was found
that simulating the Poligon system for a
problem as complex as AirTrac would take
prohibitively long. Consequently, AirTrac was
directly implemented in Lamina. Substantial
speedup was shown, which seemed to
increase linearly with the number of processors. This result was encouraging.
The second component of AirTrac, Path
Association (Noble and Rogers 1988), was significantly more knowledge intensive than the
first. This subsystem was also initially implemented directly in Lamina. However, programming in the raw Lamina framework was
too complex and time consuming, so a layer
called ELMA was built on top of Lamina that
provided the abstraction model needed for
the implementation (Noble 1988b).
The final, most abstract component of AirTrac has not been implemented. Project
members have not yet learned all they can
from the second layer and were not able to
show all the speedup that they thought was
possible in this layer, so work is continuing in
this area.
ParAble
The ParAble project (Bandini 1989) was an
attempt to test the generality of the problemsolving model offered by Poligon by choosing
a completely different application domain. To
achieve this goal, a parallel implementation
of the ABLE system (Selig 1987), which was
also developed at Stanford, was made.
The objective of the ABLE project was to
find a fast way to diagnose difficulties with
particle accelerator beam lines. These large
and complex machines are prone to beam
alignment problems because (1) the magnets
that steer and focus the beam are misaligned
or (2) the power supplies to these magnets are
incorrect, resulting in the magnets not
having the desired strength. These systems
Articles
are so complex that it can take many months
of knob twiddling simply to commission
them.
The ABLE system used an analytic model of
the beam-line–component transfer functions
and a number of heuristics that employ successive model runs and compare the results
with real data to locate the faults. It was thus
able to find faults in particle accelerator systems
in about 10 minutes. As these systems become
more complex, there might well be a need to
control them in real time. Thus, although no
immediate need exists for higher performance
in the ABLE system, it is reasonable to suppose the need might exist in the future.
A number of experiments were performed
on ParAble and are detailed in Bandini
(1989). Again, the realizable parallelism in
this project was found to be limited mostly by
the availability of data parallelism.
Numeric and Seminumeric Programs
The expert systems mentioned here are not
ideal applications for multiprocessor execution. They are irregular and data dependent.
A large body of applications already exists in
the area of numeric and seminumeric processing that will require the speedup associated
with parallel execution. Indeed, such programs are already being run on a number of
multiprocessors. Therefore, it is essential that
any machine designed to be general purpose
must also be able to execute these regular,
argorithmic problems efficiently. A number of
small numeric programs have been developed
that allow project members to test their hardware and software ideas in a much more controllable way than is possible with any expert
system application. Among these are a Gaussian elimination algorithm, a partial differential equation solver, and an integrated circuit
line simulator.
Conclusions
AAP has been running for a number of years
now. The project members have found that
they have moved from having a good understanding of AI problem-solving techniques
and knowing little about parallel computation
to having have a good understanding of all
the issues involved; the problems facing the
implementers of concurrent problem-solving
systems, and the gains they can reasonably
expect. The project members hope that the
successes and failures reported in the publications cited in the bibliography will help the
rest of the research community in the move
to concurrent computational platforms.
Notes
1. The project’s real name is Expert Systems on
Multiprocessor Architectures, but all the project
members have always felt more comfortable calling
it either the Advanced Architectures or the Architectures project for short. This name more effectively captures the fact that new architectures for both
hardware and software will be needed to meet the
project’s goals.
2. Connection Machine is a registered trademark of
Thinking Machines Corp.
3. The expansion for this acronym seems to have
been lost somewhere in the wash. I think that it
has something to do with the words concurrent
and array.
Bibliography
Advanced Architectures
Project Publications
Aiello, N. 1988. Cage: The Performance of a Concurrent Blackboard Environment, Technical Report,
KSL-88-80, Heuristic Programming Project, Dept. of
Computer Science, Stanford Univ.
Aiello, N. 1986. User-Directed Control of Parallelism: The CAGE System, Technical Report, KSL86-31, Heuristic Programming Project, Dept. of
Computer Science, Stanford Univ. Also in 1986.
Proceedings of the Defense Advanced Research Projects Agency Expert Systems Workshop, 146–151.
Arlington, Va.: Science Applications International
Corp.
Bandini, J. C. 1989. Poligon Applications, Technical
Report, KSL-89-43, Heuristic Programming Project,
Dept. of Computer Science, Stanford Univ.
Brown, H.; Schoen, E.; and Delagi, B. 1986. An
Experiment in Knowledge-Base Signal Understanding Using Parallel Architectures, Technical Report,
STAN-CS-86-1136, Heuristic Programming Project,
Dept. of Computer Science, Stanford Univ. Also in
1986. Proceedings of the Defense Advanced
Research Projects Agency Expert Systems Workshop, 93–105. Arlington, Va.: Science Applications
International Corp.
Byrd, G. 1989. Support for Fine-Grained Message
Passing in Shared-Memory Multiprocessors, Technical Report, KSL-89-15, Heuristic Programming Project, Dept. of Computer Science, Stanford Univ., 1989.
Byrd, G., and Delagi, B. 1988. A Performance Comparison of Shared Variables versus Message Passing,
Technical Report, KSL-88-10, Heuristic Programming
Project, Dept. of Computer Science, Stanford Univ.
Also in 1988. Proceedings of the Third International Conference on Supercomputing, 1–7. Boston,
Mass.: International Supercomputing Institute.
Byrd, G., and Delagi, B. 1987. Considerations for
Multiprocessor Topologies, Technical Report, KSL87-07, Heuristic Programming Project, Dept. of
Computer Science, Stanford Univ. Also in 1987.
Proceedings of the Defense Advanced Research Pro-
WINTER 1989
37
Articles
jects Agency Knowledge-Based Systems Workshop,
119–122. Arlington, Va.: Science Applications International Corp.
Byrd, G; Nakano, R.; and Delagi, B. 1987a. A
Dynamic, Cut-Through Communications Protocol
with Multicast, Technical Report, STAN–CS–
87–1178, Heuristic Programming Project, Dept. of
Computer Science, Stanford Univ.
Byrd, G.; Nakano, R.; and Delagi, B. 1987b. A Pointto-Point Multicast Communications Protocol, Technical Report, KSL-87-02, Heuristic Programming
Project, Dept. of Computer Science, Stanford Univ.
Byrd, G.; Saraiya, N.; and Delagi, B. 1988. Multicast
Communication in Multiprocessor Systems, Technical Report, KSL-88-81, Heuristic Programming
Project, Dept. of Computer Science, Stanford Univ.
Davies, B. 1986. CAREL: A Visible Distributed Lisp,
Technical Report, KSL-86-14, Heuristic Programming Project, Dept. of Computer Science, Stanford
Univ. Also in 1986. Proceedings of the Defense
Advanced Research Projects Agency Expert Systems
Workshop, 171–178. Arlington, Va.: Science Applications International Corp.
Delagi, B., and Saraiya, N. 1988. ELINT in LAMINA:
Application of a Concurrent Object Language,
Technical Report, KSL-88-33, Heuristic Programming Project, Dept. of Computer Science, Stanford
Univ.
Delagi, B.; Saraiya, N.; and Byrd, G. 1986. LAMINA:
CARE Applications Interface, Technical Report,
KSL-86-67, Heuristic Programming Project, Dept. of
Computer Science, Stanford Univ. Also in 1988.
Proceedings of the Third International Conference
on Supercomputing, 12–21. Boston, Mass.: International Supercomputing Institute.
Delagi, B.; Saraiya, N.; Byrd, G.; and Nishimura, S.
1988. CARE User’s Manual, Technical Report, KSL88-53, Heuristic Programming Project, Dept. of
Computer Science, Stanford Univ.
Delagi, B.; Saraiya, N.; Nishimura, S.; and Byrd, G.
1987. Instrumented Architectural Simulation, Technical Report, STAN–CS–87–1189, Heuristic Programming Project, Dept. of Computer Science,
Stanford Univ. Also in 1988. Proceedings of the
Third International Conference on Supercomputing, 8–11. Boston, Mass.: International Supercomputing Institute.
Delagi, B.; Saraiya, N.; Nishimura, S.; and Byrd, G.
1986. An Instrumented Architectural Simulation
System, Technical Report, KSL-86-36, Heuristic Programming Project, Dept. of Computer Science,
Stanford Univ. Also in 1988. Artificial Intelligence
and Simulation: The Diversity of Application. San
Diego, Calif.: The Society for Computer Simulation
International. Also in 1986. Proceedings of the
Defense Advanced Research Projects Agency Expert
Systems Workshop, 106–118. Arlington, Va.: Science Applications International Corp.
Delaney, J. 1986. Multi-System Report Integration
Using Blackboards, Technical Report, KSL-86-20,
Heuristic Programming Project, Dept. of Computer
Science, Stanford Univ. Also in 1986. Proceedings
of the Defense Advanced Research Projects Agency
38
AI MAGAZINE
Expert Systems Workshop, 179–184. Arlington, Va.:
Science Applications International Corp.
Hailperin, M. 1988. Load Balancing for Massively
Parallel Soft–Real-Time Systems, Technical Report,
KSL-88-62, Heuristic Programming Project, Dept. of
Computer Science, Stanford Univ.
Nakano, R., and Minami, M. 1987. Experiments
with a Knowledge-Based System on a Multiprocessor, Technical Report, KSL-87-61, Heuristic Programming Project, Dept. of Computer Science,
Stanford Univ.
Nii, H. P. 1986. CAGE and POLIGON: Two Frameworks for Blackboard-Based Concurrent Problem
Solving, Technical Report, KSL-86-41, Knowledge
Systems Laboratory, Dept. of Computer Science,
Stanford Univ. Also in 1986. Proceedings of the
Defense Advanced Research Projects Agency Expert
Systems Workshop, 142–145. Arlington, Va.: Science Applications International Corp.
Nii, H. P., and Rice, J. 1989. Signal Understanding
and Problem Solving: A Concurrent Approach to
Soft-Real-Time Systems, Technical Report, KSL-8973, Heuristic Programming Project, Dept. of Computer Science, Stanford Univ. Also in 1989.
Proceedings of the Twenty-Third Asilomar Conference on Signals, Systems, and Computers. San Jose,
Calif.: Maple. Forthcoming.
Nii, H. P.; Aiello, N.; and Rice, J. 1988a. CAGE and
Poligon: Two Frameworks for Concurrent Problem
Solving, Technical Report, KSL-88-02, Heuristic Programming Project, Dept. of Computer Science,
Stanford Univ.
Nii, H. P.; Aiello, N.; and Rice, J. 1988b. Experiments on Cage and Poligon: Measuring the Performance of Parallel Blackboard Systems, Technical
Report, KSL-88-66, Heuristic Programming Project,
Dept. of Computer Science, Stanford Univ. Also in
1989. Readings in Distributed Artificial Intelligence,
Volume 2, eds. M. N. Huhns and L. Gasser, San
Mateo, Calif.: Morgan Kaufmann. Forthcoming.
Noble, A. 1988. ELMA Programmer’s Guide, Technical Report, KSL-88-41, Heuristic Programming Project, Dept. of Computer Science, Stanford Univ.
Noble, A., and Rogers, E. 1988. AIRTRAC Path Association: Development of a Knowledge-Based System
for a Multiprocessor, Technical Report, KSL-88-41,
Heuristic Programming Project, Dept. of Computer
Science, Stanford Univ.
Okuno, H., and Gupta, A. 1987. Parallel Execution
of OPS5 in QLISP, Technical Report, KSL-87-43,
Heuristic Programming Project, Dept. of Computer
Science, Stanford Univ.
Okuno, H.; Osato, N.; and Takeuchi, I. 1987. Firmware
Approach to Fast Lisp Interpreter, Technical Report,
STAN-CS-87-1184, Heuristic Programming Project,
Dept. of Computer Science, Stanford Univ.
Rice, J. 1989. The Design and Implementation of
Poligon, a High-Performance, Concurrent Blackboard System Shell, Technical Report, KSL-89-37,
Heuristic Programming Project, Dept. of Computer
Science, Stanford Univ.
Rice, J. 1988a. The Advanced Architectures Project,
Technical Report, KSL-88-71, Heuristic Programming
Articles
Project, Dept. of Computer Science, Stanford Univ.
Other Publications
Rice, J. 1988b. The Elint Application on Poligon:
The Architecture and Performance of a Concurrent
Blackboard System, Technical Report, KSL-88-69,
Heuristic Programming Project, Dept. of Computer
Science, Stanford Univ. Also in 1989. Proceedings of
the Eleventh International Joint Conference on
Artificial Intelligence, 212–217. Menlo Park, Calif.:
International Joint Conferences on Artificial Intelligence, Inc..
Abelson, H., and Sussman, G. 1983. Structure and
Interpretation of Computer Programs. Cambridge,
Mass.: MIT Press.
Rice, J. 1988c. Problems with Problem Solving in
Parallel: The Poligon System, Technical Report, KSL88-04, Heuristic Programming Project, Dept. of
Computer Science, Stanford Univ. Also in 1988.
Proceedings of the Third International Conference
on Supercomputing, 25–34. Boston, Mass.: International Supercomputing Institute. Also in 1989. Artificial Intelligence, Simulation, and Modelling, ed.
Lawrence Widman, 231–253. New York: Wiley.
Rice, J. 1986a. Poligon, A System for Parallel Problem Solving, Technical Report, KSL-86-19, Heuristic
Programming Project, Dept. of Computer Science,
Stanford Univ. Also in 1986. Proceedings of the
Defense Advanced Research Projects Agency Expert
Systems Workshop, 152–159. Arlington, Va.: Science Applications International Corp.
Rice, J. 1986b. The Poligon User’s Manual, Technical Report, KSL-86-10, Heuristic Programming Project, Dept. of Computer Science, Stanford Univ.
Rice, J., and Aiello, N. 1989. See How They Run:
The Architecture and Performance of Two Concurrent Blackboard Systems, Technical Report, KSL-8908, Heuristic Programming Project, Dept. of
Computer Science, Stanford Univ. Also in 1989.
Blackboard Architectures and Applications: Current
Trends, eds. V. Jagannathan and R. Dodhiawala,
153–178. San Diego, Calif.: Academic.
Saraiya, N. 1989. Design and Performance Evaluation of a Parallel Report Integration System, Technical Report, KSL-89-16, Heuristic Programming
Project, Dept. of Computer Science, Stanford Univ.
Saraiya, N. 1988. A Shared-Memory Lisp Package for
CARE, Technical Report, KSL-88-85, Heuristic Programming Project, Dept. of Computer Science,
Stanford Univ.
Saraiya, N. 1986. AIDE: A Distributed Environment
for Design and Simulation, Technical Report, KSL86-56, Heuristic Programming Project, Dept. of
Computer Science, Stanford Univ. Also in 1986.
Proceedings of the Defense Advanced Research Projects Agency Expert Systems Workshop, 185–190.
Arlington, Va.: Science Applications International
Corp.
Schoen, E. 1986. The CAOS System, Technical
Report, KSL-86-22, Heuristic Programming Project,
Dept. of Computer Science, Stanford Univ. Also in
1986. Proceedings of the Defense Advanced
Research Projects Agency Expert Systems Workshop, 160–170. Arlington, Va.: Science Applications
International Corp.
Stanford University. 1988. Expert Systems on Multiprocessor Architectures: Phase One Final Report,
RADC-TR-88-187, Rome Air Development Center.
Engelmore, R., and Morgan, T. 1988. Blackboard Systems. Reading, Mass.: Addison-Wesley.
Gabriel, R., and McCarthy, J. 1984. Queue-Based
Multi-processing Lisp. In Proceedings of the ACM
Symposium on Lisp and Functional Programming,
25–44. New York: ACM Press.
Gupta, A. 1986. Parallelism in Production Systems.
Ph.D diss., Dept. of Computer Science, CarnegieMellon Univ.
Nii, H. P., and Aiello, N. 1979. AGE: A KnowledgeBased Program for Building Knowledge-Based Programs, Technical Report, HPP-79-4, Heuristic
Programming Project, Dept. of Computer Science,
Stanford Univ. Also in 1979. Proceedings of the
Sixth International Joint Conference on Artificial
Intelligence, 645–655. Menlo Park, Calif.: International Joint Conferences on Artificial Intelligence,
Inc.
Nii, H. P.; Feigenbaum, E.; Anton, J.; and Rockmore,
A. 1982. Signal-to-Symbol Transformation:
HASP/SIAP Case Study, Technical Report, HPP-82-6,
Heuristic Programming Project, Dept. of Computer
Science, Stanford Univ. Also in AI Magazine 3(2):
23–35.
Selig, L. 1987. An Expert System Using Numerical
Simulation and Optimization to Find Particle Beam
Line Errors, Technical Report, KSL-87-36, Heuristic
Programming Project, Dept. of Computer Science,
Stanford Univ.
Williams, M.; Brown, H.; and Barnes, T. 1984.
TRICERO Design Description, ESL Inc.
Acknowledgments
The author gratefully acknowledges the support of
the following funding agencies for this project: The
Defense Advanced Research Projects Agency/Rome
Air Development Center under contract F30602-85C-0012, the National Aeronautics and Space
Administration under contract NCC 2-220, and
Boeing Computer Services under contract W-266875.
James Rice has been a researcher
in Stanford University’s Knowledge Systems Laboratory since
1985, concentrating on concurrent problem-solving models, particularly the Poligon system. He
received his B.Sc. in Cybernetics at
the University of Reading in England in 1981. He worked for four
years at SPL Int., a software house, on expert system
tools and the United Kingdom’s first blackboard
system, MXA.
FALL 1989
39