T55.4
CSWEY
.W2
no.
A
Compararive Analysis of Design
Rationale Representations
Jintae Lee
Kum-Yew
March 1992
Lai
WP #
84-92
INTERNATIONAL CENTER
FOR RESEARCH ON
MANAGEMENT OF TECHNOLOGY
Massachusetts Institute of Technology
Sloan School of Management
Cambridge, Massachusetts
The International Center for Research
on the Management of Technology
A
Compararive Analysis of Design
Rationale Representations
Jintae Lee
Kum-Yew
Lai
WP
March 1992
Sloan
WP#
3295
CCS TR#
A
revised
and condensed version
Human-Computer
©
Interaction
&
# 84-92
3405
121
of this report appears in the special issue of
on design
rationale, v.6(3-4), pp.251-280.
1992 Massachusetts Institute of Technology
Sloan School of Management
Massachusetts Institute of Technology
38 Memorial Drive, E56-390
02139-4307
Cambridge,
MA
d/l.l.T.
TF^
l,f'''^A»''^"1
? 3 1993
A
and condensed version of
revised
Human-Computer
this report
appears in the special issue of
on design
rationale, v.6(3-4), pp. 251-280
Interaction
A Comparative Analysis of
Design Rationale Representations
Jintae Lee
and
Kum-Yew
Lai
Center for Coordination Science
and MIT Artificial Intelligence Laboratory
ABSTRACT
A
few representations have been used
what ways they
we
for capturing
are adequate or limited so that
develop a framework
generic design tasks.
We
design rationale.
we know how
to
important
It is
improve them.
for evaluating design rationale representations
to
know
in
In this paper,
based on a
set of
build the framework by progressively differentiating the elements of
when made explicit, support an increasing number of the design tasks.
we evaluate the expressiveness of the existing representations. We also
present a language, DRL, that we believe is the most expressive of the existing representations
without being too complex for human users. We also discuss the limitations of DRL as open
design rationale
With
this
problems
that,
framework,
for further research.
1.
INTRODUCTION
As
the articles in this issue point out, an explicit representation of design rationale can bring
many
benefits.
Such a representation can lead
[McCall 1987; Yakemovic
principles underlying
&
human-computer
provide a basis for learning,
The
1990a].
largely
extent to
were represented
from what
we
understanding of the issues involved
justification,
interaction [Carroll
&
benefits,
1989].
we
already get from the notes on paper that
obtain from
we
It
of the
can also
et al. 1989;
Lee
however, would depend
for representing design rationale.
in free text, the benefits
and
et al. 1989],
Campbell
and computational support [Fischer
which we can actually reap these
on the language we use
rationale
to a better
Conklin 1990], of the design space [MacLean
If,
it
for
example, design
would not be
take in design meetings.
different
Also, the
Jintae Lee and Kum-Yew Lai, Center for Coordination Science,
02139.
Massachusetts Institute of Technology, E40-140, 1 Amherst Street, Cambridge,
Electronic mail addresses: iin@ai.mit.edu and kumyew@eagle.mit.edu.
Authors' present address:
MA
Comparative Analysis of Design Rationale Representations
CONTENTS
1.
INTRODUCTION
2.
WHAT DO WE WANT TO DO WITH
3.
WHAT'S IN DESIGN RATIONALE?
4.
EXISTING REPRESENTATIONS
1
3
5
15
4.1.
IBIS (Issue Based Information System)
16
4.2.
Toulmin's Model of Argumentation
21
4.3.
QOC
24
4.4.
Other Representations
(Question, Option, and Criteria)
26
DRL (DECISION REPRESENTATION LANGUAGE)
5.
5.1.
Introduction
5.2.
Description of
29
29
DRL
DRL
30
as a Design Rationale
5.3.
Evaluation of
5.4.
Relation to Other Studies
Language
35
40
CONCLUSIONS
6.
42
kinds of computational support that
explicit
to
DESIGN RATIONALE?
we
can provide depends on what a representation makes
and how formal the representation
is.
A few systems have been built and actually used
& Rlttel 1970; McCall 1987; Conklin & Begeman
capture design rationale or arguments [Kunz
1988; Fischer et
al.
1989; Lee 1990a, 1990b;
McCall
1990],
based on the earlier studies of design activity [Kunz
1958].
However, there
is
no systematic attempt
and most of them used representations
& Rittel
1970] or of argumentation [Toulmin
to justify the choice of these representations or
discuss the rationale for using them.
This paper
is
motivated by the following questions:
representations?
how do we
Do
they allow us to easily represent what
How
adequate are the existing
we want
to represent?
evaluate a language for representing design rationale? This paper
answer these questions by identifying the elements of design rationale
explicit
and by exploring
the consequences of
making them
explicit.
is
In general,
an attempt
that could be
to
made
Laying out these elements
provides a framework for placing the existing meanings of design rationale in perspective, as
well as providing a framework for evaluating a representation language for design rationale, as
we hope
to
show
in this paper.
Comparative Analysis of Design Rationale Representations
We
proceed in the following way.
we
First,
reference against which
we
we would
choice of that artifact.
and making
resulting
which
explicit
We
what
model allows us
to define the
framework,
we
artifact is associated
support
In Section 3,
discuss.
We
start
with a
with a body of reasons for the
then elaborate this simple model by incrementally differentiating
implicit in the
is
to do.
body
of reasons.
As we do
scope of a representation and
adequacy within
its
evaluate the existing representations in Section
we
representations and overcomes
many
we
current limitations, which
so,
we
what each
discuss
These models of design rationale provide a framework
language, called DRL, which
2.
we
evaluate the representations that
to
will use these tasks as a
characterize design rationale by presenting progressively richer models.
simple model of design rationale, where an
its
we might want
identify the tasks that
Throughout the paper, we
using a design rationale representation.
3
believe
is
its
more expressive than most
of their limitations.
we
In Section 5,
4.
Using
scope.^
in
this
present a
of the existing
As we describe DRL, we
also discuss
present as open problems for future research.
WHAT DO WE WANT TO DO WITH DESIGN RATIONALE?
To evaluate
a representation,
we need
know what
to
tasks
designed
it is
to
For example, [Mostow 1985]
different levels of abstractions. ^
lists
The
support.
that a design rationale representation can or should support can be described in
tasks
many ways
at
the following tasks:
documentation, understanding, debugging, verification, analysis, explanation, modification,
and automation.
[Fischer et
al.
1991] points out that
maintenance and redesign of an
during design process.
[MacLean
documenting design rationale can support
reuse of the design knowledge, and
artifact,
et al. 1991] list
two major
benefits
The
representation: aid to reasoning and aid to communication.
critical reflection
from design rationale
tasks of achieving these
benefits are elaborated further in terms of subtasks such as enabling the designers to envisage
the available design alternatives in a
more structured way, with
the
arguments
for
and against
them.
Another way of characterizing the tasks
is
to
list
accomplish the general tasks mentioned above
want our design
In this paper,
the questions that
in the
rationale representation to help
a
we
To
the extent that
use the terms, model and representation, in the following way.
model can have several representations.
structure independent of a particular
we
often need to answer to
way
use the term representation to refer
we
answer these questions, answering these
algebraic concept, and representation a linguistic one. Since the
many ways,
we
design process.
of describing
it,
same
Therefore,
we
when we want
use the term model.
to a particular notation for
Model
is
an
structure can be described in
On
to discuss a
the other hand,
describing the structure.
There are many good discussions of the benefits or tasks for which a design ratioanle can be used.
Those not discussed in this paper include [McCall 1987 ;Yakemovic & Conklinl990 ].
Comparative Analysis of Design Rationale Representations
questions become the tasks that the representation should support.
we
representative questions that
1991a],
is
a set of
et al. 1991],
and from creating scenarios
[Carroll
1990].
•
What
is
•
What
did
•
What
are the alternative designs and
•
What
are the
•
The following
gathered from our experiments with design rationale [Lee
from walking through examples [Lewis
& Rosson
4
the status of the current design?
we
discuss last
week and what do we need
two most favorable
Sun Microsystems
what are
to
do today?
their pros
and cons?
alternatives so far?
X/NeWs
just released their
server.
How
would
the release change our
evaluations?
we do
•
What
•
Why
is
•
What
are the issues that
•
What
are the unresolved issues?
•
What's the consequences of doing away with
•
How
if
not consider portability?
portability important
anyway?
depend on
this issue?
What
did other people deal with
are
we currently doing about them?
this part?
problem? What can
this
we
learn
from the past design
cases?
The above
we
list
of the questions
did not walk through and
is
by no means complete as there are many possible paths that
many
scenarios that
we
did not construct.
We also
left
out those
questions which, though important, do not seem to be the job of design rationale to answer
How
can
we compute
the total cost of this design?)
(e.g.
Nevertheless, the questions in the
list
provide a useful framework for assessing the expressiveness of the different representations.
When we
discuss the limited or increased expressiveness of a given representation,
those questions that can or cannot be answered as a result.
question which
is
not represented in the
list,
then
we
If
We
want
to
would have
emphasize further
that
we
representations of design rationale. That
not
mean
questions.
that
any representation
for
1).
made
if
not,
it is
desirable to answer the questions in the
list
does
design rationale should support the answering of these
Each representation must weigh the costs and benefits involved
These tradeoffs should
what additional
explicit.
are assessing only the expressiveness of the existing
three general dimensions: expressiveness,
(Figure
be
to
refer to
can always evaluate the representations
by asking whether they would support answering the question and
objects, attributes, or relations
we
our task includes answering a
in turn
human
usability,
in tradeoffs
and computational
among
tractability
be motivated by the tasks that are intended
to
be
Comparative Analysis
of
accomplished using the representation.
Design Rationale Representations
we
In short,
this
paper would make the architects of a representation
what
their
what any
are not dictating
we do hope
representation should or should not have. However,
for
5
existing
that the analysis presented in
design rationale more conscious of
language can or cannot express and why.
Domain
Figure
To be
as
sure,
human
that
is
in
computer supported
we cannot separate our concern with expressiveness
not
we
Elements
to capture design rationales, but
much
point in evaluating
falls into that
discuss
if it is
is
be
difficult to evaluate tradeoffs
&
dimensions such as expressiveness
is
complex
from other concerns such
a language
for
people
Whether any
1991].
is
to
meant
to
be
manage, then
of the representations
All the languages discussed
Nevertheless,
will
we
all
succeed at
believe
it
would
the three dimensions without calibrating individual
[Brachman
(cf.
if
no guarantee that they
Yakemovic
among
entirely
an empirical question.
here have been actually used by people, but that
their "industrial strength" use [Conklin
too
expressiveness.
its
category
activities.
For example,
usability or computational tractability.
used by people
there
1.
&
Levesque 1984] on tradeoffs between
expressiveness and computational tractability for general knowledge representation).
3.
WHAT'S IN DESIGN RATIONALE?
What
is
meanings
design rationale?
in
[Yakemovic
Rosson
its
&
1990],
Design rationale
for
an
artifact
has at least three different
current usage: a historical record of the reasons for the choice of an artifact
Conklin 1990], a
and
set of psychological
a description of the design space
&
claims embodied by an artifact [Carroll
[MacLean
&
et al. 1989].'^
Conklinl990] describes logical as well as historical aspects
For example, the support relation has both the logical
We associate [Yakemovic & Conklin 1990 ], as well as [Lee
aspect and the historical aspect because
record because one of their goal is to capture and
with
historical
Potts
&
Bruns
the
1990a;
1988]
The representation used
of design rationale, as
in
we
[Yakemovic
discuss further
,
document
the actual process of design.
later.
Comparative Analysis
Design rationale often means the
Design Rationale Representations
of
historical record of the analysis that led to the choice of the
particular artifact or the feature in question.
To
command
like:
window,
specific to a
A design
window.
menubar
in the global
SAVE
is
to
put the
window commands was
that
it
saves screen space
window
in
anyway because people
such as more
designed
make
to
executing a
(e.g.,
in the sense of [Carroll
artifact is to
&
command.
we
&
Rosson
1990].
and formulated
like:
is
We can
have the global menu
provide more structure
Such structure
the set of psychological claims
i.e.
"Dimming
commands."
meaning
if
in a testable form.
"The global
learn about the
"the claims that
usually
is
embodied by an
would have
of design rationale
they were, they
to
artifact
be true
if
historical record; the claims
would have
the
how
one used by [MacLean
are these alternatives related,
Macintosh example, the design rationale
logically possible alternatives for placing
the tradeoffs are.
It is
need
be extracted from
the environment easier to use because
the irrelevant items in the global
is
to
For example, the design rationale in this sense
menu makes
what are
in this
menu makes
et al. 1989],
located in the space of possible design alternatives:
alternatives,
to
each
from
an argument supports a proposal) and/or
These claims are different from the
clutter." or
*
decided
travel
a proposal replaces another proposal).
Kellogg 1989],
reduces screen
is
mouse
be successful" or "the claims about the psychological consequences for the user"
would be something
third
commands on
requires a long
we
20.
bar at the top of the screen
discuss in the rest of the paper.
not be present in the historical record; even
the record
But,
would be something
by Mark on January
to putting the
it
a
generally agreed that that the advantage, together with others
Another meaning of design rationale
artifact
opposed
explicit the logical structure (e.g.,
the historical structure
The
as
window
the
that saves the contents of the
raised
implementation, outweigh the objection."
efficient
to this historical record, as
[Carroll
(e.g.,
all
window command, we mean
menu
into the global
as in the Star environment). Julie objected because
the currently active
bar
a
window command
a
Kevin proposed the idea of incorporating them
window,
By
rationale for this feature in the sense of historical record
"The issue of where
and pointed out
namely the placement of
at the top of the screen."*
e.g.
us take as an example a
illustrate, let
particular feature of the Macintosh operating system,
commands
6
sense would be
window commands, how
it
easier to
namely how a given
what are
the tradeoffs
it
the other possible
among them?
some
In
our
description of the
they are related, and what
often difficult to provide such a description in a systematic way, but an
Since the unit of an artifact is often ambiguous (e.g., Macintosh, Macintosh OS, Macintosh window
system, Macintosh menubar, or the position of the menubar), we will use the term artifact in the
general sense to
mean any
feature, usually a small one, for
which there
is
design rationale.
Comparative Analysis of Design Rationale Representations
example
found
is
in
[Card
et ai. 1990;
McKinlay
et al. 1990],
which provides
7
a vocabulary of the
primitives and a set of composition operators for describing the design space of possible input
This meaning of design rationale seems different from the
devices.
emphasis on design rationale not being
emphasis not on a particular
artifact
a record
first
meaning
but a construction, and from the second
but on the relation
among
in its
in its
possible alternatives.
Models of Design Rationale
We now
develop a series of progressively richer models of design rationale, which provide
framework
in
which we can place
the three different
two meanings are discussed immediately below. The
possibility space,
is
discussed shortly
Design rationale
in the
way
our
it is.
reasons as
So
in
shown
model
we
as a historical record
and
that led to the choice of
as a set of claims
an
a
first
rationale, as a
an
designed the
artifact is
associated with a
body of
to extract
and sequence
On
it
The
that led to the design,
we
justify the
design of the
artifact.
menu
is
better because
it
The record
at least
of the process
that artifact
If
was chosen.
we wanted
such a record
and provides
a basis
a logical
tells
us the
from which
can represent the logical reasons directly,
no matter how or
in
to
i.e.
what order they were
artifact is
an example because these claims
These claims are
logical also in the sense that the
embodied by an
context in which they are true have to be
why
rationale, respectively,
an example.
from the record; but
the other hand,
set of claims
is
The reasons can be
artifact.
artifact.^
one kind of reason
the reasons justifying the choice of an artifact
"The global
is
meanings of design
embodied by an
artifact tells us
we would have
infer the logical reasons.
would
why
can give for an
to the
example about the window commands above
historical circumstances
articulated.
The
in Fig. 2a.
roughly corresponding
justification,
an explanation of
is
historical or logical,
free text
meaning of design
third
of design rationale, an artifact
There are different kinds of reasons that
The
rationale.
after.
most general sense
first
meanings of design
made
explicit.
For example, a claim should not say
leads to smaller implementation"
if it
really
means "The
We believe that the distinction between historical and logical reasons breaks if we push it too far
because a purely historical record per se does not really give us a reason. It would give us a reason
only to the extent that we can extract some logical structure out of it. Nevertheless, we believe that
the distinction is useful for the purpose of evaluating representations because for a given
representation we would like to know what it makes explicit and what we have to infer from it.
Comparative Analysis of Design Rationale Representations
(a)
MODEL
(b)
MODEL 2:
I
:
An
artifact is
assoaated with a body of
Alternatives and their relations are
all
made
the
arguments relevant to the design of the artifact
explicit
and the arguments about individual ahematives
can be differentiated.
Ct^'^
''^'k^lr^
'2.^*% a
CTi)
"^^3
Figure 2
(a)-(c).
Progressively
(continued on the next page)
more
differentiated
models of design rationale
Comparative Analysis of Design Rationale Representations
^'ne,^
(d)
MODEL 4:
Criteria used for evaluations
further differentiated in the
argument
and
their relations are
made
explicit
and the arguments about them can be
space.
MODEL
5; Individual issues are made explicit, each of which contains the alternatives, evaluations, and criteria used in
(e)
discussing the issue. A part of the argument space includes the meta-arguments about the issues and their relations.
Figure 2
(d)-(e).
Progressively
more
differentiated
models of design rationale
(continued from previous page).
global
menu
is
implementation.
better
This
in
is
memory." Extracting these
the context of the Macintosh because
important
in a
logical reasons
system
is
like the
it
leads to smaller
Macintosh which has a small
not an easy task; once identified, however, they
provide the advantages of being testable and general.
Comparative Analysis of Design Rationale Representations
The
internal structure of these reasons can be
they can be completely undifferentiated.
we gave
earlier as
more
relations
such
is
an example of a
we
explicit,
the natural language description that
If
we were
make
to
the historical
&
and Replaces. An example
Rosson 1991]
that
for describing the claims
not historical
is
"embodied"
in
an
In this representation, the claims themselves are represented in natural language, but
the claims are grouped by the questions they answer
how do do
I
We
this?
made more
Qualifies,
called a
is
historical record.
Initiates,
the representation used by [Carroll
artifact.
be
At one extreme,
explicit to different degrees.
An example
can differentiate further by making explicit such roles and relations
Second-Motioner,
as: Initiator,
made
10
explicit
what can
I
do,
how
does that work, and
by providing such constructs as Logically-Implies, Supports, Denies,
and Presupposes.
body
to:
can also imagine a representation where the logical support relations can
We
Argument Space,
will use the term.
to refer to
what we have
of reasons, because the reasons are captured either as a historical record of the
various arguments relevant for the design of the artifact or logical arguments underlying the
design.
There
is
much we
can do with our
model can help us answer
first
do today?" Such a representation can
deal with this problem?
&
model
the question,
Can we
design rationale, mentioned above, suggested
in Smalltalk).
which can be
practice" of
Our
first
tested
and compiled
to form, in the
model, however, does not help very
representation based on the model has
itself
we
How
how
long run,
to
did other people
[Carroll
their representation of
issues for redesigning
much with
Saying that
cannot answer these questions. Of course,
text,
in detail
issues can be
this
an
artifact (the
View
couched as a design hypothesis,
"a
contextualized science out of
interaction.
qualify this statement immediately.
language free
many
They discuss how these
human computer
based on
week and what do we need
from the past design cases? Carroll and Rosson
Rosson 1991] provide a good example. They report
Matcher
last
answer the questions:
also help us
learn
A representation
of design rationale.
"What did we discuss
if
it
the other questions, though
does not help
the user
much
is
we
should
not to say that
works hard enough, and as long
enough information captured, even
can answer these questions. So the
real issue is
in the
we
as the
form of natural
how much
the
model
help us answer these questions either by helping us see the structure better or by enabling
us to define computational services that help us answer the questions.
differentiated
the cost in
models allow us
to
answer these questions more
some other ways [Conklin
&
Yakemovic
1991].
easily,
We
will see
how more
although they increase
Comparative Analysis of Design Rationale Representations
11
by making multiple alternatives and
Our second model
(Fig. 2b) differs
relations explicit.
Design involves formulating several alternatives, comparing them, and
from the
merging them, as many of the questions
alternative
made
is
our
the ones that have been rejected.
their attributes (e.g. current status
among
Once
the alternatives
many
of the issues
when
this set of
alternatives explicit, including
become
The
comparing
(e.g.
explicit,
we can
talk
'),
about
make
one
alternatives, other than the
and the knowledge used
finally chosen, are interesting
in evaluating
We
situational constraints change.
alternatives, displaying the
even argue about whether an alternative
them
is
because
are useful in other contexts, for
use the term. Alternative Space, to refer to
multiple alternatives and their relations.
These relations among the alternatives can also be
may
model, only a single
first
the alternatives explicit (e.g. specialization, historical precedence),
alternatives that specialize this alternative), or
worth considering.
our
such as "rejected" or 'waiting for more information
define computational operations on them
example,
indicate. In
list
their
and the multiple alternatives are present only
explicit at a given time,
argument space. Our second model make these
implicitly in the
the relations
in
first
be not only the linear sequence that
relations such as layers
we
Historical relations
historical or logical.
usually describe as versions, but also
and contexts [Bobrow
&
The
Goldstein 1980].
more complex
logical relations
may
include Specializes, Generalizes, Elaborates, or Simplifies. Or alternatives can be related
through a design space [McKinlay
et al. 1990].
To
the extent that
represent these different alternatives and their relations,
within the scope of the representation.
space within
its
scope because one of
its
glBlS, for example,
goals
is
1988].
Position, with
among
The constructs
we want
a representation to
say that the alternative space
seems
among
these alternatives" [Conklin
in glBlS for representing the alternative
which we can describe multiple
alternatives,
space consist
and the specialization
&
of:
relation
the Positions.
By now, we have an
alternative space connected to the
argument space, as shovwi
each of the alternatives, there are arguments describing the reasons for
just as in
our
alternative,
first
i.e.,
its
model there are arguments describing the evaluation
that
it
was chosen. Some
argument can support an
of the
alternative while
it
is
For
status of that single
arguments can be shared;
denying another; so
in Fig. 2b.
current evaluation,
for
example, an
better to think of the
arguments about the different alternatives forming a single large argument space, as shown
Fig. 2b.
is
to include the alternative
"to capture alternative resolutions (including
those which are later rejected), [and] trade-off analysis
Begeman
we
in
Comparative Analysis of Design Rationale Representations
Once
the alternative space
us answer
some
what are
their pros
represented,
is
of the questions in our
we
can imagine
how we
can make a system that help
To answer "What are
list.
the alternative designs
and cons?," we can associate an argument space with each
week?"
consider this alternative, and
we need
some
to
how
the representation of the alternative space,
related to the
we
week?"
consider this alternative, and
to
among
Part-Of),
can make a system help us
are the alternative designs
how
is
use some historical relations
related to the
it
(e.g. Replaces)
To answer "Why do
explicit multiple alternatives,
what
is
argument space
namely the chosen one,
Similarly, the
about
the
arguments
why
generalize,
one that we discussed
last
or structural relations (e.g. Is-A-
the alternatives.
However, once we make
the
last
With
and cons?," we can associate an argument space with each of the
their pros
we need
discussed
how we
alternatives through the links such as Supports or Objects-To, as in gIBIS.
we even
we
the alternatives.
To answer "What
can imagine
in Section 2.
one that
among
such as Rqjlaces,
historical relations,
answer some of the questions posed
and what are
is it
and
of the
To answer "Why do
alternatives through the links such as Supports or Objects-To, as in gIBIS.
we even
12
(Fig. 2b).
we need
In our first
argument space contained reasons
for the other alternatives are
about
carefully
single artifact,
for the choice of that artifact.
why
they have their particular evaluation status,
"Waiting for More Information", "Rejected".
more
to articulate
model when we had a
they were not chosen, or, to
e.g. "Still in
Consideration",
These evaluation status could be nominal
categories (such as the above examples), ordinal categories (such as "Very
Good," "Good," and
"Poor") or a continuous measure (such as the probability that the alternative will achieve a
given set of goals).
Therefore,
explicit
we
and
introduce the evaluation space (Fig.
related.
evaluation measures
Usually,
we
we do
"Very Good," "Good," "Poor," "Very Poor")
them
to
manages these
produce a higher
values mean.
values, for
level
if
is
sufficient
However,
example
if
the evaluation status are
when we
we want
among
leave
it
for the
to define
to
made
among
the
these values (e.g.
human
user to
any computational
that automatically propagates
summary, then we need
and merges
be very careful about what these
We need to specify the units of measurement, a calculus for combining them, and a
model specifying what they mean. Even
example,
where
Often, the implicit ordinal relation
use.
assign these values to the alternatives.
service that
2c),
not and need not specify any elaborate relation
the
human
summary measure,
user
then
interpretation does not
is
we need
become
in the case
where these actions are
left to
human,
for
expected to combine these values to produce a higher level
to
arbitrary.
set
down what
these values
mean
so that their
Comparative Analysis of Design Rationale Representations
Making
the evaluation space explicit allows us to differentiate
space: (1) arguments about
why an
alternative has
arguments about the alternatives themselves,
an object as an alternative or whether
That
alternative.
is,
differentiated in the
now answer
evaluations?"
We
"What are
as:
Our models
so far
can also explain
is
duplicate
do not make
For example,
menu
the
how an
two most favorable
evaluation
items"
is
it
explicit the criteria
and
used
in
we
know
can group
If
that the
"Reducing Screen Clutter"
we have
is
a
to this criterion or
criteria
way
Menu
to
at the top of the
used as a criterion for evaluation.
becomes
Knowing how
less
important, then
this goal (for
example,
this criterion is related to others (e.g.
of achieving "Easy to Use"), also allows us to assign proper
change
its
importance,
when
the related criteria changes.
We
space to refer to these criteria and their relations. As Fig. 2d shows, once
the criteria space explicit,
it is
is
to represent
the arguments that appeal to this criterion
arguments that presuppose
we can
grouping those arguments which are about the
Hence,
all
important
argument "We do not need
the criterion changes or
to all the
these arguments less important).
use the term
arguments
this particular
producing an evaluation. However,
a pro-argument for the alternative "Global
this criterion explicit,
importance
to the
and explaining how
their relations are usually quite
important to
is
can do appropriate things
making
and "Sun
the release change our
was made by pointing
artifact in question,
and weigh them against one another.
we
alternatives so far?"
How would
server.
Screen" because o/the goal of reducing screen clutter, which
By making
another
derived or computed from them or related to other measures.
the criteria used for the evaluation
explicitly.
(2)
should or should not even consider
these different kinds of arguments can be
2c,
X/NeWs
argument space behind the
evaluation measure
why we
this alternative is really a special case of
in Fig.
released their
just
two components of the argument
current evaluation status, and
its
argument space. With the representation of the evaluation space, we can
questions such
Microsystems
in the
shown
as
e.g.,
13
and
their relations.
important that a language whose scope includes the
different attributes of the criteria
argument space by
further differentiate the
criteria
criteria
and the relationship among them.
space represent the
For example,
it
should
allow us to represent the importance of these criteria and the synergistic or tradeoff relations
among them. A
set of criteria
can be sub-criteria of another in the sense that satisfying them
facilitates the satisfaction of the latter.
These sub-criteria can be related
among
themselves in
various ways. They can be mutually exclusive in the sense that satisfying one makes
it
impossible to satisfy others. They can be independent of each other in the sense that satisfying
one does not change the likelihood
of satisfying others.
These sub-criteria can be related
to
Comparative Analysis of Design Rationale Representations
ways
their parent criterion in various
satisfying
With the
of
all
them
answer the questions such
important anyway?
as:
we
what
if
can
now
we do
The answer might be
evaluation of the alternative
X were based on
in the sense that
equivalent to satisfying the parent.
is
space represented,
criteria
They can be exhaustive
as well.
14
X changes
how
see
the system might be able to help us
not consider portability?
"If
we
"High" because
to
why
or
is
portability
give up the goal of portability, then the
these claims that argue against
all
the importance of portability." or "Portability
is
important because
it
is
a
subgoal of another important goal, Have a wide distribution." These answers can be derived
from a representation
criteria,
the representation
if
these questions.
It is
not obvious
how
explicit the relation
space seems a necessary condition
if
criteria is
between evaluations,
not sufficient for answering
these questions can be answered even
space are represented explicitly.
criteria
makes
and arguments. Of course, representation of the
we
However, the
some
if
parts of the
explicit representation of the criteria
are to answer these questions. At least,
we would have
the information necessary to define an operation that will give or suggest the answers to these
We will give some examples
questions.
So
we have
far,
artifact,
and discussed the structure of a
identified
namely which
of such operations later in Section 5.
of the alternative designs should
representation of such local structures alone, namely
criteria space,
we
unresolved issues and what are
depend on
its
we
questions,
A decision often
individual issues are related.
requires
we need
A
decision can be a specialization of another
case of the second.
term Issue Space
has as
its
them.
capture
to
A
how
we can
with
this
still
the latter requires
the
some questions
problem? Can
first
is,
decision
as
we
that
learn
a
more
the
first
detailed
and we use the
Once we have an
issue as an
and "Actions Taken" with
issues
and what are we currently
among
the issues will allow us to
"What are the unresolved
we have
is
decisions.
making
therefore, a single decision that
in Fig. 2e.
Representing the dependency relation
answer the question "What are the issues that depend
There are
if
and
are the
are the issues that
these decisions are related,
shown
What
more global picture of how
associate the attributes such as "Status"
and answer questions such
doing about them?"
if
unit in this issue space
internal structure the other spaces, as
explicit element,
issues
important
It is
to refer to
as:
and /or influences many other
For example, a decision can be a sub-decision of another
decision.
a
alternative space,
such
What
currently doing about them?
To answer such
this issue?
in the list
underlying an
However, with the
choose?
argument space,
cannot ask some of the questions
still
single decision
we
this issue?"
as: How did other people deal
We argue, however, that the five
not yet covered such
from the past design cases?
Comparative Analysis
of
Design Rationale Representations
spaces so far identified, the spaces of arguments, alternatives, evaluations,
can contain enough information
to
answer these questions.
In Section 5,
15
criteria,
and
issue,
we discuss computational
operations that help us answer these additional questions by exploiting the structure of the
five spaces.
EXISTING REPRESENTATIONS
4.
There are only a handful of representations that have been used
Kunz &
rationale [Toulmin 1958;
1988; Potts
&
Rittel 1970;
for representing
Marshall 1987; McCall 1987; Conklin
Bruns 1988; Lee 1990a; MacLean
et al. 1991].
Most, though not
design
& Begeman
of these
all,
representations are heavily influenced by the IBIS structure for representing issues [Kunz
Rittel 1970] or
MacLean
propose a representation based on
91] also
In the next three subsections (4.1-4.3),
rationales.
detail
by Toulmin's model of argumentation [Toulmin
by defining
we
In Section 5,
with constructing design
discuss these three representations in
we
present a language, called
of the limitations of the existing languages that
Some
qualifications are in order before
we
them
representations, but only to evaluate
is
proceed.
DRL, which we believe overcomes
we discuss
Our
in this section.
intention
is
not to crihcize the existing
design rationale representation language. Even
as a
mainly with respect
to their expressiveness.
We would
like to
emphasize again that the adequacy of a representation can be evaluated only with respect
set of tasks.
be able
it
to
One
representation
do more with
it.
In
discuss a few other studies that bear on design rationale
many
then, our evaluation
&
et al. 89;
scope and evaluating them within the framework outlined above.
their
the next subsection (4.4),
representation.
their experiences
we
[MacLean
1958].
may
be
However,
if
much more
what
it
expressive than another; as a result
enables
is
to a
we may
not in the set of desirable tasks, or
if
enables those only by sacrificing other more important constraints, then the additional
expressiveness
is
not worth what
representations allow or
it
gives us.
do not allow us
What we
to do;
discuss below
We do not intend
to
is
only what the existing
make value judgments about
whether the representations should or should not do. Some of these languages were designed
with different goals, which
in turn
determine the tradeoffs adopted
in the designs.
Therefore,
the following discussion should not be construed as a criticism of these representations, but only
as an articulation of their scope
and
languages with respect
to expressiveness.
We
clear that
also
want
information,
their
adequacy as design rationale representation
whenever we say
some
to
make
it
does not mean that people cannot infer that information from the
that a representation cannot express
"
Comparative Analysis of Design Rationale Representations
Take
representation.
example
in the
language representation of design rationale that
a natural
beginning of the paper.
If
we
been ever expressed
if
we work
cannot express some information,
that
make
that
makes
amenable
an
can always retrieve the information
When we
say that a representation
way
that help people see the structure better or
computational manipulation.
to
IBIS (Issue Based Information System)
4.1.
The IBIS structure was
originally developed in
representing designers' argumentation activities.
[Kunz
&
used by gIBIS [Conklin
Since gIBIS
& Begeman
incarnation of the original IBIS
discuss
one used by
them
The goal
among
,
"a
&
we
some
One
variations of
variation
is
purpose of
it
have been
the representation
hypertext tool for exploratory policy discussion.
Yakemovic
and can be regarded as
1991]
a
modem
use gIBIS as the context for discussing the IBIS
Other variations include PHI (Procedural Hierarchy of Issues) [McCall 1987]
representation.^
the
1988],
most well-known [Conklin
is
Rittel 1970] for the
Since then,
used by a few systems for representing design rationale.
and
as
that the representation does not provide constructs
the information explicit in such a
it
we
hard enough.
we mean
we gave
keep a detailed enough record of what happened,
or even a video recording of the whole design process,
that has
16
[Potts
&
Bruns 1988]
for the rationale
module
in their representation.
We
briefly in Section 4.4.
of gIBIS
is
to
capture "the design problems, alternative resolutions, tradeoff analysis
these alternatives,
and
the tentative
and firm commitments
that
were made
in the
process of the decision making process". Figure 3 shows the objects and relations that form the
language of gIBIS and Figure 4 shows an example representation. In gIBIS, one raises an Issue
such as where
to
"in the global
menubar
put the
window commands.
Arguments can be created
need
to duplicate
"Requires long
in
the figure.
to
and "Commands
Support or Object-to a Position.
commands
mouse
Positions are created to Responds-to the issue (e.g.,
at the top of the screen.",
for each
travel." objects to
window
it.
"
supports the
first
position,
and the argument
Also, an Issue can be related to other objects as
The gIBIS model extends the
generalize/specialize relation
at the top of each window.")
For example, the argument "Don't
among Arguments
original
IBIS
as well as
shown
model by introducing: the
among
Positions,
an additional
We believe that the distinction between historical and logical reasons breaks if we push it too far
because a purely historical record per se does not really give us a reason. It would give us a reason
only to the extent that we can extract some logical structure out of it. Nevertheless, we believe that
the distinction is useful for the purpose of evaluating representations because for a given
representation we would like to know what it makes explicit and what we have to infer from it.
Comparative Analysis of Design Rationale Representations
17
C_J_^>
Figure
f Commands
I
top of the
at
th^l
window.
Commands
Ilside ot the
at
3.
The glBIS vocabulary.
thd
window
C ^"
-
^
^
/"•"
responds-to
responds-to
_J
Dimmed commands
by
Figure 4
.
their
still
distracting
mere presence.
An example
V
'I
(^N°J—
I
in IBIS.
commanas
rea
jistjacting?
Comparative Analysis of Design Rationale Representations
18
Other type which serve as an "escape" mechanism for representing things not representable
with the available constructs, and an External type
for external objects
except
when we
to
both IBIS and gIBIS
discuss the generalize/specialize relation for Position and Argument.
The scope of the gIBIS representation depends on what an issue
sense
general
such as documents or
common
sketches. In the following, our discussion refers to the structure
mean any question
to
takes
that
If
is.
a
we
of
set
take an issue in a very
Positions,
then
the
Issue/Position/Argument structure can represent a fairly large part of the design rationale
spaces.
The
internal structure of these spaces,
however,
is
not well differentiated in gIBIS, as
we discuss below.
The
alternative space is represented in gIBIS
by
Positions
and the
among
relation
them.
Since
multiple Positions can be created for a given issue, gIBIS allows the representation of multiple
alternatives, thus offering at least the richness of our
among
the Positions that
we
can represent
in
gIBIS
although there are other relations that can connect
Questions or ls-Suggested-By)7
model
is
2.
The only
relation,
however,
the Specializes/Generalizes relation,
a Position to objects of other types (e.g.
Thus, gIBIS can answer the question "What are the other
alternatives being considered?" but not the question
"How
are these alternatives related?"
unless they happened to be related via the Specializes relation.
The
unit of the argument space in gIBIS
applicable to the current
Position (e.g. "In the global
position
is
example,
or
we
is
Argument.
An Argument
only
among
the
if
(e.g.
"The
commands
no construct, however,
for qualifying
Argument
"the global
menu
containing
menu
all
Argument
reduces the screen
possible action items
itself" is valid.
the arguments
is
again limited to the Specializes relation.
"Irrelevant
menus can be dimmed"
Argument, "The commands not applicable
distracting."
To express
this relationship
about whether the original Argument
argument." However,
the
an argument. For
This limitation
implies that an Argumentcannot Support or Object-to another Argument directly.
the
why
at the top of the screen"), indicating a reason
is
not
be shown and distracting.") can support or object to a
cannot indicate that the existing Argument, "the global
does not become so big
relation
menubar
not a good one. There
clutter", is valid
The
is
window would
I
was not
is
is
an argument objecting
to the current
among Arguments
window would
in gIBIS,
we have
right or not, create Positions "yes"
able to find any discussion of
it
yet.
For example,
to the original
be shown and
to create
and
"no",
an
Issue
and then
Comparative Analysis of Design Rationale Representations
A
argue about these Positions.^
new
to create three
Issue, a
A more
interface that hides the intermediate details.
that
new
objects: a
However, we might be able
supporting the "No" Position.
is
We
this claim."
it
might
try to
leads to the
Argument
serious limitation of this representation
me
the claims that respond to
all
and the issues
link
responding claims. However,
that contain the
too general for that purpose because
is
tin
reduce the complexity with an
answer these questions by following the Is-Suggested-By
that connects the original claim
the Is-Suggested-By link
"No" Position, and
to
does not help us answer questions such as "Show
it
that
is
For example, in order to just point out the factual inaccuracy of a claim,
proliferation of Issues.
we would have
possible criticism of this representation
19
does not allow us to
it
distinguish the issues that contain responding arguments from those that do not.
In gIBIS
we
we
cannot argue about relational claims. For example, there
agree with
A
and
B,
you can argue about.
but not that
A
To
with another example, suppose
illustrate
whether portability should be a
only
way we can do
a criteria?",
we
so in IBIS
criterion for evaluating the
by creating
and suggest possible answers
should not". Then,
the original issue
make
is
we
by an Is-Suggested-By
explicit the relation
between the object
Being able
want
to
to
new
in the
form of
relation.
we can argue
and the
agree with
However,
We
criteria
A
commands; he may point out
irrelevant
commands.
to
when
is
is
a
that the global
The user may want
is
to
new
issue to
If
we make
important for other reasons as well. People often
to
make
that
A
supports
B.
For
bad idea and that seeing irrelevant
it
first.
That
distracts the user
menubar does not have
is,
one may
by showing
to
show
the
the distinction clear because they
do not
even more
crucial
be construed as denying either of the claims alone. The distinction
the representation
The
again does not
being questioned.
and B but not with the claim
bad idea but not because
irrelevant
want
argue about
should" or "No,
this
this representation
distracting but not that the second claim supports the
a
we
can also relate
commands
is
to
given issue.
directly about the relation, Is-a-Criteria-for,
agree that the global menubar
menubar
not an object that
consider portability as
positions, say "Yes,
may
is
for a
we
example, one
believe that the global
is
original issue.
argue about relational claims
may
no way of saying that
we want
posiHons
say "Should
Issue,
between the new issue and the
portability
say that they
a
can argue about these positions.
a relation an explicit object, then
is
Supports B because a link in gIBIS
is
be used to provide computational services. Mixing denial of node
is to show the first argument as supporting the position that the second objects
instead of objecting to the second argument directly because an argument cannot respond to
another argument in gIBIS. This representation does not make explicit the relation between the two
Another alternative
to,
arguments. Probably for that reason, gIBIS does not encourage
personal communication)
this representation (Conklin,
Comparative Analysis of Design Rationale Representations
with denial of link
this
just
is
not right semantically.
you define
a
computational service based on
type of careless semantics, such as evaluation management that propagates and merges
evaluations,
what you
get
would not make much
The evaluation space used by gIBIS
"Chosen" assigned
Information," but beyond that
try to derive
some
space
criteria
gIBIS
is
we
We could
it is
We
some nominal
among
difficult to
more
sophisticated evaluation measures can
the positions based
imagine what the calculus would be
gIBIS. gIBIS provides
we
cannot argue about them;
nor can
criteria.
Again,
we
we
Fig. 4
indicate
we
no vocabulary
for describing the
shows an
explicit the relation
any relationship, such as mutually exclusiveness, among the
between
this issue
But, again, this representation does not
and the
criterion in question.
be even more
difficult for a
For example,
this relationship;
no easy way
disagreements
to
For
Issues.
Issue ("Should small implementation be a goal?") related to our
representation of the criteria entails other limitations.
people to see
Since criteria are
cannot represent the reasons for having these
could indirectly represent these relations by creating additional
original Issue via an Is-Suggested-by link.
As we
arguments, and
like.
a serious limitation for a design rationale representation language.
criteria;
is
to the
on supporting or objecting
use in evaluating the alternatives. This lack of explicit representation of criteria in
not explicit,
example.
and
categories such as "Rejected"
use finer categories, such as "Waiting for More
could assign uncertainty measures
beyond the scope of
is
sense.
not clear what
it is
ordinal ranking
arguments. However,
criteria
consists of
to the Positions.
be defined on the gIBIS structure.
The
If
20
it
will
accommodate
the changes.
among people because
It
is
more
It
will
make
difficult for
machine. Lack of explicit
when
goals change, there
difficult to
the criteria they use in their
be
isolate the real
arguments remain
will see in the next section, explicit representation of goals
implicit.
can provide modular
representation of arguments, multiple viewpoints, and a basis for relevance matching.
In glBlS, the unit of the issue space
is
describing the relations about issues.
and Suggest another
window commands
Issue.
Issue,
Issue can Generalize, Specialize, Replace, Question,
Issue can also Question a Position or
among
we
command pop-up menu should
should have small implementation as a goal.
be
An
an Argument; an Issue can be Suggested-by a Position or by
These relations are quite important
the issues.
for
Figure 4 shows that the original Issue about the placement of
suggests other Issues: whether the
considered as an alternative and whether
an Argument.
and gIBIS provides several constructs
an
An
in
describing a more global relationship
In particular, as an issue or a decision often gets reformulated
and
Comparative Analysis of Design Rationale Representations
differentiated, the relations such as Replace
however,
if
we
can
somehow show whether
and
seem
Specialize
a given set of relations
essential.
is
21
It
would be
nice,
complete or adequate.
Toulmin's Model of Argumentation
4.2.
Stephen Toulmin, a philosopher, proposed
many
has been adopted for
[Bimbaum
et al. 1980;
model
a
of
argument
in 1958.
Since then, the model
purposes, including the computational representation of arguments
Lowe
1986; Marshall 1987; Streitz et
al.
Figure 6 shows an
1989].
example Toulmin representation of an argument.^
CLAIM
Placing window
commands in the
global menu reduces
commands
need not be
usually
duplicated for
X
each window
screen clutter.
unless
No duplication
saves space.
tlie
global
containing
menu
all
possible
commands becomes
5
on-account-of
Common
so big
Sense
Knowledge
Figure
A
Claim
menu
is
the
5.
An example of an argument in Toulmin's
window commands
in the global
support the Claim by producing a Datum,
"Commands
main assertion being made
reduces the screen
need not be duplicated
clutter.
for
by producing a Warrant,
")
We
(e.g.,
each window."^^
e.g. "If
representation.
"Placing the
We
justify the leap
you need not duplicate
from the Datum
things, that saves space."
usually an inference rule or a principle that licenses the inference from
Warrant
is
not often absolute; so
Datum
inference from the
We
will use the
interested in the
if
to the
we want
Claim,
to
we
express
how
Datum
to the
A
Claim
Warrant
is
A
to Claim.
strongly this Warrant supports the
use a Qualifier.
In
our example, the Qualifier
term Toulmin's representation rather than Toulmin's model because
to the extent it has been or can be used as a representation.
we
is
are
model
As pointed out in [Newman & Marshall 90], Toulmin wants the datum to be a singular fact, such as
something that you can point to, because we think he still wants to retain the syllogistic form, where
one gets the conclusion by having as the minor premise, a singular fact. So he seems shll under the
bound of syllogism, at least at the time of his writing, despite his rebellion against it. The datum of our
example is a more general statement. But we use this example first because the distinction itself
between a singular fact and a universal statement has been called into question in philosophy and
because in design it would be unusual to support a claim always with a singular fact.
Comparative Analysis of Design Rationale Representations
which
"usually",
have
sum
be larger than the
to
question, then
we
appeal
In
through Rebuttal
fails
i.e.,
a
our example,
body
we
of
of the
called the
argument space. The goal
There
supported.
how
Toulmin structure
is
no attempt
they are related. As such,
module
elements of design rationale.
Toulmin was
that
of
facts,
tell
why
us
called into
is
we
in general
or legal precedents.
Toulmin structure
is
to
make
limited to
is
how
explicit
what we
a claim
is
any evaluation measure, or
to represent alternatives, goals,
we
be shared might
to
a Warrant
as a design rationale representation
evaluate Toulmin's representation only as a representation
argument space, which might
for the
knowledge
If
')
appeal to our commonsense knowledge. Other
types of backing might be physical laws, historical
The scope
indicate the excepting
"The object
(e.g.,
of these objects locally existing.
to a Backing,
should accept the warrant.
We
the strength of the inference.
qualifies
circumstances where the Warrant
22
Even
to delineate the logical
then,
we
interface with representations for the other
should keep
in
mind
that the original goal of
support structure of an argument, not necessarily
provide
to
a representation to be used dynamically for capturing design rationale.
As
argument space of design
a representation for the
we
For example,
menu
that the global
possible
only
command
way
to
do
so
is
(let
us assume that this
is
problem by
It is
much
all
the
of denying a Claim;
Even with such
explicitly
to
menus were
menu
items" as the
as supporting
and
in
construct, such
naming
any
"A does not reduce
representation
[Bimbaum et al.
people
corresponding construct.
Claim
itself is
We
B",
is
it
would
would slow down
1980]
Datum and Claim
have solved
and
as Supports
So Toulmin's representation should be extended
at
claims.
we want
to qualify a
We can
qualify a Warrant
Claim or a Datum, there
no
is
could of course build into the Claim any qualification so that the
of the form, "Global
not too big," but then
if
and
Claim.
without some construct that
also not easy to qualify a claim using Toulmin's representation.
using Qualifier and Rebuttal, but
for this
other, the denial relationship
awkward
the original link between
accommodate denying
case,
Datum
all
The
invented).
new
introducing another link called Attacks.
least this
deny
to
has to contain
clutter"
has to contain
human comprehension or machine computation. Some
this
limitations.
menu
two Claims are the negation of each
all.
menu
many
you want
by creating a Claim "the global menu does not reduce the screen
awkward way
indicates that the
negation. Suppose
before pull-down
Whether denying "A reduces B" means the same
not be explicit at
its
reduces the screen clutter because the global
items
presents "the global
certainly an
rationale, Toulmin's has
can only deny a Claim by supporting
we might
menu
reduces screen clutter provided that the global
menu
as well use a natural language as our representation.
is
The
Comparative Analysis of Design Rationale Representations
purpose of making these components distinct
to
is
23
allow us to see the relations more clearly, to
formally manipulate these components and relations, and to incrementally construct pieces of
the argument.
Toulmin's representation also suffers from building into object types context-dependent
such as warrant or backing. The difference between warrant and backing
the object
itself
but only in the
fact that
backing
is
its
we
context-dependent role
[Marshall 1987; Streitz et
in
some other way,
we show
as
Most
if
we want
to
al.
1989;
Newman &
support a Datum
change
So
context.
a separate type called Backing.
its
role to Claim, instantiate the
now
without Backing) schema, and use a Datum
about Warrant that
we need
we
first
change
fill it in.
we want
If
to
However, then
(that
back the Claim?
a separate type for
its
latter
is,
It is
the
it is
to Claim,
not clear
why
why
not
Toulmin structure
not clear what
when we do
support
For
approach.
from Datum
role
its
support or deny a Warrant,
four-components
to
an object type should
in Section 5, or
of the Toulmin-based representations
Marshall 1990] take the
object,
then instantiate the five-component schema, and
we need
Datum, depending on the
assign a type to an object based on the inherent properties of the object and indicate
dynamically change depending on the context.
example,
roles,
not in the nature of
something that supports warrant. The same
object can be a Warrant or a Backing or even Claim or
either
is
is
special
not need a separate
type to support other things such as a Datum.
We
understand that Toulmin's reason
for introducing Backing as a separate type is to
category for the kind of knowledge that
common
sense knowledge, or physical laws.
must be something
that
it is
is
provide a
rarely called into question, such as the written law,
We agree that for an argument to be resolved, there
that the involved parties can agree
and appeal
However, we believe
to.
misleading to represent such knowledge as a distinct component of a rigid structure
such as Toulmin's. For one thing, the five-component schema leads people
argument has
to
make
explicit all these
components
to
be complete.
people to unnecessarily represent the obvious; for example
many
to believe that
warrants,
when
explicitly
represented, are often as obvious as the one in our example, "No duplication saves space".
often in the process, people are led to argue about whether something
rather than about the substance of the argument.
leads people to believe that
having
five
components
is
if
Warrants, or chain of Warrants before
it is
five
we
Also
a warrant or backing,
components,
because there might have
better to define the representation in
is
Conversely, the closed nature of the schema
an argument has these
arbitrary,
an
This belief often forces
get to something that
an open-ended
way
to
we
it
is
complete.
But,
be multiple Data, multiple
agree on.
that allows as
We
many
believe that
objects to be
created as needed without necessarily specifying the boundary of an argument and without
Comparative Analysis of Design Rationale Representations
requiring the obvious to be represented unless
it
needs
to be, e.g., unless
it is
24
called into question
or argued about.
There are numerous other limitations with Toulmin's representation, as
[Newman &
Marshall
1990] point out from their experiences of using Toulmin's for representing legal arguments. For
example, they had
to
make
Rebuttal have at least four additional meanings
what
different types of objections, each corresponding to
is
being objected
these different types of objections should explicitly indicate
can do so
in a graceful
way, as
we hope
to
show
what they
to.
to represent the
We
object to,
believe that
and
that
we
in Section 5.
QOC (Question, Option, and Criteria)
4.3.
QOC
is
rationale.
a representation proposed by
Design rationale
in
QOC
is
[MacLean
et al. 1989, 1991] for "constructing"
a co-product of design that has to be constructed alongside the artifact
makes
QOC
different from, say gIBIS,
whose goal
DATA:
commands not applicable
to current window would
be shown and distracting
DATA
I
In global
al
menubar
lop of screen
On
each
window
Positive evaluation
Negative evaluation
itself.
is
This emphasis
at least includes capturing the rationale as
unfolds.
I
design
said to be not a record of the design process, but ir\stead
it
Comparative Analysis
The major constructs
this paper.
QOC
is
of
QOC are straightforward
The
a Question.
and map
unit of the alternative space
to Issues
and Positions
the criteria space with Criteria.
"bridging criteria"
(e.g.
Design Rationale Representations
clearly to the
25
framework proposed
in
Figure 6 shows an example represented in QOC.^^ The unit of the issue space in
roughly correspond
one
of
Easy
a
if it is
The
to use).
corresponding to
more
A
Criteria (e.g.
specific
one
Questions and Options
an Option.
is
However, unlike gIBIS,
in gIBIS.
QOC
Reduce Screen Clutter)
that derives
its
can represent
said to be a
is
from a more general
justification
units of evaluation space are links labelled with "+"
whether an option does or does not achieve a given
representing the argument space are Data, Theory,
"-,"
and
The constructs
criterion.
One supports
and Mini-Theory.
for
the
evaluation ("+" or "-") of an option with respect to a criterion by appealing to empirical Data
"The mouse
(e.g.
there
is
is
a Fitt's
no relevant data
Law
at
Device") or to an accepted Theory
hand or
construct a Mini-Theory, which
[MacLean
existing theory to
is
draw
(e.g. "Fitt's
Law").
on, the designers
When
may have
to
an approximate explanation of part of the domain.
provide an illuminating discussion of the other forms of justifications for
et al. 1991]
design, such as various forms of dependencies and metaphors, no specific constructs are discussed
for representing them.
QOC as we understand
argument
it
has a
number
of limitations as a representation language. First, in the
space, constructs like Data, Theory, or
aspects of arguments.
For example,
connmands can be dimmed" should be
nor a theory. Nor
in theories.
is it
clear
it
is
Mini-Theory
not clear
treated given that
whether and how
In the alternative space, there
is
we can
seems
to treated as a fixed type, rather
ir\flexibility.
but
is
That
it is
it.
dependency, but no
In the criteria space. Bridging Criterion
role into a
being a Bridging Criterion
is
fixed type results in unnecessary
not a property inherent in the object
a relation that the object has to another criterion.
has with another criterion.
many
as "Irrelevant
neither a piece of empirical data
Otherwise,
we have
itself
As such, one should not have
classify a given object as a Bridging Criterion but instead indicate
it
capture
than a role that a criterion plays with respect to
However, building a
is,
to
argue about theories, or individual claims
a reference to cross-option
specific constructs are discussed for representing
another criterion.
do not seem
how an argument such
its
role
to unnecessarily
to
through the relation
change object types
depending on which object we focus on.
The
to
1^
QOC
constructs for the evaluation space are "+" and "-" links.
These evaluations are said
be supported by appealing to empirical data and accepted theories.
We
could represent only a part of our example because
rest in
QOC.
it
was not
It is
clear to us
not clear, however.
how
to
represent the
Comparative Analysis
how we
of
Design Rationale Representations
can differentiate those arguments that support the overall evaluation from those that
deny or qualify
Suppose
it.
"Commands can be found
that the empirical Data,
positions, "produces the positive evaluation of the Option,
"The global menubar
screen," with respect to the Criterion "Can easily access commands."'
there
is
another argument
same
respect to the
create
26
criterion.
two evaluation
The
is.
links,
is
mouse
Suppose, however, that
option with
travel.") against the
can represent the design rationale in one of two ways:
one labelled + and the other
-,
(1)
between the same Option/Criterion
how
option has the disadvantage of not differentiating explicitly
latter
each datum contributes
evaluation link
We
requires longer
only one link labelled with whatever the net effect of these pro and con
pair, or (2) create
arguments
(e.g. "It
in fixed
at top of the
to the net
evaluation since the link between any Data
The
not labeled (such as with "Supports" or "Objects to").
keeping two evaluation links avoids
problem, but
this
we
first
and
QOC
are not sure whether
the
option of
allows
multiple evaluation links from one Criterion to one Option.
Other Representations
4.4.
In this section,
otherwise,
we
discuss a few other representations, explicitly for design rationale or
whose scope
includes the aspects
PHI (Procedural Hierarchy
we
discussed above.
of Issues) [McCall 1987] overcomes
gIBIS by allowing a quasi-hierarchical structure
semantics of the hierarchical relation
an Issue, A,
is
resolving B.
In the
the
more
first is a
(i.e.
issues,
if
A
"serves" B
second
~
for
it
a child
sharing nodes
(i.e.
node
Hence, unlike
of the
in gIBIS,
is
is,
Answer
In the issue space,
if
is
A
resolving
helps
a child of another
a child of another
can respond
Argument. Furthermore, PHI
argument space
On
the other hand, the expressiveness of
is
extended over that
is
to
if
the
if
a
is
first is
a
an Argument directly by
i/uasi-hierarchical
and allows
in gIBIS.
PHI
is
limited in the following aspects.
to express the specialization of issues in
specialization,
we
that
multiple parents) and cyclic structures. This way, the expressive power for
the
how
of the limitations with
example, a mechanical system
subanswer of a system. In the argument space, an Argument
making
--
the alternative space), an
specific version of the
response to the second.
many
answers and arguments. The
different for the different spaces.
is
a child of another Issue, B,
answer space
among
namely the
By the same token,
it
is
PHI because
hierarchical relation, has been
not clear
how
preempted by the serve
to represent the serve relation in the
because the hierarchical relation in the answer space
is
It is
not clear
the natural representation of
relation.
answer space
the specialization relation. That
is.
Comparative Analysis of Design Rationale Representations
how do you
(e.g.
Use
represent an Answer that serves another Anszver in the sense that adopting the
a large screen) will facilitate adopting the second (e.g.
also inherits
many
make
claims, for example, that
A
[Fischer et
components,
e.g.
1989]
al.
CRACK,
the criteria explicit,
Supports
is
B.
an attempt
interesting as
argument
to
make
the criteria space explicit.
domain
about kitchen design. The other component, ViewPoints, uses PHI
make.
JANUS
integrates the
cannot
relational
One
bridge two representations.
uses a rule-based language for representing
for the decisions they
we
space:
and cannot argue about the
Again, PHI does not
first
Reduce Screen Clutter)? PHI
of the limitations of IBIS with regard to the
qualify a claim, cannot
JANUS
27
specific
of
its
knowledge,
to represent the rationale
two representations by finding the
appropriate rationales represented in PHI for the particular issue that designers face in the
construction phase, that
while using
is
CRACK. Although
the current interface
is
limited to
that of locating the relevant parts of the representations, bridging a design rationale
representation and a
domain representation
is
a very important topic of research because such a
bridge can allow us to represent the relations
domain
specific
&
[Potts
Fig. 7.
among
the alternatives or the criteria in
Bruns 1988] outlines a generic model
The model extends
model
the IBIS
One
design including a software design.
formatter, associates with
it
for representing
design deliberations, shown
to represent the derivation history of
starts
and so on
until
we make
each Alternative
may
be a
is
the plan concrete
justification
essentially an IBIS structure,
representation as well.
Artifacts,
and
making
better.
logical relationship
rationale,
explicit,
it
The
enough
the plan
more
On
among
its
path, this
to
to a
of
what we
more concrete
be implemented. Associated with
showm
in Fig. 7,
said about IBIS applies to this
series of progressively
more concrete
model represents the alternative space and
the other hand, this representation
the alternatives.
in
artifact
concrete, associates
internal structure of rationale, as
much
However, by describing a
Justifications for
argument space
and
an
with an abstract Artifact, such as a plan for a
the Issues that arise in
with each of the Issues the Alternatives considered, some of which lead
plan,
more
ways.
still
its
does not represent the
Relying on the IBIS structure for representing
its
shares the problems of IBIS such as: the criteria used for justification are not
an argument cannot respond
to another,
and we cannot argue about
relational claims.
Comparative Analysis of Design Rationale Representations
J
28
Comparative Analysis of Design Rationale Representations
however,
represent argument structures typically found in academic papers and to serve as
to
is
an underlying structure
arguments
in
for aiding "reasoned discourse
ARL
ARL would
different structures in different contexts?
may
questions
5.
we
especially writing.
The structure
of the
from the structure of the arguments
not be a good design rationale representation language.
raises interesting questions
grained are the distinctions
",
sufficiently different
does not provide any construct
ARL
However,
seems
in design deliberation
academic papers that
Also,
29
for representing goals, alternatives, or evaluations.
about the domain of arguments:
Or
are willing to
Do arguments have
how
are the differences only a matter of
make
fine-
Studying these
in the different contexts?
help us better understand the nature of arguments in general.
DRL (DECISION REPRESENTATION LANGUAGE)
5.1.
Introduction
DRL
[Lee 1990a]
a
is
language that
we have developed
for representing
and managing the
qualitative elements of decision making: for example, the alternatives being considered, their
current evaluations, the arguments responsible for producing these evaluations, and the criteria
We
used for the evaluations.
elements, and
environment
we
is
the representation of these qualitative
management system a system that provides an
and computational services using
a subset of design rationale in that
aspects of design rationale.
how
rationale
for capturing decision rationale
rationale in our sense
about
call decision rationale
call a decision
it
For example, a design rationale
to generate the alternative designs.
The scope
it.
Decision
does not represent some important
of
may
DRL,
include the deliberations
at least for
now, does not
include the representation of such deliberations. ^^ jhe exact relation between decision
and design
rationale
that
DRL
and
that
is
it
rationale,
however, has yet
overcomes many of the limitations
simple enough for the user.
In
this
representation language, and point out
^2
to
be articulated.
Nevertheless,
we
believe
the most expressive language that has been used for representing design rationale
its
in the existing
languages in a
DRL
section,
we
evaluate
strengths
and
limitations as such.
way
that
is still
as a design rationale
does not help us generate alternative designs. It does in a couple of ways.
argue about the existing alternative designs, thus helping them to see more
clearly their strengths and the limitations. It also helps people retrieve the past decisions that
contain useful alternative designs which are still useful for the current decision or those that can be
That
is
DRL
allows people
not to say that
so adapted.
the
new
DRL
to
Furthermore,
alternative that
design alternative
is still
DRL
can represent the relationship between the existing alternative and
derived from it. However, being able to generate a new
how it was generated
may have been
different from representing the rationales for
Comparative Analysis of Design Rationale Representations
5.2.
30
DRL
Description of
Figure 8 shows the objects and the relations that form the vocabulary of DRL. The argument
types for a relation are
DRL
Relations, for
shown
inside the parenthesis following the
between them graphically.
relations
we
example, are subclasses of Claim, as
represented in DRL.
We
is
not a description of the
of the relation.
All
Figure 9 shows the
Figure 10 shows an example decision rationale
DRL
describe the basic features of
briefly,
they allow us to represent the five spaces of design rationale.
following
name
explain below.
way
that the user
and discuss
One should
would use DRL. The
in detail
how
note that the
actual interface
is
briefly described later.
A
qualification
is
in order before
we
proceed further.
DRL
is
an extensible language, and
have a developed a method, called Dependency Analysis, that can be used
add additional constructs
below are what
we
given set of tasks. Therefore, the constructs of
would be
For example,
if
specialized to
would be important
to
to
more
among
presented
dependency analysis method
how
dependency
the requirements
goals such as Replaces, Elaborates, and Merges
represent explicitly. Unfortunately,
we
cannot discuss here, for want of
that allows us to identify
provide and specify their semantics precisely.
snnn
task-specific constructs using
a task involves looking at the history of
evolved over time, then the relations
space, the
DRL
believe are essential for the generic task of decision making. For a given set
of tasks, these constructs
analysis.
for a
we
to systematically
We
hope
which additional constructs
to describe
it
in
another paper
13
Very
briefly,
however, dependency analysis works as follows. Given a task, we generate all of its
to form a task hierarchy. From the leaves of the hierarchy, we identify the
subtasks recursively
objects
and the
attributes that
generate a set of tables.
change
correspond
relation.
to a relation,
we
need.
For
all
possible pairs of the attributes,
we
can systematically
Each of these tables corresponds to the influence that a particular type of
can exert on the other attribute. A subset of the cells in a table
and the type of change signified by the cells in the set gives semantics to the
in the first attribute
Comparative Analysis of Design Rationale Representations
Decision Proble
Achieves (altemarive, goal)
Supports (claim, claim)
Denies (claim, claim)
Presupposes (claim, claim)
Is
a Subgoal of (goal, goal)
Is
a
Good
Alternative for
Subdecision of (dec. prob., dec. prob.)
Suggests
(object, object)
Comments
Answers
(claim, object)
(claim, question)
k
an Ansvi-ermg Procedure for (procedure, question)
a Subprocedure of (procedure, procedure)
Is
a Result of (claim,
Is
Stronger Than (object, object, attribute)
Is
[
Is a
(alternative, dec. prob.)
procedure)
Question
\ Croup
Wiewpoint
(procedure
\
Procedure Description
Figure
8.
Is Equal To (object, object, attribute)
Tradeoff (object, object, attribute)
k
Executable Procedure
The
a
DRL
Kind of
(object, object)
vocabulary.
^o^
is
a subgoal of
I
denies
I
Claim K^^^
is
a
good
alternative for
supports
y
|aaim[
Figure
9.
The structure
of a
DRL decision graph.
I
laaiml
Comparative Analysis of Design Rationale Representations
An example
Figure 10.
A
Decision Problem represents the
the
problem
window commands. An Alternative
at the top of the screen."
alternatives: e.g.
A
"Minimize screen
clutter".
We
menu
is
A
clutter".
elaborated to
Every relation
in
Goal
where
to place
menu
DRL
in Fig.
is
elaborated further in terms of
is
mean "Can
easily access
command
a subclass of Claim, as
shown
its
items"
and
in Fig. 8.
For
10 represents the Claim that the Alternative,
at the top of the screen," achieves the Goal
"Reduce screen
clutter."
evaluate an Alternative with respect to a Goal by arguing about the Achieves relation
between the Alternative and the Goal,
We
that requires a decision; for example,
represents an option being considered: e.g. "global
example, the rightmost Achieves link
"the global
decision graph.
Coal represents a desirable state or property used for comparing the
subgoals: e.g. "Easy to use"
"Minimize screen
32
i.e.,
the claim that the Alternative achieves the Goal.
argue about a Claim by producing other Claims that Support or Deny the Claim or by
qualifying the Claim by pointing out the Claims that
it
Presupposes.
Each Claim has the
following attributes: "Evaluation," "Plausibility," and "Degree." The evaluation of a Claim,
Comparative Analysis of Design Rationale Representations
represented by the value of
and "Degree"
true,
"Evaluation" attribute,
its
The
attribute values.
and the "Degree" of
a Claim tells us to v^hat extent
the goal
true.
and the Goal
tells
The
overall
us
tells
it is
question.
in
a function of both of
"Plausibility" of a Claim
the Achieves link between the Alternative
alternative achieves
is
33
how
its
"Plausibility"
probable the claim
is
For example, the "Degree" of
what degree
us to
that the
evaluation of an alternative
is
represented by the "Degree" attribute value of the Is-a-Good-Alternative-for link between the
Alternative
and the Decision Problem,
resolution for the issue.
This degree
i.e.,
is
Not
the Alternative to the different Goals.
evaluation.
For example,
we might
the claim that the alternative
is
good alternative
a
a function of the degrees of the Achieves claims that link
all
of the three attributes have to be used for the
require that a Claim be entered only
if its
"Plausibility"
above a certain threshold, and ignore the "Plausibility" once the Claim has been entered. In
case,
we
attributes
is
that
can do away with the "Plausibility" attribute, and the "Evaluation" and the "Degree"
become synonymous.
There are other auxiliary objects
A
DRL.
in
Group object groups a number of objects and has the
attribute,
"Member
Group of
objects rather than a single object.
which
Relations,"
other Goals through a Is-a-Subgoal-of
tells
link.
how
us
the objects are related.
For example, a Goal
The other
objects in
may
DRL
A
relation can take a
be related to a Group of
such as Question, Procedure,
and Viewpoint represent somewhat auxiliary aspects of decision making such as the questions
raised
and the procedures used
for
answering the questions.
We
more
discuss
details of
DRL
[Lee 1990a].
DRL
al.
has been implemented in a system called SIBYL, which runs on top of Object Lens [Lai et
1988].
Although the above description of
provided by SIBYL for using
DRL
is
DRL may seem
quite simple
decision tasks such as designing a workplace layout.
create objects like a Decision Problem, Goals,
menus and template
are specified,
cell of
object.
editors.
Once
SIBYL displays them
the matrix, the user gets the
a Decision Problem
of
all
and some of
shown
such as creating a
new
Goals
its
in Fig. 11.
it
group
easy
to
and Alternatives
By mouse-clicking on a
the actions that can be performed
For example, by mouse-clicking on a Goal, the user can get a
other Goals.
for real
For example, SIBYL makes
and Alternatives by providing context-sensitive
in a matrix such as
menu
complex, the actual user interface
and SIBYL has been used
menu
subgoal, or displaying a goal tree showing
on the
selected
containing action items
how
this
Goal
is
related to
in
Comparative Analysis of Design Rationale Representations
Close
Show
Goal Lalike
34
Comparative Analysis of Design Rationale Representations
provides for using
DRL
described in more detail in [Lee 1990b]. Using the decision rationale
is
represented in DRL, the computer can provide
among
claims, propagating
many
and merging the
al.
We
providing multiple
discuss these services in
1988].
Evaluation of
5.3.
managing the dependencies
services such as
plausibilities automatically,
viewpoints, and retrieving useful knowledge from past decisions.
[Lai et
35
DRL as
a
Design Rationale Language
The Argument Space
An argument
people might
which
is
represented in
call facts,
DRL
sometimes arbitrary and
is
A
as a set of related Claims.
assumptions, statements, or
difficult
"Plausibility," that indicates
how much
advantage of not imposing a
set of
to
make,
confidence
Claim subsumes what other
Instead of
rules.
we have
if
we need
to
make
these distinctions,
has the attribute,
in the claim.
This has the
predetermined categories on the user, and avoiding the
ambiguity resulting from the disagreement among people on what
Also,
making
DRL Claim
a
the distinction, say,
between
simply or by specializing a claim or by using nominal categories
as values for the "Plausibility" attribute in different Claims.
facts or
assumptions
We
like "facts"
and "assumptions"
can
do so post
in fact
facto or
dynamically by using a numeric measure as the plausibility value, and mapping between
measure and the measure based on the norrunal categories such
when
discuss different plausibility measures
A
are.
and assumptions, we can do so
facts
as facts or assumptions.
this
We
discussing the evaluation space.
Claim can be Supported, Denied, or Presupposed by another Claim.
These relations among
the Claims allow us to respond to a Claim directly without, as in IBIS, having to indirectly
respond
to the Position that
responds
commands can be dimmed,"
current
window would be shown and
Claim for the Alternative
the logical
and the dynamic
relation,
we
are
making
Of,
is
a Claim,
commands
distracting," rather than
structure of the
For example,
argument more
when we
the claim that Claim
from an Alternative
achieves the goal.
second Claim. For example, the Claim, "Irrelevant
having
not applicable to the
to
be formulated as a
These direct relations among the Claims allow us
in question.
special types of Claims.
relation
to the
directly denies the Claim "The
link a
1
Claim
1
easily.
to
All
DRL
to see
relations are
Claim 2 through a Supports
supports Claim
2.
Likewise, an Achieves
object to a Coal object represents the claim that the alternative
Hence, any
DRL
relation, like Supports, Denies, Achieves, Is-A-Subgoal-
and can be argued about;
i.e.
people can support, deny, or qualify them.
For
Comparative Analysis of Design Rationale Representations
example,
"Commands
not applicable
to the current
window would be shown and
denied by "Irrelevant commands can be dimmed". That the
itself is a relational
by
their
Claim, which
is
36
then denied by
first
Claim
is
'Dimmed commands
are
Unlike Toulmin's argument structure, the roles of a Claim are represented by
its
Datum
types such as to
equivalent of the Toulmin schema. Qualifier
is
still
distracting
its
relations to
is
not
shown because
it is
ambiguous what
Figure 12 shows the
or to Warrant.
represented as the value of the
is
"Plausibility" attribute of the Supports relational claim
Rebuttal
"
mere presence."
other Claims, not by changing
DRL
distracting.
denied by the second
between the two Claims
a rebuttal
is
at the top.
denying. This representation of
roles in terms of relations allows us to see the global picture of the
argument structure without
being confined to the micro-structure defined by the Toulmin schema and to expand the structure
where and when we need
deny rather than having
to
We can also
to.
deny exactly and
directly the thing that
ambiguously overload the meaning of
we want
to
rebuttal.
Supports
Supports
/
Supports
Claim
Figure 12.
The Toulmin model
The
Criteria Space
DRL
represents the criteria space fairly well.
In
DRL,
in
DRL.
criteria are
we
corresponding goal, namely the goal of achieving the criterion, and because
the richer relationship
among
A
can always define a
we want
to
convey
these goals than what the term criteria usually conveys.
example, a Coal Is-a-Subgoal-of another Coal
achievement of the second.
set of
if
achieving the
subgoals can be related
first
Goal
among themselves
by creating
a
Group
object
For
facilitates
in various
they can be mutually exclusive, independent of each other, or partially overlapping.
relationships are represented
DRL
represented by Goals.
uses the term "Goal" rather than "Criterion" because for each criterion,
and specifying these Goals
to
the
ways;
These
be
its
Comparative Analysis of Design Rationale Representations
members; the
the Group, as
relations
shown
among
these Goals are specified in the
"Member
37
Relations" property of
in Fig. 13.
is-a-subgoal-of
Members:
Member Relations!"
conjunctive
partially specifying
Figure 13.
A Gro»p of Subgoals
in
DRL
Decision Problem represents the goal of choosing the best alternative.
the decision
it
means
to
problem are subgoals of the decision problem
in the
choose the best alternative. For example, the Goal "Easy
Decision Problem of our
example
the property 'Easy to use."'
if
we
In other
interpret
it
to use" is a
mean "Choose
to
words, satisfying
All the other goals for
sense that they elaborate what
subgoal of the
the alternative that has
this goal facilitates the
achievement of
the goal of choosing the best alternative.' '^
Since the Is-a-Subgoal-of relation
whether
about
a goal
is
this relational claim.
should be a goal at
going
to
is
a Claim, as
desirable or whether
For example,
In Fig. 10, there
all.
it
is
is
we
small implementation
is
relation,
we
can argue about
it is
"It
should not be a goal because
Mac
a subgoal of the decision problem,
is
i.e.,
not a desirable property that should be used to compare alternatives.
of these Claims
Elsewhere [Lai
DRL
can argue about whether small implementation
an argument,
have large memory soon," denying that
The record
any other
contributes to achieving another goal by arguing
and
et al. 1988],
create multiple viewpoints,
their relations represents the
we
and
discuss
extract
how
argument space
this representation of
Goals in
for the goal space.
DRL
from past decisions knowledge useful
allows us to
to the current
decision.
of the model underlying DRL is more complicated and fully explained in [Lee,
Roughly, for a given decision problem of the form, what is the best alternative for X, its
underlying interpretation is 'the goal of choosing the best alternative for X". hs subgoal of the form,
G, is strictly speaking "the goal of choosing the alternative that satisfy G". An alternative of the form,
A (e.g. the global menubar at top of the screen), is to be interpreted as "Choosing the alternative A".
It is in this sense that a decision problem is the parent of the other goals and that an alternative
achieves a goal. This nicety, though important for computational purposes, can be ignored by
The precise semantics
1991a].
human
users.
Comparative Analysis of Design Rationale Representations
38
The Alternative Space
DRL
"Commands
that
DRL
represents only parts of the alternative space well.
the specialization relation
among them through
at the top of the
However, design
window".
through the specialization
window"
alternatives
An
relation.
can represent alternatives and
Thus, v^e
the Is-a-Kind-of relation.
a special case of the alternative,
is
may be
much more complex ways
related in
space. These relations are
more complex way,
DRL
example,
of
represents the arguments about the alternative space the
alternative at
alternative,
all
We
is
or whether an alternative
an alternative only
if
""the
is
commands
same way
at the top of the
represents the
it
The
version of another
really a specialized
relations,
relations can be also qualified
For example,
window system
main window" by linking the two claims with
object to this Claim, in turn,
design
DRL.
can argue about whether an alternative should be an
another Claim via a Presupposes relation.
window"
than
Next-Version-of.
in the context of a
by creating Claims that deny or support the appropriate
Good-Alternative-for or Is-a-Kind-of.
to the
for
beyond the current expressive power
arguments about the goal space.
say
Alternative can be related to another Alternative via,
example, the following relations: Elaborates, Simplifies, or Is-the
for
Alternatives can be related in a
Ccin
"On each
we
such as
Is-a-
by linking them
to
can say that "commands on each
allows the attachment of
a Presupposes relation.
menu windows
One can
of course
by pointing out another way of implementing the window
window.
The Evaluation Space
In
DRL, each Claim has the
"Evaluation" attribute
"Plausibility"
The
(how
tells
us
how
likely the claim
overall evaluation of
important the claim
is
true)
an alternative
is
is,
and "Degree"
and
(to
its
value
is
The
a function of both
what extent the claim
is
true).
represented as the "Evaluation" attribute value of
the relational claim, Is-a-Good-Alternative-for,
Problem.
and "Degree."
attributes: "Evaluation," "Plausibility,"
between the Alternative and the Decision
This value represents the extent to which the alternative satisfies the overall goal.
This value, in turn,
is
a function of the evaluations of the Achieves relations that link the
Alternative to the subgoals of the Decision Problem
satisfies the subgoals).
It is
goal, such as the extent to
also a function of
how
(i.e.,
the extent to
which the alternative
the subgoals interact to satisfy the parent
which tradeoffs and synergies
exist
among
these goals.
Comparative Analysis of Design Rationale Representations
DRL
does not commit
come with
evaluation measures should
produce evaluation measures
the
measure of
In the global
over the subgoal relations
global
we
menubar
should also
know how
if
and merging them
we want
to
'
...
subgoals are related
menubar
the probability of the
achieves the goal 'Reduce screen clutter'" - combine
...'
achieves the goal 'Easy to use'," given that
among themselves and
an algorithm based on Bayes' theorem,
However, as discussed
for
to the
That
parent goal.
algorithm
the algorithm
is, if
We
we
also
might
'In
know how
try to
the
these
work out such
example.
in Section 2, the exact
user can trust the algorithm.
two Claims -
commands'" and
produce the probability of the Claim "The alternative
to
to
use probability as
the global menubar...' achieves the goal 'Can easily access
'In
"The alternative
For example,
can use nominal
However, such
for evaluation.
the algorithm for propagating
at a higher level.
plausibility, then
"the alternative
DRL
measure of evaluation. The user of
to a particular
numeric measures, or whatever they devise
categories,
39
is
is
important only
based on
to the extent that the
many assumptions
that the
user feels are seriously violated, then the exactness of the algorithm does not contribute much.
We might as well concentrate on how
this
philosophy and
to
tries
to
DRL
takes
explicit
the
support people for making these judgments.
help by modularizing and helping to
make
relationship that need to be considered for these judgments.
The Issue Space
In
DRL, the
unit of the issue space
Issue of gIBIS
and a Question
Decision Problem
if
Problem
is,
e.g. tiling or
the
if
Is
we
a part
a decision problem.
overlapping.
decision problem
first
A
QOC.
A
is
is
A
require deciding
Decision Problem IS-A-KIND-OF another Decision
a special case of
"Where
to place the
"Where
We
are sure that there are
many
Also, the Is-Suggested-By relation in IBIS
relation to represent
on the philosophy
any
is
other possible relations.
method
for
may not be useful in
a
Is
a kind of and
For example, the
dynamic aspect of the issue space.
vague but nevertheless useful as a
not described by others. However, again,
that the vocabulary should
better to provide a
tiiat
relation that
is
to place the
window commands?" Of
can relate the decision problems through the generic relations such as
of.
constructs
For example,
what the window layout
a special case of the second: e.g.,
Replaces relation in IBIS seems important for describing the
it is
Decision Problem corresponds to an
Decision Problem Is-A-SubDecision-Of another
window commands might
emacs window commands?"
course,
is
a decision for the first requires a decision for the second.
deciding where to place the
algorithm
in
be extended
to tailor the task in
catch-all
DRL is
based
hand and
that
extending the vocabulary as needed rather than provide
general.
Comparative Analysis of Design Rationale Representations
40
Relation to Other Studies:
5.4.
In this section,
we
relate
DRL
to the existing representations that
we
discussed in the previous
section. 15
The DRL structure helps us
to
&
Rosson
artifact in the
sense of [Carroll
its
user with the designer's intention or reasons for
1990], the designer's claims captured in the design process are
To
often about the psychological consequences of the artifact.
that extent,
designer's claims explicit help us to identify the psychological design rationale.
we
&
be careful not to confuse the claims about the
psychological consequences of an artifact for
the artifact [Carroll
embodied by an
extract the claims
Although we have
Kellogg 1989].
can extract from the example in
Fig. 10
the following claims
embodied by
making
the
For example,
the artifact of the
Macintosh menubar:
•
The
•
Placing the
irrelevant
commands
in the global
window commands
menubar
in the global
reduce the screen clutter by not duplicating the
•
Longer mouse
travel required
is
not distracting
menubar
facilitates
commands
for
if
dimmed.
ease of use because
each
by placing the window commands
it
would
window
in the global
menu
is
not a
serious detraction from the easy usability.
Some
of these claims
may
be obvious, but even then they are worth stating explicitly because
they can then become subject to tests or further elaboration.
decontextualize claims,
might
all
global
menu
agree that "Longer
is
mouse
travel required
embodied by an
Furthermore, not
all
some
DRL and
artifact,
the claims that
claims in Carroll's sense,
identify
is
at
most small.
DRL
but only
if
try to
in the
the screen size
does not allow us
is
to identify all
only those that come up explicitly in the course of design.
come up
in the
course of design are worth stating as the
and can be viewed as an
Position to Alternative,
DRL
through SIBYL help
tool for extracting these claims.
gIBIS have similar structures at least at a high level.
DRL,
"
the representation of design rationale in
of these claims,
Decision Problem in
when we
by placing the window commands
not a serious detraction from the easy usability
small so that the extra distance to travel
the claims
Especially,
some obvious claims may not be obvious any more. For example, we
and Argument
Issue in gIBIS corresponds to
to Claim.
However, gIBIS has
ArgNoter [Stefik et al. 87] is one of the earliest attempts to capture design rationale. Although DRL is
designed to support many of the goals underlying Argnoter, we do not discuss ArgNoter here
because no specific representation has been proposed for realizing its goals.
Comparative Analysis of Design Rationale Representations
quite limited expressiveness, as
we
argument;
we have
discussed above:
We
cannot represent or argue about the evaluation
the relationship
among
We
alternatives or arguments.
Supports B without disagreeing with
A
nor
DRL
evaluating the alternatives remain implicit.
ways: the explicit representation of the
argument space, provision of the
argument space not only
cannot directly respond
criteria used.
cannot say that
Most notably,
B.
41
to
an
We cannot indicate
we disagree that A
the criteria used in the
arguments
can be viewed as extending gIBIS in several
criteria space,
the richer representation of the
infra-structure for defining evaluation measures,
and the
for the evaluation but also for the alternative
and the
criteria
spaces
as well
The gIBIS
structure has the advantage of being simple,
buys or costs
this simplicity
us.
represent most of the discussion, although
it
among
If
components
the
use the language, then
used
at
However,
all.
of design rationale.
and B but not
they
want
to
A
that
objects to B.
say and yet
From a computational
with them. Also,
we
this
is
If
natural
It is
DRL
what they want
for
people
more expressive
should remember that often
in the context of
structure. But
QOC
perhaps the closest
is
SIBYL
not any
is
an empirical question, and
it is
DRL
to
to say, e.g., that
to use,
we would
structures
it is
not only the language
more cumbersome than
we plan
and Coal
subgoals
maps
to
A
what
do more
to
itself
we
things
but also the
believe that the
the use of the gIBIS
less clear to us.
Decision
map
roughly
to Criterion.
Claims
Mini-Theory depending on whether the claims are empirical statements or
,
parts of an established theory, or an informal theory.
how
agree with
Both have the five spaces
Problem maps
not clear
I
to find out.
at least in its basic structure.
to Question, Alternative to Option,
be
prefer such a language.
would allow us
argument space are
is
to
People can be
usability.
clearly delineated, although the constructs for the
to Data, Theory, or
enough
of the relations
better to be simple than not to
always enhance
user interface that determines usability. Based on our limited experiences,
use of
general
is
many
a language with a finer structure allows people to say
enough
point of view,
an empirical question what
Argument)
simple structure makes people more likely to
a simpler structure does not
frustrated with not being able to say exactly
is
it
does not explicitly reveal
worth the while.
certainly
it is
and
structure {Issue, Position,
Its
argue about these
to the
QOC
concept of Bridging
However, we pointed out
constructs for the argument space.
Criteria.
Also, at least
some
earlier that
The concept
it
of
of the links can be argued
about: e.g. the evaluation link between an option and a criterion can be supported by Data or
Theory.
WTiether
the relation
we
can argue about other
between Data and Theory,
is
links, for
not clear.
example, the relation
among
Criteria or
Comparative Analysis of Design Rationale Representations
It
seems
that
QOC
That
language.
is
is,
currently
more
an attempt
is
it
a
to
model rather than
42
a fully-developed representation
understand and categorize the elements of design
rationale without providing yet a specific set of vocabulary for expressing them.
a
worthy endeavor and consistent with
their
warning against
representation. Given the similarity in the underlying structure,
of course
It is
premature commitment
a
we hope
DRL
that
representation language adequate for representing most of the elements that the
to a
provides a
QOC
research
has been articulating.
many
There are
Answers
in
between PHI and DRL.
parallels
PHI corresponds
among
quasi-hierarchical relation
The quasi-hierarchical
relation
Denies relations in DRL.
But
Issues in
PHI corresponds
among Arguments
many
constructs in
Goals, Presupposes, or Is-a-Subgoal-Of,
further the extensions that
The quasi-hierarchical
to the Is-a-Kind-Of relation in the alternative
PHI made
complex relations and making
PHI
in
DRL
and Is-a-Part-Of
to IBIS
explicit
is
among
DRL. The
in
to the Is-a-Subdecision-of relation.
and
specialized into the Supports
no correspondence
find
.
relation
space
DRL
Thus,
in PHI,
can be viewed as pushing
by generalizing the hierarchical structure
some other elements,
such as
to
more
especially those in the criteria
space.
&
In [Potts
Bruns 1988], IBIS
representation.
used as the representation for the rarionale component
is
The component
modular enough, however,
is
alternate representation for the module.
[Fischer et
VIEWPOINTS. Again, DRL can be used
which we believe
is
more
is
more expressive and can
6.
CONCLUSIONS
A
large
body
can get and
how
The
easily
we
is
its
issue base,
CRACK,
because
DRL
better support icnowledge-base operations.
task of using
choice of representation
in their
can be viewed as an
1989] also uses IBIS for
natural to interface with the other component,
of research in the last
1985].
DRL
as the alternative representation for the module,
two decades or so points
right representation for a given task [Amarel 1968;
Levesque
al.
that
Bobrow
to the
1975;
and reusing design rationale
is
importance of choosing a
Winston 1984; Brachman
no
different.
The
can get them depends heavily on the representation
especially important
as in design rationale capture, because a
task altogether, attributing the failure
when
wrong
a
human
is
we
use.
&
we
The
the user of the representation,
representation can turn people
and frustration
benefits
away from
to the task itself rather
the
than the
inadequacy of the representation used. People might conclude that capturing design rationale
is
not worth the trouble because
it is
so hard and because
it
does not provide enough rewards for
43
Comparative Analysis of Design Rationale Representations
the efforts. But the real problem might be that the representation does not allow us to represent
easily
what we want
way
to represent or in a
we know how
important that
to
much
that can provide
Thus,
benefit.
evaluate a representation for a given task,
is
it
our case,
in
for
capturing design rationale.
In this paper,
we made
a step forward by characterizing the
domain
of design rationale,
by
i.e.,
identifying the kinds of elements that form the rationale as well as the relations that hold
Characterizing this domain
among them.
represent,
what we have decided not
helps us to
map
is
we
important because
to represent,
and what are
know what we can
then
their consequences.
different parts or aggregates of the domain.
In other
words,
it
We
Using the
defined the scope of the existing representations and discussed their adequacy.
we
have also presented
a language, called
DRL, which we believe
most of the existing languages and overcomes many
natural to
human
However,
users.
that
is
a testable claim that
by many
DRL
DRL, which we hope
will be the topics for future research
with
The step we made, however,
many
is
understand the important issues
We
a small one and
in
to represent the
its
We also
users.
we
we have
a long
for
way
to
tractability. We
We also need to
so that
need
much
for the task
other categories:
many
we have
human
to articulate the characteristics that
identify the computational services that
we know what
us.
go before
more
we
fully
in
make
we
usability
tfian
being able
other characteristics,
provide abstractions, that are important [Bobrow 1975].
we mentioned two
is still
discussed the limitations of
We
need
to think
hand, and in what
they matter. Then, there are other dimensions to a representation than
In Section 2,
that
evaluating a design rationale representation along
elements in the domain explicitly or not. There are
ability to
way
plan to investigate
by others as well as by
expressive power. Even then, expressiveness involves
about whether these characteristics matter
way
more expressive than
designing an ideal representation for representing design
provided a framework
one dimension -
such as the
tasks
is
of their limitations in a
empirically by using
rationale.
also
provides a framework for
defining the scope and assessing the expressive adequacy of a representation.
framework,
It
them with the
the different meanings of design rationale by associating
its
expressiveness.
and computational
a representation
more
usable.
can provide with design rationale
they require and any tradeoff between their requirements and other
requirements such as hunnan usability. There are some existing studies which address these
problems [Lee 1989;
Yakemovic
Newman &
1991; Fischer et
al.
Marshall 1990; Yakemovic
1991; Lee 1991b;
focus on the representation being used, and
Shum
1991].
We
&
Conklin 1990; Conkiin
&
need more such studies, more
more systematic categorization
of the results.
We
Comparative Analysis of Design Rationale Representations
believe that the benefits from explicit representation of design rationale
for the efforts that
we
44
would more than pay
put into such studies.
Acknoivledgments. The whole project of articulating the domain arose from our frustration
with not knowing exactly
to
Tom Malone
DRL.
how
to evaluate the existing representations
suggested that
we
which components of a decision matrix
One
analysis here.
great environment in
Jeff
Tom
Conklin,
influenced
which he wrote
much by
the
Mark
MIT AI
This
to relate
them
it
makes
like to
That insight triggered much of the
explicit.
thank Frank Halasz for providing him with the
this paper.
The comments from
anonymous
the
reviewers,
Moran, Randy Trigg, and Austin Henderson were valuable. The paper was
the
Danny Bobrow, Frank
Rogers,
would
of us (Jintae)
and how
define the scope of a representation by thinking about
Stefik,
comments from
Halasz,
Bill
and Norbert
the
members
ARG
of
(Argumentation Reading Group):
Janssen, Cathy Marshal, Susan
Streitz.
We also
Lab, especially Patrick Winston, Rick Lathrop, and
work was supported,
in part,
Newman, Dan
Russell,
Russ
thank the members of the learning group at
Gary Borchardt.
by Digital Equipment Corporation, the National Science
Foundation (Grant Nos. IRI-8805798 and IRI-8903034), and
DARPA
(Contract No. N00014-85-K-
0124).
REFERENCES
Amarel,
&
On
S. (1968).
the representation of Problems of Reasoning about Actions, in Webber, B.
N. Nilsson (Eds.) Readings
Alto,
in Artificial Intelligence.
Tioga Publishing Company: Palo
CA.
Bimbaum,
M. Flowers
L.,
&
R.
McGuire
Towards an AI Model
(1980).
Bobrow, D.
(1975).
Dimensions of Representation,
Representation and
Francisco,
Bobrow, D.
Understanding: Studies
in
in
Bobrow, D.
Cognitive Science.
of Argumentation.
Stanford,
Proceedings of the First National Conference on Artificial Intelligence
&
CA.
A. Collins (Eds.)
Academic
Press:
San
CA.
&
Intelligence
Goldstein (1980). Representing Design Alternatives. Proceedings of
and Simulation of Behavior
Amsterdam, Netherland.
I.
Artificial
R. & H. Levesque (1984).
The Tractability of Subsumption in Frame-Based
Description Languages. Proceedings ofAAAI-84
pp. 34-37. Austin, TX.
Brachman,
Brachman,
R.
Kaufmann
Card,
S., ].
&
H. Levesque, Eds. (1985). Readings
Los Altos, CA.
Publishers, Inc.
D. Mackinlay
Proceedings of
CHI
'90
&
in
Knowledge Representation.
Morgan
:
G. G. Robertson (1990).
117-124. Seattle,
WA.
The Design Space
of Input Devices.
Comparative Analysis of Design Rationale Representations
45
Human Computer Interaction Scenarios as a Design
J. & M. B. Rosson (1990).
Representation. Proceedings of Proc. of HICSS-23: Hawaii International Conf on System
Sciences.
Los Alamitos, CA. IEEE Computer Society Press.
Carroll,
Carroll,
&
J.
M.
Rosson
B.
Deliberated Evolution: Stalking the
(1991).
Space. Human-Computer Interaction
Carroll,
M.
J.
&
R. L.
Human-Computer
Campbell
View Matcher
in
Design
(this issue).
The Case of
Artifacts as Psychological Theories:
(1989).
Interaction. Behaviour and Infonnation Technology
8(4)
247-256.
Artifact as Theory-Nexus: Hermeneutics Meets TheoryJ. M. & W. A. Kellogg (1989).
Based Design. Proceedings of CHI '89 7-14. Austin, TX.
Carroll,
Conklin,
J.
ACM
Conklin,
& M.
L.
Begeman
(1988).
gIBIS:
A
Hypertext Tool
Transactions on Office Information Systems
&
J.
K. C.
Yakemovic
Hutrmn-Computer
Lemke, R. McCall
Human-Computer Interaction
Fischer, G., A.
Fischer, G., R. McCall
&
A.
&
H.
Rittel
(1970).
&
ACM
Malone
&
K.-C.
Exploratory Policy Discussion.
Process-oriented Paradigm for Design Rationale.
Morch
A.
(1991).
Making Argumentation Serve Design.
(this issue).
Morch
Design Environments
(1989).
of
CHI' 89
for Constructive
Yu
of California, Berkeley,
(1988).
and
pp. 269-276. Austin, TX.
Center for Planning
Issues as Elements of Information Systems.
and Development Research Univ.
Lai, K.-Y., T.
for
pp. 303-331.
(this issue).
Argumentative Design. Proceedings
Kunz, W.
A
(1991).
Interaction
6(4)
Object Lens:
Transactions on Office Information Systems
A
6(4)
Working Paper.
"Spreadsheet" for Cooperative Work.
332-353.
Task Embedded Knowledge Acquisition through a Task-Specific Language.
J.
Detroit, MI.
Proceedings of IJCAI Workshop on Knowledge Acquisition
Lee,
(1989).
Lee,
J.
(1990a).
SIBYL:
Shellard (Eds.)
Cambridge,
A
Qualitative Decision
Artificial
Intelligence at
Management System,
MIT: Expanding
in
Winston,
Frontiers,
vol.
1.
P.
H.
MIT
&
S.
Press:
MA.
SIBYL: A Tool for Managing Group Decision Rationale.
J. (1990b).
Computer Supported Cooperative Work
LA, CA. ACM Press.
Lee,
Proceedings of
Lee,
Decision Rationale Management System: Computational Representation and
J. (1991a).
Use of the Reasons for Decisions. Ph.D. Thesis forthcoming. Department of Electrical
Engineering and Computer Science, MIT.
Lee,
J.
(1991b).
Extending the Potts and Bruns model for recording design rationale. Proceedings
Austin, TX.
of 13th International Conference on Software Engineering
Lewis,
Problem-Centered Design for Expressiveness and Facility
J. Rieman & B. Bell (1991).
Graphical Programming System. Human-Computer Interaction (this issue).
C,
in a
Lowe, D.
(1986).
Information.
SYNVIEW: The Design
of a
System
for
Proceedings of Computer Supported Cooperative
Cooperative Structuring of
Austin, TX.
Work
Comparative Analysis of Design Rationale Representations
MacLean,
A., R.
Young, V.
Bellotti
Elements of a Design Rationale
&
46
Moran (1991). Questions, Options, and Criteria:
User Interfaces. Human-Computer Interaction (this
T.
for
issue).
MacLean,
&
M. Young
A., R.
Artifact.
T. P.
Moran
Proceedings of CHI' 89
(1989).
Design Rationale: The Argument behind the
Austin, TX.
Exploring Representation Problems Using Hypertext.
Marshall, C. (1987).
Hypertext '87
Proceedings of
PHIBIS: Procedurally Hierarchical Issue-Based Information Systems.
McCall, R. (1987).
Proceedings of Conference on Planning and Design in Architecture
Society of Mechanical Engineers.
Boston,
McCall, R. (1990). PHIDIAS: A PHI-based Design Environment integrating
Dynamic Hypertext. Proceedings of European Conference on Hyupertext
MA. American
CAD Graphics
into
Versaille, France.
A Semantic Analysis of the Design Space of Input
J., S. Card & G. Robertson (1990).
Devices. Human-Computer Interaction 5(2-3).
McKinlay,
Mostow,
Newman,
Toward
(1985).
J.
S.
&
Better
C. Marshall
&
/4/
Mflgflzme
6(1) pp. 44-57.
Pushing Toulmin Too Far: Learning From an Argument
Xerox PARC, Technical Report.
(1990).
Representation Scheme.
Potts, C.
Models of the Design Process.
G. Bruns (1988).
Recording the Reasons for Design Decisions. Proceedings of 10th
pp. 418 -427.
International Conference on Software Engineering
Shum,
S. (1991).
to
Cognitive Dimensions of Design Rationale, submitted
to
HCr91
Lewis (1988). Computer-Aided Reasoned Discourse or. How
Argue with a Computer, in Guindon, R. (Eds.) Cognitive Science and Its Application to
Smolensky,
P., B.
Fox, R. King
Human-Computer
Streitz, N.,
].
& C.
Interaction, pp. 109-162. Ablex:
Hannemann & M. Thuming
(1989).
Travelling through Activity Spaces.
Toulmin,
Winston,
S. (1958).
P. (1984).
The Uses of Argument.
Artificial Intelligence.
From
Norwood, NJ
Ideas and
.
Arguments
Proceedings of Hypertext '89
Cambridge Univ.
Press:
to
Hyperdocuments:
pp. 343-364.
Cambridge, England.
Prentice Hall: Reading,
MA.
Yakemovic, K. C. B. & J. Conklin (1990). Observations on a Commercial Use of an Issue-Based
LA, CA.
Information System. Proceedings of Computer Supported Cooperative Work
il
JJie
International Center for Research on the
Management of Technology
Sloan School of Management
Massachusetts Institute of Technology
Working Paper
List
Number
Date
1-89
11/89
Author(s)
Title
Netgraphs: A Graphic Representation of Adjacency Tool for Matrices as a
Network Analysis
2-90
Strategic Transformation
3-90
Managing
CAD Systems
and the Success
in
of
High Technology Companies
The Personality and Motivations
Roberts
Robertson
Allen
Mechanical Design Engineering
1/90 (Rev. 3/91)
4-90
George
Allen
of Technological Entrepreneurs
1/90
5-90
Current Status and Future of Structural Panels in the
Wood Products Industry
6-90
Do Nominated Boundary Spanners Become
Effective Technological Gatekeepers?
7/90
-90
The Treble Ladder Revisited:
as They Grow Older?
Why Do
Engineers Lose Interest in the Dual Ladder
8/90
10-90
The Emergence
of Fiber Optics
Professionals:
A
People and Technology Transfer
Exploring the Dynamics of Dual Ladders:
A Longitudinal Study
8/90
12-90
8/90
13-90
8/90
14-90
Managing the Introduction
in a Multi-Plant
of
New Process Technology:
International Differences
Networic
Task Characteristics and Organizational Problem Solving
Process Change
The Impact
of "Sticky Data"
in Technological
on Innovation and Problem-Solving
8/90
15-90
5/90
16-90
7/90
McCormack
Utterback
Organizational Relationships and Advancement of Technical
Ten Year Longitudinal Study in One Organization
Work Environment,
8/90
11-90
Allen
Katz
Technological Discontinuities:
8/90
9-90
Allen
Nochur
6/90 (Rev. 7/91)
7-90
Montrey
Utterback
4/90
Underinvestment and Incompetence as Responses to Radical Irmovation: Evidence
from the Photolithographic Aligriment Equipment Industry
Patterns of
Communication
Among Marketing, Engineering and Manufacturing —
A Comparison Between Two New Product Teams
Working Paper
List (continued)
Number
Date
17-90
Author(s)
Title
Age, Education and the Technical Ladder
Allen
9/90 (Rev. 8/91)
18-90
Katz
A Model of Cooperative R&D Among Competitors
1/90
19-90
4/90
20-90
Strategy, Structure,
the
and Performance
in
Product Development: Observations from
Auto Industry
Sinha
Cusumano
Cusumano
Nobeoka
Organizing the Tasks
in
Complex Design Projects
Eppinger
Whitney
Smith
6/90
Gebala
21-90
The Emergence of a
New Supercomputer Architecture
7/90
22-90
Superceded by 39-91.
23-90
Software Complexity and Software Maintenance Costs
8/90
24-90
Leadership Style and Incentives
Banker
Datar
Kemerer
Zweig
Rotemberg
9/90
25-90
Afuah
Utterback
Saloner
Factory Concepts and Practices in Software Development
Cusumano
11/90
26-90
Going
Public: Sell the Sizzle or the Steak
Roberts
10/90
27-90
11/90
28-90
Evolving Toward Product and Market -Orientation: The Early Years of
Technology-Based Firms
The Technological Base of the
New
Enterprise
Roberts
Roberts
11/90
29-90
Innovation, Competition, and Industry Structure
12/90 (Rev. 6/91)
30-91
Product Strategy and Corporate Success
Roberts
1/91
31-91
Utterback
Suarez
Meyer
Cognitive Complexity and
CAD Systems:
Beyond the Drafting Board Metaphor
1/91
Robertson
Ulrich
Filerman
32-91
CAD System Use and Engineering Performcmce in Mechanical Design
1/91
33-91
6/91
34-91
5/91
Robertson
Allen
Investigating the Effectiveness of Technology-Based Alliances: Patterns and
Consequences of Inter-Firm Cooperation
The High Centrality
of
Boundary Spanners:
Organizational Networks
A Source of Natural Efficiency in
George
Allen
Ancona
George
Robertson
Working Paper
List (continued)
Number
Date
35-91
2/91
Title
Impacts of Supervisory Promotion and Social Location on Subordinate Promotion
RD&E Setting: An Investigation of Dual Ladders
in
an
Allen
(Rev. 11/91)
36-91
Demography and
Design: Predictors of
New Product Team Performance
1/91
37-91
Author(s)
Katz
Tushman
Tlie
Changing Role
of
Teams
Ancona
Caldwell
in Organizations: Strategies for Survival
2/91
38-91
Schrader
Informal Alliances: Information Trading Between Firms
3/91
39-91
3/91
40-91
3/91
41-91
Supplier Relations and Management:
and
U.S.
A Survey of Japanese,
Japanese-Transplant,
Maneuvering and Mass-Market Dynamics: The Triumph
Strategic
of
VHS
Over Beta
The Software
Cusumano
Takeishi
Auto Plants
Factory:
An
Entry for the Encyclopedia of Software Engineering
Cusumano
Mylonadis
Rosenbloom
Cusumano
3/91
42-91
Suarez
Utterback
Dominant Designs and the Survival of Firms
4/91 (Rev. 7/92)
43-91
An Environment for Entrepreneurs
Roberts
6/91
44-91
7/91
45-91
Technology Transfer from Corporate Research
of Perceptions on Technology Adoption
When Speeding
Concepts
to
to Operations: Effects
Nochur
Allen
Market Can Be a Mistake
Utterback
Meyer
3/91
Tuff
Richardson
46-91
Paradigm Shift: From Mass Production
to
Pine
Mass Customization
6/91
47-91
8/91
48-91
The Marketing and
10/91 (Rev. 2/92)
49-91
Murotake
Computer Aided Engineering and Project Performance: Managing
a Double-Edged Sword
R&D
Allen
Interface
Systematic' Versus Accidental' Reuse in Japanese Software Factories
10/91
50-91
Flexibility
and Performance:
A
Literature Critique
and
Strategic
Framework
11/91
51-91
11/91
52-91
12/91
53-91
12/91
Shifting Econonues:
From Craft Production to
Flexible
Systems
and Software Factories
Beyond
Understanding the Commitment of Pioneers in Emerging Fields
and Technology
Persistence:
of Science
Institutional Variations in
Pioneering Researchers
Problem Choice and Persistence among
Cusumano
Working Paper
List (continued)
Number
Date
54-91
Title
The Role
of Students in Pioneering
New
Fields of Science
and Technology
12/91
55-91
Debackere
Rappa
Technological Communities and the Diffusion of Knowledge
12/91
56-91
Debackere
The Voice
of the
Customer
Griffin
10/91
57-92
Hauser
The Influence
of Inter-Project Strategy
on Market Performance
in the
Auto Industry
1/92
58-92
1/92
59-92
Nobeoka
Cusumano
Linking International Technology Transfer with Strategy and Management:
A
Literature
Commentary
Using the Literature
in the
Cusumano
Elenkov
Study of Emerging Fields of Science and Technology
Rappa
Garud
7/91
60-92
Author(s)
Rappa
Technological Progress and the EHu'ation of Contribution Spans
12/91
Rappa
Debackere
Garud
61-92
9/91
62-92
Technological Trajectories and Selection Mechanisms in the Development of
Cochlear Implants
On the Persistence
of Researchers in Technological
Development
An International Comparison of Scientists in an Emerging Field
Life
on the
The
Social Construction of Technological Reality
Frontier:
Core Competencies, Product Families and Sustained Business Success
2/92
66-92
4/92
68-92
2/92
Creating Occasions For Technological
—
- Bennett
the Renaissance™ Spirometry System: Listerung
Voice of the Customer
Puritan
to the
Time
Flies
When You're Having Fun: How Consumers Allocate Their Time When
Evaluating Products
(Rev. 11/92)
69-92
Moving
Ideas to Market and Corporate Renewal
7/92
70-92
Utterback
Meyer
Windows Of Opportunity:
3/92
Adaption In Organizations
(Rev. 9/92)
67-92
Garud
Rappa
1/92
65-92
Debackere
Rappa
1/92
64-92
Garud
Rappa
12/91
63-92
Garud
Rappa
Tyre
Orlikowski
Hauser
Hauser
Urban
Weinberg
Meyer
Utterback
Project
Management
in
Technology Innovation, Application and Transfer
Frankel
5/92
71-92
Investments of Uncertain Cost
Pindyck
9/92
72-92
Identifying Controlling Features of Engineering Design Iteration
10/92
73-92
10/92
Smith
Eppinger
Objectives and Context of Software Measurement, Analysis and Control
Cusumano
Working Paper
List (continued)
Number
Date
74-92
Tide
An Empirical Study
Author(s)
of Manufacturing Flexibility in Printed-Circuit Board
Assembly
Suarez
Cusumano
11/92
Fine
75-92
11/92
76-92
Japanese Technology Management: Innovations, Transferability, and the
Limitations of 'Lean' Production
Hauser
Customer-Satisfaction Based Incentive Systems
Simester
Wernerfelt
11/92
77-92
The Product Family and the Dynamics
of
Meyer
Core Capability
Utterback
11/92
78-92
11/92
79-92
Multi-Project Strategy
and Organizational Coordination
Nobeoka
Cusumano
in
Automobile Product Development
Pattern of Industrial Evolution: Dominant Design and the Survival of Firms
12/92
80-92
11/92
81-92
Cusumano
King
Innovation from Differentiation: Pollution Control Departments and
Iimovation in the Printed Circuit Industry
Answer Garden and
Utterback
Suarez
Ackerman
the Organization of Expertise
1/92
82-92
Skip and Scan: Qeaning
Up Telephone
Resnick
Virzi
Interfaces
2/92
83-92
Developing
New Products and Services by Listening to
the Voice of the
Customer
Roberts
8/92
84-92
A
Comparative Analysis of Design Rationale Representations
3/92
85-93
1/93
An Introductory Note for Using
Analyze Nodes, Relationships, Partitions and Boundaries
Relational Data in Organizational Settings:
AGNI and
Netgraphs
to
Lee
Lai
George
Allen
The International Center
for
Research on the Maiugement of Technology
Working Paper Order Form
Name:
Title:
Company
:
Address:
n
I
would
like to
become a working paper subscriber and
during the current year. ($125
D
I
would
like to
U.S.,
Canada, Mexico; $150
all
#
#
papers published
other countries)
order working papers individually. Please send
papers:
#
receive
all
me
the following
^)668
Date Due
MAY
2 5 2002
.IBRARIES DUPl
3
TOAD OOflSMDflT
M