From: AAAI Technical Report SS-92-02. Compilation copyright © 1992, AAAI (www.aaai.org). All rights reserved.
Situated Cognition
in Diagrammatic Reasoning
Michael Lewis Jozsef Toth
Deparunent of Information Science
University of Pittsburgh
Termssuch as intuition are frequently used to describe
the experk.nce of immediacyin the comprehensionof
possible behavior fromdiagramsor computerinterfaces.
If this aspect of cognition is operationalized as
automatic processing reflecting certain environmental
constraints (st a~nements),
the effects of intuition can
accommodated
within a more general model of cognitive difficulty. Theavailability of resourceindependent
but specialized processing suggests that systematic
methodsfor reducingcognitivedifficulty by substituting
intuitive situations for nonintoitive ones(metaphor)are
possible. Results from pilot experimentsusing Tower
of Hanoiismmq~to investigate the role of attunement
and form of ~ in such substitutions
are
reported.
1. Introduction
Reasoningfrom all%crams,tying our shoelaces, or
imaginingtrains traveling near the speedof light all rely
on our atam~entsto the regularity of certain types of
events within our natural environment.Ecological cognition involving imaginingrather than deducing,understanding rather than inferring, or intuition rather than
rule appfication wesentsa stumblingblock to cognitive
theories relying strictly on propositionalrepresentation.
[Lewis, 1991a, 1991b] has Woposed
a situated modelof
cognition in whichsuch ecological informationprocessing plays a central role. The modeldoes not discriminate between the wocessing of perceptual or mental
representations but only betweentheir memory
requiremerits. Theprimaryadvantageof visual representations
such as all%crams is attributed to apprehension of
dynamicsrather than the explicit representationof state
~aukin and Simon, 1987] although the modelaccounts
for both. The powerof this approachto mental models
lies in incorporatingthe decomposition
of situations into
states of affairs and constraints from situation theory
[Barwiseand Perry, 1983] and the characterization of
cognitive tasks as search of a problem space due to
[Newell and Simon. 1972]. This synthesis allows a
unified treatment of representation, task difficulty and
metaphoras allocation of processingproblems.
47
Twomental operations, envisioning and inspection, are hypothesizedto accountfor non-propositional
information processing. Inspection refers to the controlled act of attending to non-inferential "facts" in a
cognitivestate of affairs, such as noting the presenceof
a coffee cup on mydesk. Envisioning is an automatic
process whichfor familiar situations updates cognitive
states of affairs in response to someforms of real or
imagined action. For example, imagining an event in
which I movemykeyboard to the far end of the desk
(the cord pushes the coffee cup over the edge of the
desk, it falls, andcoffee spills on the carpet). Situations
vary in their ability to exploit ecological information
ranging from those which are largely envisionable
(everydayinteraction with our physical environment)
somethat are practically unenvisionable (unfamiliar
symbolicproblems).
The key to this account lies in treating the
envision/inspect cycle as a formof informationprocessing. This allows cognition to be described in a uniform
wayas an allocation of processing betweenrelatively
effordess, memoryindependent ecological operations
and memoryintensive controlled ones. Because the
external demandsposed by a task are in no wayaltered
by this allocation, they can still be modeledas a problem space.
The modelattributes the difficulty of cognitive
tasks to intrinsic difficulty associatedwith the size and
complexity of the problem space and the extrinsic
difficulty associated with the non-automaticcomputations neededto update states, extract information, and
supply additional constraints. Attunementsto constraints can makecognitive tasks easier in two ways.
Theability to update states automaticallythroughenvisioning provides humancognition with immunityto the
frame problem under special conditions. Wheretask
and attuned constraints can be broughtinto coincidence,
unwantedstates and paths are excludedautomatically,
reducingthe size of the search space.
This modelof cognitive difficulty can be illustrated using the Towerof Hanoiand its isomorphs(intrinsically equivalent problemsshownin table 1). Subjects find the Monster-Globeproblemsmuchmore
anchored
(order)
una~hored
(nominal)
Rule 1
Rule 2
Form
ai
a~>a1 ~ut~ ^ut.aj
--~-~ Aul~
Table 1: Problem Isomorphs
Tower of Hanoi ] Monster Globe Move] Monster Globe Change
disk size
globe size
monstersize
disk location
globe location
globe size
By Auunement
A monster mayonly
pass its largest globe
Alarger disk
may not be moved
ontop of a smaller
disk
A monster maynot
pass its globe to a
monster holding a
larger globe
If monstershold
globes of the same
size, only the largest
can chan~e
A monster maynot
change its globe to a
size held by a larger
monster
length of 1.
d~culL[Hayes and Simon, 1977], for example, reports
differences in averse solution times of less than two
As these examplesillustrate, cognition is conminutesfor the three disk Towerof Hanoiproblem,and
ceived to be a heterogeneousmixture of automatically
half an hour for the corresponding Monster-Globe
Ul~_8!ing modelsand resource consumingrules. A com(change) problem. The Monster-Globe(move) problem monsenseinterpretation of this dichotomyis that cogniis of intermediate (14 min). [Kotovskyet al., 1985]
tive tasks are direct, intuitive, and easy to the extent
diifwadty. The IVkmster~31obechange problem is the
that they do not requireinstructions.
mostdifficult becauseit violates object constancy(globe
The three representations used to describe these
size unanchored),a basic
attunementwhichplays a prieffects
are:
maryrole in theories of psychologyranging fromcognitive develolxnentto perception. Searchingits problem state of affairs (state)- an n-ary relation on n objects
space requires the use of limited working memory
S ----<r,xI,X 2, " " " Xa>
resources to determine the changes in state resulting
from actions because events do not follow environmen- interactive situation- (S) the states of affairs formable
tal constraints to whichweare attuned. The Monster- froma canonicalstate of affairs by alternate determinaGlobe moveproblem relates states through the move- tions of r.
meat of objects, to whichweare ~tn~nedand therefore
S = <f,x~,x2, ¯ ¯ ¯ x~ >
eliminates the need to use controlled processing and
intermediate storage to update states. The problem problemspace (CS)A forgetful version of a constrained
space madeavailable through these atamements,how- situation, CoS,whichstrips awaythe structure of its
ever, is substantially larger than the official onebecause states of affairs.
we can envision globes being movedamongany of the
Assuming
closure under consWaints,the difficulty
monsters, while the problemrules constrain
these
moveof interacting with a situation can be operationalizedas
ments. Because rule 1 requires information about the
the difficulty, D(O,associated with the instructed coninitial state of a moveand rule 2 requires information straints, f, the user mustactively supply. In this model,
about its terminating state, both states and the action
f describes the controlled processing needed to move
must be referenced to apply the problemrules. In the
betweenstates (the extrinsic difficulty). Anyother conTowerof Hanoi rule 1 is subsumedby aUanementsand
trolled
processing(required for planning, search, etc.)
violation of rule 2 is determinableby inspection alone,
mustbe intrinsic and therefore irreducible. This relation
becauseof the illegal state whichresults. As a consebetweenan interactive situation, S, problemconstraints,
quence we are mentally constrained
to ignore move- C, envisionedconstraints, A, instructed conswaints, f,
mentsof disks from the bottomof stacks (rule 1) and
the problemsituation, CoS,the problemspace, CS, the
can judge legality by inspecting the terminating state
cognitive space searchable using attunements, AoS,and
(rule 2) without additional reliance on workingmemory. the cognitivespace restricted by problemrules, foAoS,is
This reduces the problemto a controlled search of a
succinctly expressedas:
space of 50 states and 75 wansitions in whicheach of
CS-= CoS-- foAoS
the 36 prohibited movesare ruled out by inspection for
the illegal "larger on top of smaller" slate at a path of
48
Because attunements, A, are determined by the
situation, S, and the difference between AoS and CoS
determines the instructions needed in f, the only avenue
for reducing a problem’s difficulty. D, is to inmxluce a
new sit,ration, S’, leading in a lower D(f’). This strategy of using one situation to reason about another is
metaphor. A metaphor, G, is defined to be an isomorphism betweenconstrained situations. If a task is incompletely characterized
byits problemspace,CS,there are
additional goals or constraints associated with the
objects or relations of S which are extrinsic information
about the problem domain relevant to the agent but
missing from the problem space. Under these conditions,
mental tnm,_~latmns, g, between S and S’ are nece~ry
and require additional controlled processing. The
difficulty of a task can therefore be reduced iff there
exists a melaphor, G:
G: foAoSmf, oA,oS, such that
IXI0 + D(f’) < D(0 if translation
incomplete)
D(F’) < d(F) if translation
plete)
is required (G
is not required (G is
Three descriptions of possible relations between S and
S’ are shownin the commutative diagrams below:
Statesof Affairs
g c,x~
r~
~r’
s Ox~
ProblemSpaces
s
C~
s
Situations
h HS
~foA
xi g C, xi
C~r ~
h HS
~foAor’
x~ 8 Oxj
Whe~Xi meobjemin ,ima~. S. r and r’ are the relatiomwithin
the two .~n,-ti~s, C. the problem constraints in S, C, C’s forgetful
o0tmterlmrt, A, the mmmmnen*,
umdated with S’, f, the
comuaints for S’. and G sad H me the mmdafion, between the mates
mdmeproblem,p.a~ ~re,~__’velyd S mdS’.
The important thing to note about these diagrams
is that they are not the same. An isomorphism between
states of affairs is not nec~_essarily a metaphor, and a
metaphoris not necessarily isomorphic in states of
affairs. Properties of these forms of correspondence,
discussed at gre~:,~" length in [Lewis91a.b]. include the
reflection and Im~ervation of tmmodeledrelations and
c.onslraints by isomorphicsituations.
Assuming a conventional model of cognitive
difficulty in whichdifficulty. D(g), is proportional to the
numberof chunks and number of rule applications, isomorphic situations pose the minimumdifficulty because
they require only a single wanslation between each
object and relation in S and S’. Any other form of
correspondence would require either a greater number of
49
rules or more than one chunk per rule and would therefore lead to a larger D(g). This hypothesis along with
the hypothesis that difficulty is proportional to D(0 is
tested in a series of experiments which isolate predicted
difficulty of translation, difficulty of problem solving,
and their combined effect on problem solving using
metaphor.
2. Isomorph Experiments
1 investiThissectiondescribesa pilot experiment
gating these hypotheses that to improve performance, a
metaphor must both:
1) allocate more information processing tasks to ecological processing
2) be state equivalent
The basic strategy of these experiment is to separate the
effects of translating between reIaesentations, D(g),
from the effects of allocation of processing within
representations, D(0, by use of experimental controls.
The hypotheses are tested by comparing pairs of isomorphic problems which vary in the envisionability of
actions. D(0. and in correspondence between states.
D(g). Table 2 shows these characteristics of the problems. Problem pairs investigated are shown in table 3.
For problem pairs, the problem on the left of the arrow
serves as the metaphor and the problem on the right as
the aided problem. The differences in arity shown in
Table 2 refer to the objects binding the anchc~’ed (at
and unanchored (u/) properties from table 1. The Monster Change Color problem (monsters as chameleons
instead of globe holders) was created to provide an isomorph equivalent in extrinsic difficulty to Monster
Globe Change (actions are not envisionable, so D(f)
must include state updating) yet isomorphic in state to
Tower of Hanoi. The Tic-Tac-TOH problem, in which
tokens are movedwithin columns of a tic-tac-toe grid,
was created to provide an isomorph, equivalent in state
to the Monster Globe Change problem (ai---~olumn,
uj--~token-row), yet involving envisionable actions,
thereby reducing D(f). 2 The primary comparisons
involve
the TOH-~MCC, TOH--~MGC,
and
X Additional mbje~ are p~umtly being nm in these experimeats. The preseat data are based on results from N--4 standalone
¢onditim, Nffi5 t~dation condition, and Nffi7 yoked condition.
ZThcTIT isomoq3hwas derived using ¯ rewriu; method
describedin [Lewis,1991a].Thethird role rmtfictingtokensmwithin
columnmovement
wasgmeratedin a series of steps in whichobj.size whichis normallyanchoredby ammcmm~
was rewritten as
obje~-locafionwhichis normallyunmchomi,
leavingthe constraint
uncovm’vdby altunement
MCC--,MGM
pairinp. The additioaM imir Trr--,MGC
is included in cell I Io test the hypothesis the MGC
Woblem
can be reded by ¢oastn~ting ¯ simplon which
i, both envisioMblcmidsimaliolmllyequivalent.
the goal state to prevemthe developmentof blind
search str~gi~. Damfrom sU~l-Moneproblem solving provides a control for performancein the yokedproblemsolving conditim.
Table 2: ProblemC’hara~risfics
Envisionable
actions
Constituent
objects
3-w/ 16-~
~ MGM
MCC
Y@,$
No
~"anr.,m,lrwap,-- ... m
TIT
MOC
I¯
. ¯ ~’--~--=.-::--"=
~----.,.-u.~.---
.~~ .~
Table 3: ProblemPairs
Formof
ExtrinsicDifrgulty
Equiv~r~
’S=S
$#$"
TOH-~MCC M~-4MGM
Trr-~MGC
TOH--*MGC
gillm’e 1: MGM
standalono o(mditi~
Zl D~p~ys
Eachof these problemshas been implementedas
an intcngtive tableau in the X windowsystem. Subjects interact with IXtddemtableaus by manipulating
~
~
¯
pm~m
obiocu din~ay. T~ imerrace (k~ not ~]f
cmumaintho~ In the TOHtableau,
for insUugo,
changein the problemstate evenif this involves
aa "illegality" such as movinga disk fromthe bottomof
a stack. Problemmbleatmam linked through a control
WoOmn
which umciatm tabkau statm between the
pmtknb.’ Butmmfor viewing pmblemndm, a
2: "HT~MC, C ~ c~Ililion
gin.=,=.
cmmspomlonco
table, and task inlngtiom ate avAmmle
m ~ed problm condifi~ Prc~em ab~ms for ~
*_*_fj
±I I
live problems in the display fcmmfor the ~e asks
are shownim figurm1-3.
m
2.2 I~qperimentai Condltiom
Threetasks, stand-aloneproblemsolving, problem
i
m
i
i
i
mdyol~-pmb~m
Jolving wereused in
tt~ ~ Five problenu and four
TOH--~MCC. TOH-->MGC. MCC-~MGM.
Trr->MGC.aJzl TOH-~MGM
were tested.
Task 1: Stand.Aiome ~bkmSolving
In the sumd-akxm condition sub.iswere
presented
with the problem’s
cov~r~ and m~themble~
until
theproblem
issolved.
Errors
are
andthe last valid state restoreduponreaching
5O
3: TOH~MOC
yoked conditi~
Task 2: Problem
Transla~on
In the problemtranslation condition suborn
wcm~
presented with a l~r of tableaus. Their task wmto
repli~ c~ges m ~ ~ ~ by
min
Figure 4: Initial Solution Times
35,
30
25
( $ 25.2
20
15
10.6
q
14.1
10
10.9
D
8.4
6.8
~
5.
3.3
MCS
TOHMCSTITMCS
MCC TOHMCC
manipulating
thewoblem
tableau. State changesw~e
evenly divided bctweenIcgal and illegal uan~tiom.
This cooditioa wovidm
ped’omance
mmsmm
reflecting
lit
MGM TOH
2.3 Results
Resultsfromthis pilot experiment
are generallyin
medirection predictedby mehypotheses.Thesample
is too small,
however,
to draw
strongconclusions.Fog
example,neither order nor problemeffects reached
which are both ~ly equivalent and l~avi~
significancefog mes~d-aloneproblemsalthoughisoan eavisiormble action to copy (TOHMCC,
TITMCS) morph
differencesof this sort mea robust
finding.
will be the ensiest to tnmslate,l"nese data providea
Differences we~found amongmeyokedproblemsfog
contmtfog the yot~d-problemsotvin8 maditimsby
time to solution (p < .002) and lime spent reading
memginSuamumm
difacuUy fog pm~mruin. If, instructions(p < .005). Anorder effect wasfoundfog
fog instmce, twopairs of problemshaveequivalent meMonsterGlobeMoveproblemwith subjects spenddifficulty
Oa me ~ lask yet diffeg
in yokedinglesstimereading
insm~tioes on mesecond
problemsolving, this diffeazncemybe attributed to
encounter
(p < .05). Subjects’initial solutiontimesfog
otherfactorssuchas differences
in IZf’).
change pmbkmswere on me avetase substantially
shorter thanmehalf hourreportedelsewhere[Hayesand
Task 3: Yoked.Problem
Solving
Simon1977,Kotovsky
et al.. 1985].Wesus _pe~__that
Yoked-problem
solving is the focus of these
boththe unexpectedly
short solutiontimesandapparent
~ts because it duplicates the conditions under superiorityof the standaloneconditionmaybe artifacts
metq~ogm~htaid orob~.msolve. In yctedof mesmall standalonesample(N=4).Thetimes found
problemsolving, subjects w~el~eaented with two fog meyokedcondition(N=7)are muchcloser to throe
tableaus. Their mkwmmsolve meodginal problem reportedelsewhere.It seemsunlikelythat inm3dtgtion
whichthey manipulated
direcey. Theaid in the upper ofdirect manipulation
computer
graphicscouldaccount
tableau updated in reqxmem dumgesin meproblem fogsucha large discrepancy
in mestandalonecondition.
being mlved.This duplicates normaluse of an aid
[Kotovsky
eta/., 1985],for instance, foundmesimilar
of providingsubjectswith modelmonsters
becauseit requir~subjectsto act onme"actual"prob- numipulation
lem while using meaid in planning and in--rig me withinflatableballoons,ineffectivein speeding
upsolueffectsof the~jctims.
tions.
R~ 4 shows the solution times for each d
~
~
~c
d~
for
~aUem~
m be su~,~n uansfm" of ~g as
by ~~ et al.,
1985, ~yes and S~
1977], ~ueafly only ~,J~t ~u~ times are
shown ~ discussed.
The most ~t ~emr.e
apparent in flStwe 4 is between TOI-IMCSand
~. ~ TIT produced
slighdy
longer
solution limes than TOHin the mandalonecondition,
whenpaired with the difficult
MCSproblem,it led to
substantially better performance.This agrees with our
hypothesesthat an effective metaphormustprovideboth
¯ ~wer ~0 (be ~ef) and a low ~ (be state
~en0.
A ~ com~
can
be
~~
and
~~
w~,
for
~
com~le
~, ~g the easiest
~ as a metaphor imix’oved peffmnancew~ state
equivaleat CrOHMC~
and degraded pmfcmmaucew~
not crOHMC:S). The ~ of sointim
~ for
maadalone problems also agrees with the hypmbesm
bo~ for previously investigated pmblenmand
created for this ex~ The longest solution limes
were associawdwith ~ems having uncnvisimable
scticm (MCS,MCC),
imumed/a~solutim times we~
associatedwith problemshavingeavisionablemi3es
but ammvisimedccmm~u
cITr.lvlOM), and
times associs~ with TOHwhich has ~ emisk~d
sodoms~l coususim.
F~ 5 shows rite average numbwof m:mssful
uansladmm
withia
II~eem
minute
blodm
for~eproblem
pare 0~ msm s: TrI~t,I
’- U--’Ymmlsl~i
s31/-
compatiblemetaphorfares evmless well. The compatible TIT~MGS
pairing leads to more than twice as
manytranslafiom as MGC’s
incompatible pairing with
TOILThe compatible TOH-~MCC
pairing, by coneast, produces the seccmdhighest rate of Uanslatim
suggesting
thatit isnotanincompatib’dity
between
change and move dynamics but an incompatibility
between situations which impairs ummlation. These
results Re in accord with our h~ ¯
mustbe both sip~#!ionally equivalent and provide mvi~onableactions toallow
easy
retaliation.
3. Concluding Remarks
Thesefindings metenlalive as is apWOl~te
for a
pilot study but suggest that our apwoachto dissecting
difficulty and differentiating betweenease of soludoe
and translation
may be a ~ way to ~h
use of me~ in computer graphics. ~ model is
pa~iculadysuitable for interface design and evaluadoo
becauseit specifies the controlled processing(for which
conventionalmeasuresof difficulty exist) neededto per.
form a task. This ~l~_~ce-sthe ~’s dilemmato the
diflicedt but familiar problemof progrmnmins
to rake
advantage
of special peqx)seprcx:e___~3rs.
References
[Ba~e and Pew/, 1983] J. Barwise and J. PenT.
S~raado~r a~ At~ude:. Cambridge: Mrr
1983.
[Ham and Simm, 1977] J. Ham md IL SLmcm.
Psychological differences amongproblem ismnm~.
In NJ. CasteOan, D.B. Pismi, k OAt. Ports (Eds.)
Co&W~w
theory. WdlsJak,NJ:
Eribe__.,. 1987.
[Kotov~ es al., 1985] K. Komvsky,J. Hayes, aad IL
Simon.
Why m some ~ bar~ Evidence
from Tower of Hanoi. Cog~ PowAo/ogy, 17,
248-294,1985.
roll1614-
[Lewis, 1991a]M. Lewis.Vis~mli,Jui~!and silllalioll&
In Barwise, ~wron, ~, and ~ ~) S/moron Tkem7and its ,4pp//cat/oas, Vo/2. Stanford, CA:
CSLI, 1991.
121@I64-
[Lewis, 1991b]M. Lewis. Situated vimmlizatiogBuilding intm-faces from the mindup. (Fall issue), M~dt/med/a Re~,w,1991.
20l-,1
~Jr~I
I
I
I
~H-~M~ ~g %nln leads m ~ ~ormancealthoughthe MCC-=~MOMlaOblemwhich
requires usr~la~g fmman nonemisiomble elmugh
52
[Newell and Simon, 1972] A. Neweil and IL Simm.
HumamProb/em Solemn&.lnglewood Cliffs, NJ:
Prentice-Ban,1972.