Cognitive Models
Material from
Authors of Human Computer Interaction
Alan Dix, et al
Overview
Cognitive models represent users of interactive
systems
 hierarchical - user’s task and goal structure
 linguistic – user-system grammar
 physical and device – human motor skills
 architectural – underlie all of above
Cognitive models
 They model aspects of user as they interact:
 understanding
 knowledge
 intentions
 processing
 Common categorization:
 Competence – represent kinds of behavior
expected of user
 Performance – allow analysis of routine
behavior in limited applications
Goal and task hierarchies
Solve goals by solving subgoals
- Mental processing as “divide-and-conquer”
produce report
gather data
. find book names
. . do keywords search of names database
…further sub-goals
. . sift through names and abstracts by hand
…further sub-goals
. search sales database
..further sub-goals
layout tables and histograms
..further sub-goals
write description
..further sub-goals
Issues for goal hierarchies
 Granularity
 Where
do we start?
 Where do we stop – how far to subdivide?

Get down to a routine learned behavior, not
problem solving - the unit task
 Conflict
 More
than one way to achieve a goal
 Treatment of error
Techniques
 Goals, Operators, Methods and Selection
(GOMS)
 Cognitive Complexity Theory (CCT)

can represent error behavior
GOMS
 Goals - what the user wants to achieve
 Operators- basic actions user performs
(granularity)
 Methods - decomposition of a goal into sub
goals/operators

may be more than one way or method to do that
 Selection - means of choosing between
competing methods (GOMS attempts to predict)
GOMS example
GOAL: ICONIZE-WINDOW
[select
GOAL: USE-CLOSE-METHOD
MOVE-MOUSE-TO-WINDOW-HEADER
POP-UP-MENU
CLICK-OVER-CLOSE-OPTION
GOAL: USE-L7-METHOD
PRESS-L7-KEY]
For a particular user Sam:
Rule 1: Select USE-CLOSE-METHOD unless
another rule applies.
Rule 2: If the application is GAME, select
L7-METHOD.
GOMS as a measure of
performance
 selection rules can be tested for accuracy
against user traces
 stacking depth of goal structure can estimate
STM requirements
 good for describing how experts perform
routine tasks


not for comparing across tasks
not for predicting training time
Cognitive Complexity Theory - CCT
- basic premises of goal decomposition
- provides more predictive power
Two parallel descriptions:
 User - production rules of the form:
if condition then action
 Device - generalized transition networks
covered under dialogue models
Example: editing with vi
Production rules are in long-term memory
- 4 rules in the text on page 425
User sees a mistake - Model contents of working
memory as attribute-value mapping
(GOAL perform unit task
(TEXT task is insert space)
(TEXT task is at 5 23)
(CURSOR 8 7)
Example: editing with vi
Rules are pattern-matched to working memory,
e.g.,
LOOK-TEXT task is at %LINE %COLUMN
is true, with LINE = 5 COLUMN = 23.
Four rules model inserting a space –1st one only one that
can fire:
SELECT-INSERT-SPACE
INSERT-SPACE-DONE
INSERT-SPACE-1
INSERT-SPACE-2
//bind to location
//finished - unbind
//move cursor
//hit insert key and space
Example: editing with vi
When fired, binds the LINE and COL to 5 and 23
respectively and adds to working memory
(GOAL insert space)
(NOTE executing insert space)
(LINE 5)
(COLUMN 23)
Now INSERT-SPACE-1 will fire
Notes on CCT
 Rules don’t fire in order written, may repeat
 Parallel model – rules can fire simultaneously
 Novice versus expert style rules
 Error behavior can be represented
 Measures



Depth of goal structure
Number of rules (more means interface more
difficult to learn)
Comparison with device description
Problems with goal hierarchies
 description can be enormous
 a post hoc technique – risk is that it is defined
by the computer dialog and not user
 expert versus novice
 Simple extensions possible

goal closure (makes sure subgoal satisfied)

eg. ATM example
Linguistic notations
 User’s interaction with a computer is often
viewed in terms of a language.


Backus-Naur Form (BNF)
Task-Action Grammar (TAG)
BNF
 Very common notation from computer science
 A purely syntactic view of the dialogue
Basic syntax:
nonterminal ::= expression
An expression contains terminals and nonterminals combined
in sequence (+) or as alternatives (|).
Terminals lowest level of user behavior
CLICK-MOUSE, MOVE-MOUSE
Nonterminals ordering of terminals; higher level of abstraction
select-menu, position-mouse
draw line ::= select line + choose points + last point
select line ::= pos mouse + CLICK MOUSE
choose points ::= choose one | choose one + choose
points
choose one ::= pos mouse + CLICK MOUSE
last point ::= pos mouse + DBL CLICK MOUSE
pos mouse ::= NULL | MOVE MOUSE + pos mouse
Measurements with BNF
 Number of rules or number of + and | operators
 Complications



same syntax for different semantics
reflects user’s actions, not user's perception of
system responses
enforcement of consistency in rules
 Extensions


include “information-seeking actions” in grammar
parameterized grammar rules
Task-Action Grammar - TAG
 Making consistency in language more explicit
than in BNF
 Encoding user's world knowledge

(eg. up is opposite of down)
 Accomplished by


Parameterized grammar rules
Nonterminals are modified to include
additional semantic features
Consistency in TAG
In BNF, three UNIX commands would be
described as
copy ::=
|
move ::=
|
link ::=
|
cp
cp
mv
mv
ln
ln
+
+
+
+
+
+
filename + filename
filenames + directory
filename + filename
filenames + directory
filename + filename
filenames + directory
Consistency in TAG
 In TAG, this consistency of argument order
can be made explicit using a parameter, or
semantic feature for file operations.
file op[Op] ::= command[Op]+ filename +
filename | command[Op]+ filenames +
directory
command[Op = copy] ::= cp
command[Op = move] ::= mv
command[Op = link] ::= ln
Notes
 Ignore system output

(there are extensions to BNF and TAG)
 Hierarchical and grammar-based techniques
initially developed when systems were mostly
command-line or keyboard and cursor based.
Physical and device models
 Based on empirical knowledge of human
motor system
 User's task: acquisition, then execution.
 These models only address execution
 Models are complementary with goal
hierarchies
 Models


The Keystroke Level Model (KLM)
Buxton's 3-state model
Keystroke Level Model - KLM
Six execution phase operators
Physical motor
K keystroking
P pointing
H homing
D drawing
Mental
M mental preparation
System
R response
Times are empirically determined.
Texecute = TK + TP + TH + TD + TM + TR
Example
GOAL: ICONISE-WINDOW
[select
GOAL: USE-CLOSE-METHOD
MOVE-MOUSE-TO-WINDOW-HEADER
POP-UP-MENU
CLICK-OVER-CLOSE-OPTION
GOAL: USE-L7-METHOD
PRESS-L7-KEY]
Models so far
GOMS – cognitive processing involved in
deriving subgoals to carry out a task to
achieve a goal
CCT – distinction between LTM (rules) and
STM (working memeory)
Linguistic (BNF and TAG) – focus on syntactic
KLM – motor and mental operators
Architectural models
All of cognitive models make assumptions
about the architecture of the human mind.
 Problem spaces – behavior viewed as sequence
of agent/environment states (can predict
erroneous behavior)
 Interacting Cognitive Subsystems




provides model of perception, cognition, and action
9 subsystems (5 physical, 4 mental)
view of user as information processing machine
concerned with determining how easy particular
procedures of action sequences become
Last notes
 Cognitive models attempt to represent users
as they interact with the system
 Three categories – what were they?
 Most cognitive models do not deal with user
observation and perception.
 Some techniques have been extended to
handle system output, but problems persist.
 Issues:


Level of granularity
Exploratory interaction versus planning