 This slide contains information in Note View.
Switch to note view and print all of these out.
 Be sure to set your printer to black and white.
 You may need to change fonts if we have used
some that you do not have (or if your printer or
computer does funny things to our slides)
George Mason University
1
Human Factors & Applied Cognitive Program
George Mason University
2
Human Factors & Applied Cognitive Program
George Mason University
3
Human Factors & Applied Cognitive Program
Cognitive Analysis of Dynamic Performance:
Cognitive process analysis and modeling
Workshop Presented at the HFES/IEA-2000
Conference in San Diego, CA
Wayne D. Gray, Ph.D.
Deborah A. Boehm-Davis, Ph.D.
George Mason University
4
Human Factors & Applied Cognitive Program
Outline of Workshop
 Intro to the GOMS family of models
 Keystroke Level GOMS
 NGOMSL GOMS
 CPM-GOMS
REVISED Tutorial Materials may be downloaded from:
http://hfac.gmu.edu/~graypubs
George Mason University
5
Human Factors & Applied Cognitive Program
Definition of GOMS
 Characterization of a task in terms of

The user's Goals

The Operators available to accomplish those goals

Methods (frequently used sequences of operators and sub-
goals) to accomplish those goals, and

If there is more than one method to accomplish a goal, the
Selection rules used to choose between methods.
George Mason University
6
Human Factors & Applied Cognitive Program
What GOMS Is
 GOMS is a task analysis technique

Very similar to Hierarchical Task Analysis (Indeed, GOMS
is a hierarchical task analysis technique)

Hard part of task analysis is goal-subgoal decomposition
 Hard part of GOMS is goal-subgoal decomposition

Different members of the GOMS family provide you with
different sets of operators
 With established parameters
 But set is not complete (extensionable)
George Mason University
7
Human Factors & Applied Cognitive Program
Where does GOMS go?
(When do you need to do a CTA versus a TA?)
 Task analysis versus cognitive task analysis?
 What distinguishes GOMS cognitive task analysis
from other task analysis techniques?

E.g., Cognitive Task Analysis™
George Mason University
8
Human Factors & Applied Cognitive Program
TASK ANALYSIS
George Mason University
9
Human Factors & Applied Cognitive Program
Time Scale for GOMS
Levels of analysis (based on Newell’s Time Scale of Human Action)
Syst em
Analysis
Act iviti es/
Processes
Task
Task analysis
Subtasks
min
Subtask
Unit Task Analysis
Unit Tasks
101
10 sec
Unit task
Cognitive task analys is
Activities
100
1 sec
Activity
Embodied cognition
Microstrategies
10-1
100 ms
Microstrategy
Computational models
of embodied cognition
Elements
10-2
10 ms
Elements
Architectural
Parameters
10-3
1 ms
Parameters
Scale
(sec)
Time Unit s
105
days
104
hours
103
10 min
102
World ( th eory)
Bound ed
Rationality
Cognitive Band
Biological
Band
George Mason University
10
Human Factors & Applied Cognitive Program
Level 1: Task Analysis: Task 
Subtasks  Subtasks 
Duration
Complete
Argus Prime
Experiment
TASK
SUBTASKs
Complete
Session #1
SUBTASKs
Scenario
#1
SUBTASKs
Hook
Target
ID#1
Complete
Session #2
Scenario
#2
Hook
Target
ID#2
5 hr
Complete
Session #3
Scenario
#3
• • • •
Hook
Target
ID#3 • • • •
Hook
Target
ID#n
Scenario
#n
1-2 hrs
12-30 min
30 sec to 3 min
George Mason University
11
Human Factors & Applied Cognitive Program
Levels of analysis (based on Newell’s Time Scale of
Human Action)
Scale
(sec)
Time Unit s
Syst em
Analysis
Act iviti es/
Processes
105
days
104
hours
Task
Task analysis
Subtasks
103
10 min
102
min
Subtask
Unit Task Analysis
Unit Tasks
101
10 sec
Unit task
Cognitive task analys is
Activities
100
1 sec
Activity
Embodied cognition
Microstrategies
10-1
100 ms
Microstrategy
Computational models
of embodied cognition
Elements
10-2
10 ms
Elements
Architectural
Parameters
10-3
1 ms
Parameters
World ( th eory)
Bound ed
Rationality
Cognitive Band
Biological
Band
George Mason University
12
Human Factors & Applied Cognitive Program
Level 2: Subtask  Unit Tasks
Target
acquisition
yes
Classified?
no
no
Target
classification
Feedbk
check?
yes
Feedback check
George Mason University
13
Human Factors & Applied Cognitive Program
Levels of analysis (based on Newell’s Time Scale of
Human Action)
Scale
(sec)
Time Unit s
Syst em
Analysis
Act iviti es/
Processes
105
days
104
hours
Task
Task analysis
Subtasks
103
10 min
102
min
Subtask
Unit Task Analysis
Unit Tasks
101
10 sec
Unit task
Cognitive task analys is
Activities
100
1 sec
Activity
Embodied cognition
Microstrategies
10-1
100 ms
Microstrategy
Computational models
of embodied cognition
Elements
10-2
10 ms
Elements
Architectural
Parameters
10-3
1 ms
Parameters
World ( th eory)
Bound ed
Rationality
Cognitive Band
Biological
Band
George Mason University
14
Human Factors & Applied Cognitive Program
Level 3: Cognitive Task Analysis: Unit
Tasks  Activities
Target
acquisition
yes
Classified?
Classified?
no
Target
classification
no
Method for goal: Determine if target is classifed
Step 1 Select target by mouse click
Step 2 Note target ID#
Determine if radio button in info
Step 3 window is black
Verify that info window id# is same
Step 4 as target id#
Feedbk
check?
yes
Feedback check
George Mason University
15
Human Factors & Applied Cognitive Program
Levels of analysis (based on Newell’s Time Scale of
Human Action)
Scale
(sec)
Time Unit s
Syst em
Analysis
Act iviti es/
Processes
105
days
104
hours
Task
Task analysis
Subtasks
103
10 min
102
min
Subtask
Unit Task Analysis
Unit Tasks
101
10 sec
Unit task
Cognitive task analys is
Activities
100
1 sec
Activity
Embodied cognition
Microstrategies
10-1
100 ms
Microstrategy
Computational models
of embodied cognition
Elements
10-2
10 ms
Elements
Architectural
Parameters
10-3
1 ms
Parameters
World ( th eory)
Bound ed
Rationality
Cognitive Band
Biological
Band
George Mason University
16
Human Factors & Applied Cognitive Program
Level 4: Embodied Cognition:
Activity  Microstrategy
Method for goal: Determine if target is classifed
Step 1 Select target by mouse click
Step 2 Note target ID#
Determine if radio button in info
Step 3 window is black
Verify that info window id# is same
Step 4 as target id#
17
http://hfac.gmu.edu/~mm
George Mason University
Human Factors & Applied Cognitive Program
Levels of analysis (based on Newell’s Time Scale of
Human Action)
Scale
(sec)
Time Unit s
Syst em
Analysis
Act iviti es/
Processes
105
days
104
hours
Task
Task analysis
Subtasks
103
10 min
102
min
Subtask
Unit Task Analysis
Unit Tasks
101
10 sec
Unit task
Cognitive task analys is
Activities
100
1 sec
Activity
Embodied cognition
Microstrategies
10-1
100 ms
Microstrategy
Computational models
of embodied cognition
Elements
10-2
10 ms
Elements
Architectural
Parameters
10-3
1 ms
Parameters
World ( th eory)
Bound ed
Rationality
Cognitive Band
Biological
Band
George Mason University
18
Human Factors & Applied Cognitive Program
Level 5: Microstrategies  Elements
(Where GOMS doesn’t go!)
 Production Rules
 Declarative memory elements

Internal (created on RHS of production rules)

External (created by shifts of attention in the external
environment)
George Mason University
19
Human Factors & Applied Cognitive Program
Levels of analysis (based on Newell’s Time Scale of
Human Action)
Scale
(sec)
Time Unit s
Syst em
Analysis
Act iviti es/
Processes
105
days
104
hours
Task
Task analysis
Subtasks
103
10 min
102
min
Subtask
Unit Task Analysis
Unit Tasks
101
10 sec
Unit task
Cognitive task analys is
Activities
100
1 sec
Activity
Embodied cognition
Microstrategies
10-1
100 ms
Microstrategy
Computational models
of embodied cognition
Elements
10-2
10 ms
Elements
Architectural
Parameters
10-3
1 ms
Parameters
World ( th eory)
Bound ed
Rationality
Cognitive Band
Biological
Band
George Mason University
20
Human Factors & Applied Cognitive Program
Level 6: Elements  Parameters
(GOMS doesn’t go here either!)
 w -- amount of attentional capacity
 d -- decay rate
 s -- fluctuations in the strength of declarative memory
elements
 rt -- retrieval threshold
George Mason University
21
Human Factors & Applied Cognitive Program
KR not KE
 GOMS (as with most task analysis methods) focuses
on

Knowledge representation, not on

Knowledge elicitation
 There are many sources of knowledge elicitation
techniques
George Mason University
22
Human Factors & Applied Cognitive Program
Task Analysis versus Functionality
 A task analysis does not guarantee functionality, see
Kieras, D. E. (in press). Task analysis and the design of
functionality, CRC Handbook of Computer Science and
Engineering.: CRC Press, Inc.
for a cogent discussion of this issue
George Mason University
23
Human Factors & Applied Cognitive Program
Definition of GOMS
 Characterization of a task in terms of

The user's Goals

The Operators available to accomplish those goals

Methods (frequently used sequences of operators and sub-
goals) to accomplish those goals, and

If there is more than one method to accomplish a goal, the
Selection rules used to choose between methods.
George Mason University
24
Human Factors & Applied Cognitive Program
Example of G-O-M-S
 To carry out a GOMS analysis of the following
task involving a digital clock:
 Set the clock
 Top level goal: SET CLOCK
George Mason University
25
Human Factors & Applied Cognitive Program
Example of G-O-M-S: Goals
Goals and subgoals
Set Clock
Set Hour
Set Min
George Mason University
26
Human Factors & Applied Cognitive Program
Example of G-O-M-S: Operators
Operators are the most elementary steps in which
you choose to analyze the task.
 Reach <type> button
 Hold <type> button
 Release <type> button
 ClickOn <type> button
 Decide: if <x> then <y>
 Verify
George Mason University
27
Human Factors & Applied Cognitive Program
Example of G-O-M-S: Methods

Top-level user goals
SET-CLOCK

Method for goal: SET-CLOCK
Step 1. Hold TIME button
Step 2. Accomplish goal: SET-HOUR
Step 3. Accomplish goal: SET-MIN
Step 4. Release TIME button
Step 5. Return with goal accomplished

Method for goal: SET-<digit>
Step 1. ClickOn <digit> button
Step 2. Decide: If target <digit> = current <digit>, then return with goal accomplished
Step 3. Goto 1
George Mason University
28
Human Factors & Applied Cognitive Program
Example of G-O-M-S: Selection rules

No selection rules in this example as this clock has only ONE
method for accomplishing each goal, but . . .
Selection rule for goal: SET-HOUR
If target HOUR ≤ 4 hours from current HOUR, then Accomplish
Goal: ClickOn HOUR
If target HOUR > 4 hours from current HOUR, then Accomplish
Goal : Click&Hold HOUR
George Mason University
29
Human Factors & Applied Cognitive Program
Applications of GOMS
Case 1. Design of mouse-driven text editor
Case 2. Directory assistance workstation
Case 3. Space operations database system (for orbital objects)
Case 4. Bank deposit reconciliation system.
Case 5. CAD system for mechanical design.
Case 6. Television control system.
Case 7. Nuclear power plant operator's associate.
Case 8. Intelligent tutoring system.
Case 9. Industrial scheduling system.
Case 10. CAD system for ergonomic design.
Case 11. Telephone operator workstation.
List compiled by John & Kieras (1997a).
George Mason University
30
Human Factors & Applied Cognitive Program
Where does GOMS go?
Usability writ large
Organization Issues
[anthropology, social

Conceptual Issues, Mental Models
[cognitive science]
Procedural Issues (methods)
[GOMS]
Perceptual Issues
[tradtional HF, cognitive
engineering, Tullis, Wickens]
George Mason University
31
Human Factors & Applied Cognitive Program
Development process
without
analytic
modeling
George Mason University
32
Human Factors & Applied Cognitive Program
Development
process
with
analytic
modeling
GOMS as Analytic Modeling
 GOMS analysis produces a model of behavior
 Given a task, the model predicts the methods, or
sequences of operators, that a person will perform to
accomplish that task
 Can look at the GOMS model in different ways to
qualitatively and quantitatively assess different types
of performance
George Mason University
34
Human Factors & Applied Cognitive Program
Scope of GOMS: What it can do

Predict the sequence of operators an expert will perform

Predict performance time of expert users - even in real-
world situations

Predict learning time in relatively simple domains

Predict savings due to previous learning

Help design on-line help and manuals
George Mason University
35
Human Factors & Applied Cognitive Program
Scope of GOMS: What it can do, con’t
 GOMS has been applied to both:

User-driven interaction

“Situated” or event-driven interaction
George Mason University
36
Human Factors & Applied Cognitive Program
Scope of GOMS:
What it might be able to do
 Research has made progress on

Predicting the number of some types of errors
 see discussion in: Gray, W. D. (2000). The nature and
processing of errors in interactive behavior. Cognitive Science,
24(2), 205-248.

Predicting the effects of display layout on performance time
George Mason University
37
Human Factors & Applied Cognitive Program
Scope of GOMS: What it can't do
 Predict problem-solving behavior
 Predict how GOMS structure grows from user
experience
 Predict behavior of casual users, individual
differences...
 Predict the effects of fatigue, user preference,
organizational impact...
George Mason University
38
Human Factors & Applied Cognitive Program
General Factors to Consider
in GOMS Models
 When deciding what type of GOMS model you
need, you must consider...

what control structure

what level of analysis

whether to approximate behavior with serial or parallel
processes

...different uses of GOMS models lead to different values of
these factors
 These factors will be a recurring theme
George Mason University
39
Human Factors & Applied Cognitive Program
GOMS Family of Analysis Methods
Keystroke-Level Model
CMN-GOMS
NGOMSL
CMN-GOMS for Highly Interactive Tasks
CPM-GOMS
= “worked example” provided in this HFES workshop
George Mason University
40
Human Factors & Applied Cognitive Program
Break time
George Mason University
41
Human Factors & Applied Cognitive Program
Keystroke-Level Model: Intro


The simplest of all GOMS models: OM only!!!

No explicit goals or selection rules

Operators and Methods (in a limited sense) only
“Useful where it is possible to specify the user’s interaction
sequence in detail” (CMN83, p. 259).

Control structure: Flat

Serial or Parallel: Serial

Level of Analysis: Keystroke-level operators
George Mason University
42
Human Factors & Applied Cognitive Program
Keystroke-Level Model: Example
 TAO example
area code
1
2
3
num
7
0
3
exchange
4
5
6
9 9 3
7
1
line
8 9 10
3 5 7
pin
11 12 13 14
1 2 3 4
George Mason University
43
Human Factors & Applied Cognitive Program
Keystroke-Level Model: Overview
 Step 1: Lay out assumptions
 Step 2: Write out the basic action sequence (list the keystrokelevel physical operators involved in doing the task)
 Step 3: Select the operators and durations that will be used
 Step 4: List the times next to the physical operators for the task
 Step 4a: If necessary, include system response time operators
for when the user must wait for the system to respond
 Step 5: Next add the mental operators and their times
 Step 6: Sum the times of the operators
George Mason University
44
Human Factors & Applied Cognitive Program
Keystroke-level Model: Operators
 K: Keystroke
 T(n): Type a sequence of n characters on a keyboard
 P: Point with mouse to a target on a display
 B: Press or release mouse button
 BB: Click mouse button
 H: Home hands to keyboard or mouse
 M: Mental act of routine thinking
 W(t): Waiting time for system to respond
George Mason University
45
Human Factors & Applied Cognitive Program
Card, Moran, and Newell on “Mentals”
 “M operations represent acts of mental preparation for applying
physical operations. Their occurrence does not follow directly
from the physical encoding, but from the specific knowledge
and skill of the user” p. 267
 “The rules for placing M’s embody psychological assumptions
about the user and are necessarily heuristic, especially given
the simplicity of the model” p. 267.
George Mason University
46
Human Factors & Applied Cognitive Program
Heuristics for inserting mental operators
Basic psychological principle: physical operations in
methods are chunked into submethods.
RULE 0: Insert M’s in front of all K’s or B’s that are not part of
argument strings proper (e.g., text or numbers). Place M’s in front of all
P’s that select commands (not arguments) or that begin a sequence of
direct-manipulation operations belonging to a cognitive unit.
•Pointing to a cell on a spreadsheet is pointing to an argument -- no M
•Pointing to a word in a manuscript is pointing to an argument -- no M
•Pointing to a icon on a toolbar is pointing to a command -- M
•Pointing to the label of a drop-down menu is pointing to a command -- M
George Mason University
47
Human Factors & Applied Cognitive Program
Heuristics for inserting mental operators
 Rules 1-4 are heuristics (rules of thumb) for deleting
mentals

“A single psychological principle lies behind all the deletion
heuristics . . . physical operations in methods are chunked into
submethods” p. 268
George Mason University
48
Human Factors & Applied Cognitive Program
Heuristics for inserting mental operators
Basic psychological principle: physical operations in
methods are chunked into submethods.
RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument
strings proper (e.g., text or numbers). Place M’s in front of all P’s that
select commands (not arguments) or that begin a sequence of directmanipulation operations belonging to a cognitive unit.
RULE 1: If an operator following an M is fully anticipated1 in an
operator just previous to M, then delete the M (e.g., PMK --> PK or
PMBB --> PBB).
•That is, the “M” drops out because the “P” and “BB” belong
together in a chunk -- mental unit.
•The button press “BB” is fully anticipated as the cursor is being
moved to the target.
George Mason University
49
Human Factors & Applied Cognitive Program
Heuristics for inserting mental operators
Basic psychological principle: physical operations in
methods are chunked into submethods.
RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings
proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not
arguments) or that begin a sequence of direct-manipulation operations belonging to
a cognitive unit.
RULE 1: If an operator following an M is fully anticipated1 in an operator just
previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB).
RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the
name of a command), then delete all M’s but the first.
•Works with command names -- but what is a command name in a GUI
interface?
•Physical actions: P(File)+ B + P(Save) + B
•RULE 0: MP + MB + MP + MB
•RULE 1: MPB + MPB
•Does rule 2 apply to eliminate the middle mental? MPBPB ?
George Mason University
50
Human Factors & Applied Cognitive Program
Heuristics for inserting mental operators
Basic psychological principle: physical operations in
methods are chunked into submethods.
RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument
strings proper (e.g., text or numbers). Place M’s in front of all P’s that select
commands (not arguments) or that begin a sequence of direct-manipulation
operations belonging to a cognitive unit.
RULE 1: If an operator following an M is fully anticipated1 in an operator
just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB).
RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the
name of a command), then delete all M’s but the first
RULE 3: If a K is a redundant terminator (e.g., the terminator of a
command immediately following the terminator of its argument),
then delete the M in front of it.
•Applies to clicking OKAY in dialog buttons after you select a
command; e.g., in Powerpoint, you have selected text, gone to
the FORMAT:FONT palette, clicked on bold, and now point and
click on OKAY -- pointing to and clicking on OKAY is PBB, not
MPBB
George Mason University
51
Human Factors & Applied Cognitive Program
Heuristics for inserting mental operators
Basic psychological principle: physical operations in
methods are chunked into submethods.
RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings
proper (e.g., text or numbers). Place M’s in front of all P’s that select commands
(not arguments) or that begin a sequence of direct-manipulation operations
belonging to a cognitive unit.
RULE 1: If an operator following an M is fully anticipated1 in an operator just
previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB).
RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the name of a
command), then delete all M’s but the first
RULE 3: If a K is a redundant terminator (e.g., the terminator of a command
immediately following the terminator of its argument), then delete the M in front
of it.
RULE 4: If a K terminates a constant string (e.g., a command name), then
delete the M in front of it; but if the K terminates a variable string (e.g., an
argument string), then keep the M in front of it.
George Mason University
52
Human Factors & Applied Cognitive Program
Heuristics for inserting mental operators
 The four heuristics do NOT capture the notion of
method chunks precisely -- these are only
approximations
 Ambiguities: Is something “fully anticipated” or is
something else a “cognitive unit”?
 Much of this ambiguity stems from variations in
expertise of the users we are modeling
George Mason University
53
Human Factors & Applied Cognitive Program
Heuristics for inserting mental operators
Basic psychological principle: physical operations in
methods are chunked into submethods.
RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings
proper (e.g., text or numbers). Place M’s in front of all P’s that select commands
(not arguments) or that begin a sequence of direct-manipulation operations
belonging to a cognitive unit.
RULE 1: If an operator following an M is fully anticipated1 in an operator just
previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB).
RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the name of a
command), then delete all M’s but the first
RULE 3: If a K is a redundant terminator (e.g., the terminator of a command
immediately following the terminator of its argument), then delete the M in front
of it.
RULE 4: If a K terminates a constant string (e.g., a command name), then delete the
M in front of it; but if the K terminates a variable string (e.g., an argument string),
then keep the M in front of it.
George Mason University
54
Human Factors & Applied Cognitive Program
KLM--mentals: example 1.

Example: SET COLUMN WIDTH 5<cr>

List the keystroke level physical operators involved in doing the task



RULE 0

M+KKKK+M+KKKKKKK+M+KKKKKK+K+M+K or

M+4K(set_)+M+7K(column_)+M+6K(width_)+1K(5)+M+1K(<cr>)
RULE 1


M+17K(set_column_width_)+1K(5)+M+1K(<cr>)
RULE 3


no change in this example
RULE 2


KKKKKKKKKKKKKKKKKKK (19 K’s)
No change in this example
Rule 4

No change in this example
George Mason University
55
Human Factors & Applied Cognitive Program
KLM--mentals: example 2

Example: spellcheck “catelog”

List the keystroke level physical operators involved in doing the task


RULE 0


n/a (“catelog” + spellcheck do not form a cognitive unit)
RULE 3


n/a
RULE 2


P+M+BBBB+M+P+BB
RULE 1


P+BBBB+P+BB (where BB is a mousedown + mouseup, and BBBB is a doubleclick)
n/a
RULE 4

n/a
George Mason University
56
Human Factors & Applied Cognitive Program
KLM--mentals: example 3
 Example: save a file on a Mac using menus
 List the keystroke level physical operators involved in doing the
task

P+B+P+B
 RULE 0

M+P+M+B+M+P+M+B
 RULE 1

M+P+B+M+P+B
 RULE 2

n/a or M+P+B+P+B ???
 Issue: Is this FILE-->SAVE menu selection a single cognitive unit
or two?
George Mason University
57
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m1 current
 Step 1: Lay out your assumptions

There are several fields on the display, first thing that any error
recovery method must do is to identify the field to be changed.
In this case the field is the calling-card field (CCN).

For purposes of this exercise, we assume the error is made in
the second number of the exchange.

TAO’s hands are on the keyboard
 Step 2: Write out the basic action sequence (the
physical operators)

ƒkey(ccn) + digit(14) + enterKey
George Mason University
58
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m1 current
 Step 3: select the operators and durations that will be
used

We will use the ones from Kieras (1993).
operator
mental
keystroke-<key>
mouseDown or Up
click (mouseD & Up)
homing
pointing w/mouse
doubleClick
abbrev
M
K
B
BB
H
P
BBBB
duration
1200msec
280msec
100msec
200msec
400mec
1100msec
400msec
George Mason University
59
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m1 current
 Step 4: List the times next to the physical operators
for the task.
Method 1 of TAO task
press reset function keyfKEY(ccn)
type digits
digit
outpulse new number toenter
dbase
NUM op type
1
K
14
K
1
K
total time
time
0.28
3.92
0.28
4.48
George Mason University
60
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m1 current
 Step 5: Next add the mental operators and their times
Method 1 of TAO task
M before cmd
press reset function key
type digits
M terminates argument string
outpulse new number to dbase
fCCN
digit
enter
NUM op type
1
M
1
K
14
K
1
M
1
K
total time
time
1.20
0.28
3.92
1.20
0.28
6.88
 Step 6: Sum the times of the operators

Predicted time for current method is 6.88 sec

(note: this time is the same regardless of “where” the error is
made)
George Mason University
61
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m2 bs/delete


Step 1: Lay out your assumptions

s/a model 1 except;

delete key backs up and deletes each digit
Step 2: Write out the basic action sequence (the physical
operators)


ƒkey(ccn) + delKey(10) + digit(10) + enterKey
Step 3: Same operators as for model 1.
George Mason University
62
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m2 bs/delete

Step 4: List the times next to the physical operators for the task.
Method 2: bs/delete
press reset function key
bs/delete to digit
digits to retype
outpulse new num to dbase
total time
fCCN
delKey
digit
enter
NUM op type
1
K
10
K
10
K
1
K
time
0.28
2.80
2.80
0.28
6.16
George Mason University
63
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m2 bs/delete

Step 5: Next add the mental operators and their times
Method 2: bs/delete
M before cmd
press reset function key
bs/delete to digit
digits to retype
verify done
outpulse new num to dbase
total time

fCCN
delKey
digit
enter
NUM op type
1
M
1
K
10
K
10
K
1
M
1
K
time
1.20
0.28
2.80
2.80
1.20
0.28
8.56
Step 6: Sum the times of the operators
George Mason University
64
Human Factors & Applied Cognitive Program
Keystroke-Level Model:m3 bkup/delete

Step 1: Lay out your assumptions

s/a model 1 except;

backup key backs up without deleting. Delete key backs up and
deletes

Step 2: Write out the basic action sequence (the physical
operators)


ƒkey(ccn) + bkupKey(9) + delKey(1) + digit(1) + enterKey
Step 3: Same operators as for model 1.
George Mason University
65
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m3 bkup/delete

Step 4: List the times next to the physical operators for the task.
Method 3: bkup-delete
press reset function key
backup to digit
delete digit
digits to retype
outpulse new num to dbase
total time
fCCN
bkup
del
digit
enter
NUM op type
1
K
9
K
1
K
1
K
1
K
time
0.28
2.52
0.28
0.28
0.28
3.64
George Mason University
66
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m3 bkup/delete

Step 5: Next add the mental operators and their times (Your turn!!!
Our answer are on the next page, no peeking!!!)
Method 3: bkup-delete
NUM op type
time
press reset function key
fCCN
1
K
0.28
backup to digit
bkup
9
K
2.52
del
1
K
0.28
digits to retype
digit
1
K
0.28
outpulse new num to dbase
enter
1
K
0.28
delete digit
total time
George Mason University
67
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m3 bkup/delete

Step 5: KLM w/mentals.
Method 3: bkup-delete
set up workstation to retype
number
press reset function key
backup to digit
delete digit
digits to retype
verify done
outpulse new num to dbase
total time
NUM op type
fCCN
bkup
del
digit
enter
1
1
9
1
1
1
1
M
K
K
K
K
M
K
time
1.20
0.28
2.52
0.28
0.28
1.20
0.28
6.04
George Mason University
68
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m4 zap-gp

Step 1: Lay out your assumptions

s/a model 1 except;

Four separate function keys, zaps (deletes) either area code,
exchange, line, or pin number. Retyping need only retype the
zapped numbers.

Step 2: Write out the basic action sequence (the physical
operators)


ƒkey(ccn) + zapExch(1) + digit(3) + enterKey
Step 3: Same operators as for model 1.
George Mason University
69
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m4 zap-gp

Step 4: List the times next to the physical operators for the task.
Your turn!! (Our answer are on the next page, no peeking!!!)
Method 4: zap-gp
NUM op type
time
total time
George Mason University
70
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m4 zap-gp

Step 5: Next add the mental operators and their times (Your turn!!!
Our answer are on the next page, no peeking!!!)
Method 4: zap-gp
press reset function key
NUM op type
time
fCCN
1 K
0.28
zap-group
zap
1 K
0.28
digits to retype
digit
3 K
0.84
outpulse new num to dbase
total time
enter
1 K
0.28
George Mason University
71
Human Factors & Applied Cognitive Program
Keystroke-Level Model: m4 zap-gp

Step 5: KLM model w/mentals
Method 4: zap-gp
set up workstation to retype
number
press reset function key
zap-group
digits to retype
verify done
outpulse new num to dbase
total time
NUM op type
fCCN
zap
digit
enter
1
1
1
3
1
1
M
K
K
K
M
K
time
1.20
0.28
0.28
0.84
1.20
0.28
4.08
George Mason University
72
Human Factors & Applied Cognitive Program
Keystroke-Level Model:

Summary
Method 1: Current
Method 2: bs/delete
Method 3: bkup-delete
Method 4: zap-gp
predicted time
6.88
8.56
6.04
4.08
Of the three new methods, only one seems likely to be
fast enough to justify expense of redesign
George Mason University
73
Human Factors & Applied Cognitive Program
Keystroke-Level Model:
Dialog box interface
The task: To issue a query on telephone numbers to one out of several
databases. The users will be working in a window system, and the information
that is to be looked up in the databases can be assumed to be visible on the
screen.
Fi l e
Edi t
Dat abases
Fig 1
123234345345765-
4567
5678
6789
7899
4321
The screen as it looks before the menu choices. The telephone numbers
to be looked up are in the window at the bottom of the screen.
Thanks to Bonnie John for allowing us to use this material!!!
George Mason University
74
Human Factors & Applied Cognitive Program
Keystroke-Level Model:
Dialog box interface (2)
To use this interface, the user first pulls down a menu from the menubar at the
top of the screen. This menu contains the names of the databases and the user
positions the mouse cursor over the name of the relevant database in this list.
File Edit Databases
Databases
DB1
DB2
DB3
DB4
123-4567
234-5678
345-6789
345-7899
765-4321
Fig 2
George Mason University
75
Human Factors & Applied Cognitive Program
Keystroke-Level Model:
Dialog box interface (3)
A hierarchical submenu appears with legal queries for the chosen database.
This submenu contains an average of four alternatives, and the user moves the
mouse until it highlights the option "query on telephone number." The user
then releases the mouse button.
File Edit Databases
Databases
DB1
DB2
DB3
DB4
query on name
query on address
query on telephone
query on business
123-4567
234-5678
345-6789
345-7899
765-4321
Fig 3
George Mason University
76
Human Factors & Applied Cognitive Program
Keystroke-Level Model:
Dialog box interface (4)
This causes a standard-sized dialog box to appear in the middle of the screen as
shown in Figure 4. The large field at the top is initially empty. This field will
eventually list the telephone numbers to be submitted to the database as a
query. The dialog box does not overlap the window with the list of telephone
numbers.
File Edit Databases
Inquiry Box
Add
Delete
Update
Inquiry on telephone number
OK
Cancel
Help
123-4567
234-5678
345-6789
345-7899
765-4321
Fig 4
George Mason University
77
Human Factors & Applied Cognitive Program
Keystroke-Level Model:
Dialog box interface (5)
"The user clicks in the input field (the one-line field below the prompt "Inquiry
on telephone number") and types the telephone number.
File Edit Databases
File Edit Databases
Inquiry Box
Inquiry Box
Add
Add
Delete
Delete
Update
Inquiry on telephone number
123-4567
OK
Update
Inquiry on telephone number
123-4567
Cancel
Help
123-4567
234-5678
345-6789
345-7899
765-4321
OK
Cancel
Help
123-4567
234-5678
345-6789
345-7899
765-4321
Fig 5
Fig 6
"The user clicks on the "Add" button to add the number to the query.
George Mason University
78
Human Factors & Applied Cognitive Program
Keystroke-Level Model: Dialog box interface (6)
The state of the dialog box after the user's click on "Add."
File Edit Databases
File Edit Databases
Inquiry Box
Inquiry Box
Add
123-4567
Delete
Add
123-4567
Delete
Update
Inquiry on telephone number
123-4567
OK
Update
Inquiry on telephone number
123-4567
Cancel
Help
123-4567
234-5678
345-6789
345-7899
765-4321
OK
Cancel
Help
123-4567
234-5678
345-6789
345-7899
765-4321
Fig 7
Fig 8
"If the query is for a single telephone number, the user then clicks on the "OK"
button to submit the query.
George Mason University
79
Human Factors & Applied Cognitive Program
Keystroke-Level Model:
Dialog box interface (7)
Model 1: Dialog Box, one query
action/operator sequence
Select dbase & query type
point to "Databases" menu
clickOn menu
point to "DB3"
point to "query on telephone"
clickOn
Enter telephone number
point to "input field"
clickOn input field
move hands to keyboard
type in telephone number
move hand to mouse
point to "Add" button
clickOn "Add" button
point to "OK" button
clickOn "OK" button
#ops
KLM op
time
1
1
1
1
1
P
BB
P
P
BB
1.10
0.20
1.10
1.10
0.20
1
1
1
8
1
1
1
1
1
P
BB
H
K
H
P
BB
P
BB
1.10
0.20
0.40
2.24
0.40
1.10
0.20
1.10
0.20
total time
10.64
George Mason University
80
Human Factors & Applied Cognitive Program
Keystroke-Level Model:
Model 1: Dialog Box, one query
action/operator sequence
Select dbase & query type
begin direct-manipulation
sequence
point to "Databases" menu
clickOn menu
point to "DB3"
point to "query on telephone"
clickOn "query on telephone"
Enter telephone number
begin direct-manipulation
sequence
point to "input field"
clickOn input field
move hands to keyboard
get number
type in telephone number
begin terminate entry
move hand to mouse
point to "Add" button
clickOn "Add" button
Exit palette
begin close palette
point to "OK" button
clickOn "OK" button
Dialog box interface (8)
#ops
KLM op
time
1
1
1
1
1
1
M
P
BB
P
P
BB
1.20
1.10
0.20
1.10
1.10
0.20
1
1
1
1
1
8
1
1
1
1
M
P
BB
H
M
K
M
H
P
BB
1.20
1.10
0.20
0.40
1.20 Must either reread it or recall it.
2.24
1.20
0.40
1.10
0.20
1 M
1 P
1 BB
total time
1.20
1.10
0.20
16.64
George Mason University
81
Human Factors & Applied Cognitive Program
Keystroke-Level Model: Pop-up menu (1)
As previously mentioned, it can be assumed that the telephone number(s) in
question is/are already visible on the screen. (Figure 9)
File Edit Databases
File Edit Databases
123-4567
234-5678
345-6789
345-7899
765-4321
123-4567
234-5678
345-6789
345-7899
765-4321
Fig 9
DB1
DB2
DB3
DB4
Fig 10
To query a number, the user moves the mouse cursor to the telephone number
on the screen and clicks on the mouse button. This causes a pop-up menu to
appear over the number with one element for each database for which queries
can be performed with telephone numbers as keys. (Figure 10)
George Mason University
82
Human Factors & Applied Cognitive Program
Keystroke-Level Model: Pop-up menu (2)
The user moves the mouse until the desired database is highlighted in the popup menu. (Figure 11)
File Edit Databases
123-4567
234-5678
345-6789
345-7899
765-4321
DB1
DB2
DB3
DB4
File Edit Databases
123-4567
234-5678
345-6789
345-7899
765-4321
Fig 11
Fig 12
The user then clicks the mouse button (the system knows which number to
search on because the user pointed to it when calling up the menu). (Figure 12)
George Mason University
83
Human Factors & Applied Cognitive Program
Keystroke-Level Model:
Pop-up menu (3)
•Your turn!! Action/operator sequence only. No
mentals. (Our answer are on the next page, no peeking!!!)
Model 3: Pop-Up Menu, one query
action/operator sequence
Enter telephone number
#ops KLM op time
total time
George Mason University
84
Human Factors & Applied Cognitive Program
Keystroke-Level Model:
Pop-up menu (3)
Now try adding the mentals. (Our answer are on the next page, no
peeking!!!)
Model 3: Pop-Up Menu, one query
action/operator sequence
Enter telephone number
#ops
KLM op
time
point to first telephone number
1 P
1.10
clickOn number
1 BB
0.20
point to "DB3"
1 P
1.10
clickOn
1 BB
0.20
total time
2.60
George Mason University
85
Human Factors & Applied Cognitive Program
Keystroke-Level Model:
Pop-up menu (4)
KLM w/mentals
Model 3: Pop-Up Menu, one query
action/operator sequence
Enter telephone number
begin direct-manipulation
sequence
point to first telephone number
clickOn number
point to "DB3"
clickOn
#ops
KLM op
1
1
1
1
1
M
P
BB
P
BB
total time
time
1.20
1.10
0.20
1.10
0.20
3.80
George Mason University
86
Human Factors & Applied Cognitive Program
KLM vs “expert judgment”
Cold heuristic estimates
12
Warm heuristic estimates
10
Hot heuristic estimates
15
original GOMS (L&C College) 19
CMU students
19
PSYC645 Sp96
12
Dialog
1 query
mean
sd
20.30
26.10
12.90
10.70
13.80
6.10
16.60
3.70
14.70
3.30
15.76
2.97
Box
Pop-Up
2 queries
1 query
mean
sd
mean
sd
29.40
30.40
8.00
7.40
21.40
21.60
4.70
2.70
20.00
9.50
6.10
3.50
22.60
5.40
5.80
0.80
22.60
4.80
4.60
1.00
24.66
3.28
5.02
1.78
User testing
15.40
25.50
N
20
3.00
Difference from UT [UT - <condition> = ??]
Cold heuristic estimates
-4.90
Warm heuristic estimates
2.50
Hot heuristic estimates
1.60
original GOMS (L&C College)
-1.20
CMU students
0.70
PSYC645 Sp96
-0.36
-3.90
4.10
5.50
2.90
2.90
0.84
4.70
4.30
-3.70
-0.40
-1.80
-1.50
-0.30
-0.72
0.60
Menu
CV
2 queries
sd as % of
mean
sd
means
14.00
15.10
110%
9.10
5.30
84%
9.40
5.50
50%
11.20
1.90
21%
8.70
1.80
22%
9.30
3.09
20%
6.50
0.90
18%
-7.50
-2.60
-2.90
-4.70
-2.20
-2.80
George Mason University
87
Human Factors & Applied Cognitive Program
Break time
(Lunch Time!)
George Mason University
88
Human Factors & Applied Cognitive Program
NGOMSL
 Natural Language GOMS

based on structured natural language notation and a procedure
for constructing them

models are in program form
 Control Structure: Hierarchical goal stack
 Serial or parallel: Serial
 Level of Analysis: As necessary for your design
question
George Mason University
89
Human Factors & Applied Cognitive Program
NGOMSL - why?
 More powerful than KLM. Much more useful for
analyzing large systems
 More built-in cognitive theory
 Provides predictions of operator sequence, execution
time, and time to learn the methods
George Mason University
90
Human Factors & Applied Cognitive Program
NGOMSL - Overall Approach
 Step 1: Perform goal/subgoal decomposition
 Step 2: Develop a method to accomplish each goal

List the actions/steps the user has to do goal (at as general and highlevel as possible for the current level of analysis)

Identify similar methods/collapse where appropriate
 Step 3: Add flow of control (decides)
 Step 4: Add verifies
 Step 5: Add perceptuals, etc.
 Step 6: Add mentals for retrieves, forgets, recalls
 Step 7: Add times for each step
 Step 8: Calculate total time
George Mason University
91
Human Factors & Applied Cognitive Program
NGOMSL - Example
 Car clock
 Presetting radio stations

simple

perverse
George Mason University
92
Human Factors & Applied Cognitive Program
NGOMSL--Car Radio example (1)
Provides predictions of methods and operators used to complete a task. If
you provide estimates of operator-duration, you can get predictions of
error-free expert performance time.
SEEK
AM
–
ON
TUNE
FM
EJ
+
2:41
OFF
BASS
1
2
3
4
5
6
TREBLE
– + – +
BALANCE
L
R
FADE
L
R
PUSH CLOCK
George Mason University
93
Human Factors & Applied Cognitive Program
NGOMSL-- Car Radio example (2)
 Goal/subgoal hierarchy
set clock
set
mode
turn knob
depress knob
(and keep it
depressed)
set hour
set
minute
press tune
“-” button
press tune “+”
button
release knob
George Mason University
94
Human Factors & Applied Cognitive Program
NGOMSL-- Car Radio example (3)
 Assumptions

User's hands are NOT on the buttons at the beginning.

Time to move hand & arm to time change level is estimated by
2ft "reach" from Barnes (1963): 410 msec.

D = device time, time for clock to move forward one number, =
500 msec (we estimate this on the next slide)

I am assuming a 12-hr clock. Seems good assumption for a car
clock.

Radio is off at beginning and end.

Begin knowing the time you want to set the clock to.
George Mason University
95
Human Factors & Applied Cognitive Program
NGOMSL-- Car Radio example (4)

D = device time, time for clock to move forward one number, =
500 msec
290
Visual Perception
perceive
info (X)
50
Cognitive Operators
50
verify
info (X)
initiate release
(X)
100
Right Hand
release
(X)
Minimum time required to perceive
clockTime=targetTime and to release the toggle lever
George Mason University
96
Human Factors & Applied Cognitive Program
NGOMSL-- Car Radio example (5)
Method for goal: SET-CLOCK
Accomplish goal: set Mode to
Step 1 set Clock
Decide: If curHr°targetHr Then:
Accomplish goal: Change Hour
Step 2 display
Decide: If curMin°targetMin
Then: Accomplish goal: Change
Step 3 Minute display
stmt
time oper
0.1
op
time
tot
time
0.10
0.1
0.10
0.1
0.10
0.1
0.10
Step 4
Release "vol/on/off" knob
0.1 B
Step 5
Return with goal accomplished
0.1
0.10
assumptions
0.20
0.10
George Mason University
97
Human Factors & Applied Cognitive Program
NGOMSL-- Car Radio example (6)
Method for goal: set Mode to set Clock
Step 1
Recall "vol/on/off" is
modeSwitch
Step 5
Step 6
Decide: If location of
modeSwitch is not known then
Locate modeSwitch
Reach & Grasp "volume/on/off"
knob
Turn knob
Home thumb to "volume/on/off"
knob
Press and hold knob
Step 7
Return with goal accomplished
Step 2
Step 3
Step 4
stmt
time oper
0.1
op
time
tot
time
0.10
0.1 M
0.50
0.60
0.1 M
1.20
1.30
0.1 R
0.1 T
0.48
0.34
0.58
0.44
0.1 H
0.1 B
0.40
0.10
0.50
0.20
0.1
assumptions
retrieval from LTM or
equivalent (location and
perception of labels?) needed
and takes - 500 msec
For this example we assume
that location is NOT known. If
known then tot. time for this
step would be the stmt time,
0.10 sec
source: Barnes, 1963, reach =
410 msec, grasp = 070 msec
source: Barnes, 1963
0.10
George Mason University
98
Human Factors & Applied Cognitive Program
NGOMSL-- Car Radio example (7)
stmt
time
Method for goal: Change <time> display
Decide: If location of setTime
switch is not known then Locate
Step 1 setTime switch
Decide: If finger is NOT on
setTime switch then: Reach to
Step 2 setTime switch
Step 3
Step 4
Retrieve-from-LTM that <time>
= "+ or -"
Step 5
Determine current setting
Press and hold setTime switch
to <time>
Step 6
Step 7
Step 8
Hold until <curTime> =
<targetTime>, (500 x #digits)
Verify display <time>
Return with goal accomplished
oper
op
time
0.1
tot
time
0.10
assumptions
0.1 M
1.20
1.30 assume that loc is not known
0.1 R
0.41
0.51
0.50
retrieval from LTM or equivalent
(location and perception of labels?)
0.60 needed and takes - 500 msec
0.1 M
0.1 P
0.32
0.1 B
0.10
W
0.1 M
0.1
1.20
based upon cpm-goms analysis it should
take 420 msec for eye to move to clock
0.42 and for user to perceive the hour.
cpm-goms calc. is -250 msec, go with
0.20 KLM
System response time is 500
msec/digit, this is enough time for Ss
to perceive and initiate keyUp
response.
1.30
0.10
George Mason University
99
Human Factors & Applied Cognitive Program
NGOMSL-- Car Radio example (8)
time for SetClock goal
time for setMode goal
time for setHour from 1 to 10
time to setMin from 56 to 50
total
0.70
3.82
9.03
31.53
45.08
George Mason University
100
Human Factors & Applied Cognitive Program
NGOMSL-- PreSetting a station (1)
George Mason University
101
Human Factors & Applied Cognitive Program
NGOMSL-- PreSetting a station (2)
Goal hierarchy for optimal design
PreSET Network
Show
Set Mode to
Radio
Locate
Station
Set PreSET
etc
Select
AM/FM
seek &
identify
press & hold
preset button until
sound returns
Note differences between this and previous slide.
George Mason University
102
Human Factors & Applied Cognitive Program
NGOMSL-- PreSetting a station (3)
MODEL 1: SETTING A PRESET TO A NETWORK SHOW OPTIMAL DESIGN
Method for goal: preset network show
Step1: Decide: IF radio is not playing, THEN Accomplish goal:
set-mode-to-radio
Step2: Decide: IF show N.E. intended show, THEN Accomplish
goal: locate-station
Step3: Accomplish goal: set-preset
Step4: Return with goal accomplished
Method for goal: locate-station
Step1: Decide: IF band N.E. desired band, THEN Accomplish
goal: select AM/FM
Step2: Accomplish goal: seek&identify
Step3: Return with goal accomplished
Method for goal: select AM/FM
Step1: Reach to button for desired band
Step2: Press button for desired band
Step3: Verify band correct
Step4: Return with goal accomplished
George Mason University
103
Human Factors & Applied Cognitive Program
NGOMSL-- PreSetting a station (4)
Method for goal: seek&identify
Step1: Reach to right side of "Seek" button
Step2: Press "Seek" button
Step3: Decide: IF show N.E. intended show, THEN Goto 2
Step4: Verify show correct
Step5: Return with goal accomplished
Method for goal: set-preset
Step1: Reach to desired "Preset" button
Step2: Press and hold "Preset" button
Step3: Wait until sound returns
Step4: Remove finger from "Preset" button
Step5: Return with goal accomplished
George Mason University
104
Human Factors & Applied Cognitive Program
NGOMSL-- PreSetting perverse (1)
Goal hierarchy for perverse design
PreSET
Network Show
Locate
Station
Set Mode to
Radio
Set PreSET
etc
Select
AM/FM
Enter
seek
mode
Delete old
PreSET
Initiate
seek
seek &
identify
Exit
seek
MODE
Enter
delete
MODE
Enter PreSet
Button
Assign new
PreSET
Exit
delete
MODE
Enter
PreSet
MODE
Enter PreSet
Button
Exit
PreSet
MODE
George Mason University
105
Human Factors & Applied Cognitive Program
NGOMSL-- PreSetting perverse (2)
Radio for Perverse Preset Design
SEEK MODE
AM
FM
EJ
SEEK
–
ON
TUNE
ERASE PRESET
+
OFF
2:41
MEMORIZE PRESET
BASS
1
2
3
4
5
6
TREBLE
– + – +
BALANCE
L
R
FADE
L
R
PUSH CLOCK
George Mason University
106
Human Factors & Applied Cognitive Program
NGOMSL-- PreSetting perverse (3)
MODEL 2: SETT ING A PRESET T O A NET WORK SHOW PERVERSE DESIGN
Method for goal: preset network show (s/a optimal)
Step1: Decide: IF radio is not playing, THEN Accomplish goal:
set-mode-to-radio
Step2: Decide: IF show N.E. intended show, THEN Accomplish
goal: locate station
Step3: Accomplish goal: set-preset
Step4: Return with goal accomplished
Method for goal: locate-station (s/a optimal)
Step1: Decide: IF band N.E. desired band, THEN Accomplish
goal: select AM/FM
Step2: Accomplish goal: initiate-seek
Step3: Return with goal accomplished
Method for goal: select AM/FM (s/a optimal)
Step1: Put finger on button for desired band
Step2: Press button for desired band
Step3: Verify band correct
Step4: Return with goal accomplished
George Mason University
107
Human Factors & Applied Cognitive Program
NGOMSL-- PreSetting perverse (4)
Method for goal: initiate-seek
Step1: Accomplish goal setMode-SEEK
Step2: Accomplish goal: seek&identify
Step3: Accomplish goal: exit seek mode
Step4: Return with goal accomplished
Method for goal: setMode-<mode>
Step1: Reach for <mode> button
Step2: Press <mode> button
Step3: Verify <mode> correct
Step4: Return with goal accomplished
Method for goal: seek&identify (s/a optimal)
Step1: Reach to right side of "Seek" button
Step2: Press "Seek" button
Step3: Decide: IF show N.E. intended show, THEN Goto 2
Step4: Verify show correct
Step5: Return with goal accomplished
Method for goal: set-preset
Step1: Accomplish goal: delete-previous-preset
Step2: Accomplish goal: assign-new-preset
Step3: Return with goal accomplished
George Mason University
108
Human Factors & Applied Cognitive Program
NGOMSL-- PreSetting perverse (5)
Method for goal: delete-previous-preset
Step1: Accomplish goal: setMode-deletePreSet-on
(setMode-<mode>)
Step2: Accomplish goal: enter-preset-button
Step3: Accomplish-goal: setMode-deletePreSet-off
(setMode-<mode>)
Step4: Return with goal accomplished
Method for goal: assign-new-preset
Step1: Accomplish goal: setMode-preset-on (setMode<mode>)
Step2: Accomplish goal: enter-preset-button
Step3: Accomplish-goal: setMode-preset-off (setMode<mode>)
Step4: Return with goal accomplished
Method for goal: enter-preset-button
Step1: Reach to desired "Preset" button
Step2: Press "Preset" button
Step3: Return with goal accomplished
George Mason University
109
Human Factors & Applied Cognitive Program
NGOMSL--comparison of optimal and perverse designs
Optimal
number of different methods
Perverse
6
11
6
18
number of different steps
21
38
number of steps required
21
66
number of methods used
George Mason University
110
Human Factors & Applied Cognitive Program
Break time
George Mason University
111
Human Factors & Applied Cognitive Program
CPM-GOMS: The Embodied Cognition level
Analyzing activities into microstrategies
Syst em
Analysis
Act iviti es/
Processes
Task
Task analysis
Subtasks
min
Subtask
Unit Task Analysis
Unit Tasks
101
10 sec
Unit task
Cognitive task analys is
Activities
100
1 sec
Activity
Embodied cognition
Microstrategies
10-1
100 ms
Microstrategy
Computational models
of embodied cognition
Elements
10-2
10 ms
Elements
Architectural
Parameters
10-3
1 ms
Parameters
Scale
(sec)
Time Unit s
105
days
104
hours
103
10 min
102
World ( th eory)
Bound ed
Rationality
Cognitive Band
Biological
Band
George Mason University
112
Human Factors & Applied Cognitive Program
CPM-GOMS Extension to GOMS
 Critical Path Method (or Cognitive Perceptual Motor) -GOMS
 Control structure: Relaxed hierarchy, can be interrupted and
continued based on new information in the world
 Serial or Parallel: Parallel
 Level of Analysis: Primarily elementary perceptual, cognitive and
motor operators
George Mason University
113
Human Factors & Applied Cognitive Program
CPM-GOMS - why?
 Need for analysis suitable for parallel activities
 Human cognition is embodied cognition

In some cases we need to understand the interleaving and
interdependencies between elementary cognitive,
perceptual, and motor operations
George Mason University
114
Human Factors & Applied Cognitive Program
Brief Introduction to
PERT Charts
Talk to real
estate agents
20
20
7
Show site to
management
Visit potential
locations
4/21/89
5/18/89
3/24/89
4/20/89
3/15/89
3/23/89
0
0
Start
project
Propose
location
3/15/89
3/15/89
4/21/89
5/4/89
20
3/15/89
4/11/89
Make
presentation
to Committee
Investigate
zoning laws
4/20/89
4/20/89
Get approval from Board
of Directors
1
10
5/19/89
5/19/89
15
Prepare
offer
4/21/89
5/11/89
Aadapted from "MacProjectII: Getting Started" (Claris Corp., 1989)
George Mason University
115
Human Factors & Applied Cognitive Program
CPM-GOMS
 Schedule Charts: A Notation for Representing
Parallelism.
System
Operations
System Event
Call
Arrival
Tone
Visual
Perceptual
Operations
Aural
System Event
display
COIN PRE
display 0+
Perceive
Tone
Perceive
display
Cognitive attend attend initiate eye verify
Operations aural display movement tone
R-hand
verify
0+
attend
display
Customer
Pause
verify
COIN
PRE
Initiate
greeting
Perceive
Customer
Speech
attend
initiate
aural
function
customer key press
R-home to
function key
CPM-GOMS Model
L-hand
M otor
Operations
Verbal
EyeM ovement
Perceive
display
New England P ublic Telephone
may I help y ou?
eye
movement
George Mason University
116
Human Factors & Applied Cognitive Program
But are composites of simpler, basiclevel activities
 Activities
 Microstrategies
 Unit-tasks
We will illustrate these in the context of moving and
clicking a mouse
George Mason University
117
Human Factors & Applied Cognitive Program
George Mason University
118
Human Factors & Applied Cognitive Program
0
SLOW-M/C
ne w -curs or-loca tion
10 0
10 0
pe rce ivecursor
@ta rge t
pe rceive
-ta rge t
Predicted time:
845 msec
START
in iti tatem ovecurs or
50
50
50
atte ndta rge t
50
veri fyta rge t po s
in iti ate POG
atte ndcurs or@
ta rge t
50
verifycursor@
ta rget
50
845
initiatem ous eDn
END
10 0
54 5
m ovecursor
m ous eDn
30
POG
0
FAST-M/C
ne w-curs or-lo cati on
10 0
pe rceive
ta rge t
Predicted time:
695 msec
50
START
in iti tatem ovecurs or
50
50
atte ndta rge t
in iti ate POG
695
initiatem ous eDn
END
10 0
54 5
m ovecursor
119
veri fyta rge t po s
50
m ous eDn
George Mason University
30
POG
Human Factors & Applied Cognitive Program
George Mason University
120
Human Factors & Applied Cognitive Program
The Button Study
??
??
??
??
??
??
??
home
??
George Mason University
121
Human Factors & Applied Cognitive Program
0
0
displa y-change
ne w-curso r-locatio n
10 0
10 0
10 0
pe rc eivetarge t-btn
pe rc eive-dis play
pe rc eive-cursorlocation
Predicted time
970 sec
50
START
mo use
Dn
50
attend di spla ychang e
15 0
verifydispla ychange
50
initiate movecurs or
50
50
atte ndvisua l
50
initiate
-POG
HOME
verifycurs orloc
50
970
initiate
-mouse
Dn
END
mouse
Dn
mo ve cursor
TARGET
30
POG
122
atte ndcurs or
@ btn
50
90
30 1
mouse
Up
verify
targe t
-loc
50
George Mason University
Human Factors & Applied Cognitive Program
0
0
di spla y-chan ge
ne w-curso r-locatio n
10 0
10 0
pe rc eivetarge t-btn
pe rc eive-cursorlocation
Predicted time:
820 msec
50
START
mo use
Dn
15 0
initiate movecurs or
50
atte ndvisua l
50
50
verify
targe t
-loc
initiate
-POG
verifycurs or
-loc
50
820
initiate
-mouse
Dn
END
mouse
Dn
mo ve cursor
TARGET
HOME
30
123
atte ndcurs or
@ btn
50
90
30 1
mouse
Up
50
POG
George Mason University
Human Factors & Applied Cognitive Program
Experimental Data
Predicted
Found
Absolute Difference
HOME-toTARGET
970 msec
994 msec
(19.7)
TARGET-toHOME
820 msec
840 msec
(19.7)
2.41%
2.38%
Difference
150 msec
154 msec
Data and examples are from: Gray, W. D., & Boehm-Davis, D. A. (in press). Milliseconds Matter:
An introduction to microstrategies and to their use in describing and predicting interactive
behavior. Journal of Experiment Psychology: Applied.
Advanced copy available from:
http://hfac.gmu.edu/~mm
George Mason University
124
Human Factors & Applied Cognitive Program
Telephone Operator Workstation: The
Real-World Problem
 A telephone company wanted to replace old
telephone operator workstations with a new
workstation.
 For this company, each second saved per call
translates into a savings of $3 million/year in
operating costs.
 Will the new workstation be faster than the current
workstation, and if so, how much will it save each
year?
George Mason University
125
Human Factors & Applied Cognitive Program
Telephone Operator Workstation:
The Task
 Assist a customer in completing calls
 Record the correct billing information
 NOT directory assistance
George Mason University
126
Human Factors & Applied Cognitive Program
Telephone Operator Workstation: An
Example Call
Workstation:
TAO:
Customer:
TAO:
Workstation:
TAO:
TAO:
“Beep”
“New England Telephone may I help you?”
“Operator, bill this to 412-555-1212-1234.”
Hits a series of function & numeric keys
Displays the authorization for the calling card
“Thank you.”
Hits a key to release the workstation so
another call can come in
George Mason University
127
Human Factors & Applied Cognitive Program
Telephone Operator Workstation: Goals
 Overall Goal

Handle the call
 Subgoals explicitly trained

Determine and enter who pays for the call

Determine and enter the appropriate billing rate

Determine when the call is complete and release the
workstation
George Mason University
128
Human Factors & Applied Cognitive Program
Telephone Operator Workstation:
Operators
 Criteria for selecting a level of analysis

Captures observed behavior

Captures distinctions between workstations
 Different levels of analysis

Unit Task Level

Functional Level

Cognitive task analysis Level

Embodied cognition Level
George Mason University
129
Human Factors & Applied Cognitive Program
Telephone Operator Workstation
Operators: The cognitive task analysis level
Operators reflect the activities the
TAO must perform
• listen-to-beep
• read-screen
• greet-customer
• listen-to-customer
• enter-command
• enter-credit-card-number
• thank-customer
George Mason University
130
Human Factors & Applied Cognitive Program
Telephone Operator Workstation:
Cognitive Task Analysis
goal: handle-calls
. goal: handle-call
. . goal: initiate-call
. . . goal: receive-information
. . . . listen-to-beep
. . . . read-screen
. . . goal: request-information
. . . . greet-customer
.
.
.
.
.
.
goal: enter-who-pays
. goal: receive-information
. . listen-to-customer
.
.
.
.
.
.
.
.
.
.
.
.
. goal: enter-information
. . enter-command
. . enter-calling-card-number
goal: enter-billing-rate
. goal: receive-information
. . read-screen
Workstation: beep
Workstation: displays information
TAO: "New England Telephone
may I help you?"
Customer: “Operator, bill this to
412-555-1212-1234.”
TAO: hit F1 key
TAO: hit 14 numeric keys
and so on
George Mason University
131
Human Factors & Applied Cognitive Program
Telephone Operator Workstation:
Cognitive Task Analysis
Problems with Assumption of Sequential Operators
System RT
LISTEN-TO-BEEP
READ-SCREEN
GREET-CUSTOMER
LISTEN-TO-CUSTOMER
ENTER-COMMAND
ENTER-CREDITCARD-NUMBER
THANK-CUSTOMER
0
Time of call (seconds)
13
George Mason University
132
Human Factors & Applied Cognitive Program
Telephone Operator Workstation:
Embodied Cognition Level
 Operators at the level of elementary
operations of

Cognition

Perception

Motor acts
 Uses the Critical Path Method
George Mason University
133
Human Factors & Applied Cognitive Program
Telephone Operator Workstation:
CPM-GOMS Level
400
System RT
other
systems
330
system-rt(1)
system-rt(2)
0
workstation
display time
30
begin-call
30
display-info(2)
display-info(1)
100
290
Visual
Perceptual
Operators
100
perceiveBEEP
Aural
50
Cognitive
Operators
attend-auralBEEP
50
attendinfo(1)
perceivebinaryinfo(2)
perceivecomplexinfo(1)
50
50
initiate-eyemovement(1)
150
perceivesilence(a)
verifyBEEP
50
verifyinfo(1)
50
attendinfo(2)
50
verifyinfo(2)
50
initiategreeting
L-Hand Movements
Motor
Operators
R-Hand Movements
LISTEN-FOR-BEEP
activity-level goal
READ-SCREEN(2)
activity-level goal
1570
Verbal Responses
30
"New England Telephone
may I help you?"
eye-movement
(1)
Eye Movements
READ-SCREEN(1)
activity-level goal
GREET-CUSTOMER
activity-level goal
George Mason University
134
Human Factors & Applied Cognitive Program
Telephone Operator Workstation:
Making Quantitative Predictions
 Use the CPM-GOMS model of the benchmark call on
the current workstation as a baseline
 Modify the model to reflect design decisions

substitute the layout, response times and procedures of the
proposed workstation

propose new features or designs

obtain quantitative predictions of the effects on performance
time
George Mason University
135
Human Factors & Applied Cognitive Program
Telephone Operator Workstation:
Quantitative Predictions
 Difference between the proposed workstation
and the current workstation for 15 benchmark
calls
2
Seconds
1.5
1
0.5
0
cc01
cc02
cc03
cc04
cc05
cc06
cc07
cc08
cc09
cc10
cc11
cc12
cc13
cc16
cc18
-0.5
Call Category
George Mason University
136
Human Factors & Applied Cognitive Program
Telephone Operator Workstation:
Qualitative Explanations
Examine the critical path of the different models to explain the
quantitative predictions. E.g., why this call is longer on the
proposed than on the current workstation.
Current Works tation
operators removed from slack time
Proposed Workstation
Beginning of call
George Mason University
137
Human Factors & Applied Cognitive Program
Qualitative Explanations

Models as explanation.
Current Works tation
Proposed Workstation
operators added to
critical path
End of call
138
George Mason University
Human Factors & Applied Cognitive Program
Uses of CPM-GOMS in Design
 Project Ernestine emphasized

Quantitative comparisons between alternative systems

Qualitative explanations for differences between alternative
systems
George Mason University
139
Human Factors & Applied Cognitive Program
Uses of CPM-GOMS in Design
 CPM-GOMS can also be used to

Direct design effort

Bracketing, Profiling, and Diagnosis
George Mason University
140
Human Factors & Applied Cognitive Program
Directing Design Effort
 Designing telephone operator
workstations: Should we redesign

keyboards?

screens?

system response times?
George Mason University
141
Human Factors & Applied Cognitive Program
Directing Design Effort
Keyboards & Screens
How much of the time is each
activity on the critical path?
Call Cat.
cc01
cc02
cc03
cco4
cc05
cc06
cc07
cc08
...
Average
Sys RT Talking Keying Reading Ring/Coin
25%
40%
1%
3%
31%
3%
93%
0%
4%
0%
20%
71%
2%
6%
0%
25%
31%
6%
4%
35%
19%
44%
3%
2%
22%
12%
79%
2%
6%
0%
30%
57%
8%
3%
0%
26%
41%
23%
10%
0%
...
...
...
...
...
16%
64%
6%
5%
8%
George Mason University
142
Human Factors & Applied Cognitive Program
Directing Design Effort
 Bracketing
 Profiling
 Diagnosis
George Mason University
143
Human Factors & Applied Cognitive Program
Bracketing: Slow & Fastest Reasonable
Description
FASTEST
REASONABLE
Microstrategy
SLOW
BEST FIT
KEYPRESSMOVE
MOVE-CLICK
SLOW-KP/M
P1 FAST-KP/M
P2 MED-KP/M
SLOW-M/C
FAST-KP/M
move from zoom-point to EXP2 button
CLICK-MOV E
SLOW-C/M
FAST-C/M
mous e down on EXP2 bu tton
MOVE-CLICK
SLOW-M/C
P1 SLOW-C/M
P2 MED-C/M
SLOW-M/C
move from EXP2 bu tton to way -point
CLICK-MOV E
SLOW-C/M
mous e down on way -point
MOVE-CLICK
SLOW-M/C
CLICKKEYPRESS
SLOW-C/KP
P1 SLOW-C/KP
P2 MED-C/KP
FAST-C/KP
KEYPRESSMOVE
MOVE-CLICK
SLOW-KP/M
P1 SLOW-KP/M
P2 MED-KP/M
SLOW-M/C
FAST-KP/M
Part 1: K3 to mDn@zoo mPt
from keyp ress to move cursor
mous e down on zoo m-point
SLOW-M/C
SLOW-M/C
Part 2: mDn@zoo mPt to mDn@EXP2 btn
SLOW-M/C
Part 3: mDn @ EXP2 btn to mDn @ trgt
P1 MED-C/M
P2 SLOW-C/M
SLOW-M/C
MED-C/M
SLOW-M/C
Part 4: mDn @ trgt to K2
keyp ress to select NA V-his displa y
Part 5: K2 to mDn @ NavDesg B tn
move from center to NAVDE SG button
mous e down on NAV DESG button
SLOW-M/C
SLOW-M/C
George Mason University
145
Human Factors & Applied Cognitive Program
Bracketing
SLOW
FASTEST-REASONABLE
data
1250
1000
slow-2
750
500
250
0
1
2
3
4
5
part
George Mason University
146
Human Factors & Applied Cognitive Program
Profiling: BEST FIT models
Description
FASTEST
REASONAB LE
Microstrategy
SLOW
BEST FIT
KEYPRESSMOVE
MOVE-CLICK
SLOW-KP/M
P1 FAST-KP/M
P2 MED-KP/M
SLOW-M/C
FAST-KP/M
move from zoom-point to EXP2 button
CLICK-MOV E
SLOW-C/M
FAST-C/M
mous e down on EXP2 bu tton
MOVE-CLICK
SLOW-M/C
P1 SLOW-C/M
P2 MED-C/M
SLOW-M/C
move from EXP2 bu tton to way -point
CLICK-MOV E
SLOW-C/M
mous e down on way -point
MOVE-CLICK
SLOW-M/C
CLICKKEYPRESS
SLOW-C/KP
P1 SLOW-C/KP
P2 MED-C/KP
FAST-C/KP
KEYPRESSMOVE
MOVE-CLICK
SLOW-KP/M
P1 SLOW-KP/M
P2 MED-KP/M
SLOW-M/C
FAST-KP/M
Part 1: K3 to mDn@zoo mPt
from keyp ress to move cursor
mous e down on zoo m-point
SLOW-M/C
SLOW-M/C
Part 2: mDn@zoo mPt to mDn@EXP2 btn
SLOW-M/C
Part 3: mDn @ EXP2 btn to mDn @ trgt
P1 MED-C/M
P2 SLOW-C/M
SLOW-M/C
MED-C/M
SLOW-M/C
Part 4: mDn @ trgt to K2
keyp ress to select NA V-his displa y
Part 5: K2 to mDn @ NavD esg Btn
move from center to NAVDE SG button
mous e down on NAV DESG button
SLOW-M/C
SLOW-M/C
George Mason University
147
Human Factors & Applied Cognitive Program
Profiling
SLOW
BEST-FITTING
data
FASTEST-REASONABLE
1250 P1
1000
msec
750
500
250
0
1
2
3
4
5
part
George Mason University
148
Human Factors & Applied Cognitive Program
Diagnosis: example
 CLICK-MOVE

Used in part 2 and part 3

Use of slower microstrategies may reflect a confusion among
the three unit tasks
 Not perceptually distinct

Participants might have had to rely more on memory than is
usual for interactive behavior with current interactive technology
George Mason University
149
Human Factors & Applied Cognitive Program
Summary of CPM-GOMS

Predicts expert performance time and provides qualitative
explanations for tasks involving parallel activities

Once the models are built, schedule charts allow designers to
rapidly play what-if games with design ideas and parameters

Not intended to predict the occurrence of learning time, or errors
- other forms of GOMS models are more helpful there

Subject to the limitations of GOMS models in general (no casual
use, fatigue, etc.)
George Mason University
150
Human Factors & Applied Cognitive Program
Summary of Workshop
 Intro to the GOMS family of models
 Keystroke Level GOMS
 NGOMSL GOMS
 CPM-GOMS
http://hfac.gmu.edu/~graypubs
http://hfac.gmu.edu/~mm
George Mason University
151
Human Factors & Applied Cognitive Program
George Mason University
152
Human Factors & Applied Cognitive Program
George Mason University
153
Human Factors & Applied Cognitive Program
George Mason University
154
Human Factors & Applied Cognitive Program