Theory-Driven Design in
HCI
Mary Czerwinski
Microsoft Research
Overview
 Why

theory, especially now?
Brief history of HCI, psychology and theory
 The
importance/role of theory
 Examples of theory-driven research
 How to do theory
 Summary of benefits of theory to HCI
UIST 2004
2
Why is it a good time to revisit
theory? Yesterday…
UIST 2004
3
Dichotic Listening Task (Cherry,
1953)
UIST 2004
4
Cocktail Party Phenomenon
 Cocktail

How is it that out of a sea of voices we
can focus on a single conversation?
 Cocktail

party problem
party effect
Moray (1959)
• While you are usually unaware of identity of
words in a non-attended conversation…
• A notable exception is your name
UIST 2004
5
Attenuation Theory of Attention
(Treisman, 1960)

Blocking out
the irrelevant
content easy
until….
 It’s
semantically
meaningful or
important to
you
Hey, Mary!
UIST 2004
6
Guidelines for Speech Communication
Applications, Cockpits, etc.
 Provide
a mechanism to “pull” one voice
into focus

Mostly focusing on physical attributes of the
message
 Do
not present too much information
simultaneously
 Provide enough time for the user to fully
fuse streams if necessary
UIST 2004
7
80s and 90s: Personal Computer
UIST 2004
8
User-Centered Theories in HCI
 The
Psychology of Human-Computer
Interaction, by Card, Moran and Newell
(1983)
 Simon and Newell’s contributions to
psychology, AI and HCI
 Cognitive and perceptual psychology
theories codified into guidelines

A LOT of theory from this era is used in HCI
design practice today
UIST 2004
9
And Then, Time Stood Still…
 Advances
to GUI
desktops arguably
stalled
 Lots of evolution
on designs but
less new theory
 Do we need it?
 Where are the
breakthroughs?
UIST 2004
Evolution
10
Today: Ubicomp
UIST 2004
11
Theoretical Guidance Missing…
 Social
proxemics and etiquette?
 Multi-cursor interaction?
 New mental models?
 New metrics for productivity and
acceptance?
 New artifacts?
 Privacy?
 New input approaches?
UIST 2004
12
Tomorrow Today
 Brain-computer
interaction devices
like BrainGate
 Controlling objects
with thought is
becoming a reality
 Good science—
new theories?
UIST 2004
13
UIST 2004
14
VeriChip


FDA approved
implantable chips
RFID tags
Rooted in the skin for
accessing medical
records
Privacy issues are
becoming pervasive
in our research
Philosophy->Theory-?
UIST 2004
15



A Tools Stage of HCI

(Painting by Zdenek
Burian)
 Cro Magnons occurred
~40,000 years ago
 Hunted mainly with
spears, (bow and
arrows were later
developed).
 Made tools from
blades of Flint stone,
used for preparing
UIST 2004
animal skins.
16
Shneiderman—Why Theories in
HCI?





Descriptive: clarify terms, key concepts
Explanatory: reveal relationships and
processes
Predictive: about performance and situations
Prescriptive: convey guidance for decision
making in design by recording best practice
Generative: enable practitioners to create,
invent or discover something new
UIST 2004
17
Examples of Theory-Driven
Research
 Just
a few examples
 Not exhaustive!
 Still, a tough task to identify wellknown examples across all types
of theory
UIST 2004
18
Explanatory Theory Example
Norman’s seven stage model from POET (1988)
 An approximate model with a continuous
feedback loop








Forming the goal
Forming the intention
Specifying the action
Executing the action
Perceiving the state of the world
Interpreting the state of the world
Evaluating the outcome
UIST 2004
19
Using the model predictively…
 Miyata


& Norman (1986)
Predicted interruptions between task
execution and evaluation as less harmful
when multitasking
Untested
 In
2000, we decided to test this using IM
and multiple tasks

Examined planning, execution and evaluation
phases of tasks
UIST 2004
20
UIST 2004
21
Attention-Based Principles of
Notifications
 Early
in a task was the worst time to
interrupt if you want user to remember
 Make notifications situation-aware

Look for cognitive breakpoints in users’
interactions.
 When


possible, use smart monitoring
Monitor the user (what stage in task?)
Content of interruption—similar is better
• Obvious privacy issues, etc.
UIST 2004
22
Predictive Theory Examples

Large display research (Tan,
Czerwinski & Robertson, 20012003)
• Most early research carried out
around cockpit design
• New hardware often necessitates the
need for new software/interaction
• Serendipitous gender and spatial
cognition findings based on theories
of perception and cognition
UIST 2004
23
Dsharp Display
43"
11"
UIST 2004
24
Prescriptive Theory Examples
 Example:
Gestalt Theory of Perception
(similarity, closure, good continuity,
proximity/figure-ground)
 Example: Feature Integration Theory
(Triesman et al., 80s), guided visual
search and pop out effects
 Utilized well in design guidelines today
UIST 2004
25
Information Visualizer System
Analysis
COST STRUCTURE
OF INFORMATION
Access Costs
Goals
INFORMATION
WORKSPACE
UI Artifacts
ANIMATED GUI
Larger Workspace
Denser Workspace
Interaction Costs
Highly Interactive
Assimilation Costs
Information
Visualization
CASE STUDIES
EXPLOIT HUMAN
PERCEPTION
UIST 2004
3D/Rooms
Interactive Objects
Cognitive Coprocessor
INFORMATION
VISUALIZATIONS
26
•ACT-IF (Pirolli &
Card ’99)
•Theory based on
information
foraging,
sensemaking and
the scatter/gather
approach
•Published in
Psychology Review
UIST 2004
27
Scatter/Gather Document Browser
UIST 2004
28
UIST 2004
29
Generative Theory Examples
 Shneiderman:
“The future of HCI
must be tied to more effective
generative theories that enable HCI to
become the bright shining source of
innovation; a much stronger role than
the usability testers and refiners of
ideas initiated by others.”
UIST 2004
30
Buxton’s 3-State Model of
Graphical Input (1990)

Model inspired
Mackinlay, Card &
Robertson (1991) to
write “A Semantic
Analysis of the Design
Space of Input Devices”
 Hinckley et al. (1998)
extended the ideas to
add notation for
continuous properties
during state transitions
of devices
UIST 2004
31
Pointing and Fitts’ law – The abc’s of user
interfaces (Predictive and Generative)

Fitts’ law
T  a  b log 2 ( WD  1)
UIST 2004
32
Zhai et al.: Laws of actions
 Pointing
 Crossing
 Steering
 Thank
you Shumin!
UIST 2004
33
Accot & Zhai 2002
Crossing – more than dotting the i’s

Why crossing?


increasing interaction
“vocabulary”
Pen based computing

How does crossing
compare with pointing?

What is the theoretical
foundation of crossing?
UIST 2004
34
Accot & Zhai, CHI 2002
Systematic comparison
Target type:
pointing vs crossing
Constraint direction:
collinear vs orthogonal
Continuity:
continuous vs discrete
UIST 2004
35
Accot & Zhai 2003
Crossing-based interfaces
UIST 2004
36
Accot & Zhai 1997, 1999, 2001)
Law of steering
D
Rashevsky 1959, Drury 1971;
Accot & Zhai, 1997, 1999, 2001
W
UIST 2004
37
Law of Steering - General form
TC = a + b IDC
W(s)
d
s
IDC =

ds
W(s)
C
C
UIST 2004
38
Computing off the
desktop

Desktop computing
 HCI Frontier – beyond the
“workstation” interface desktop
foundation
 Interfaces without display


Large and personal display mouse-keyboard tripod
 Numerous difficult challenges
Input device (mouse)
Typewriter keyboard
UIST 2004
39
Zhai, Hunter, Smith 2000; Zhai, Sue, Accot 2002, Drews
Alphabetically Tuned and Optimized Mobile Interface Keyboard
Human Movement Study: Fitts’ law
(ATOMIK)
Word connectivity
N
CI   f (i )c(i )
MT = a + b Log2(Dsi/Wi + 1)
i 1
18000
16000
English Letter Corpus
(News, chat etc)
14000
12000
10000
8000
6000
4000
2000
0
sp E T A H O N S R
I
D L U W M C G Y F B P K V J X Q Z
“Fitts-digraph energy”
t
27
27

i 1
j 1
P ij
IP

 D ij

Log

1


2

W
i



W A  B  e
1
 E
kT
if E  0
if E  0
Metropolis “random walk”
optimization
UIST 2004
z
e  t    (i )( yi  xi )
i a
Alphabetical tuning
40
Limitations and hints from
ATOMIK

Tapping one key at a time – tedious. The stylus
can be more expressive and dexterous.

Does not utilize language
redundancy/statistical intelligence.

People tend to remember the pattern of a
whole word, not individual letters.
UIST 2004
41
Zhai et al.--Shark
UIST 2004
42
Metrics Development—”Subjective
Duration Assessment”
 HCI


and iterative usability metrics
Performance (task times, success rates)
Preference (user satisfaction
questionnaires)
 Usually
correlated, but not always
 Users notoriously “positive” w/ratings
 Neilsen & Levy (1994): need an
average of 5 on a 7 point scale
UIST 2004
43
Ziegarnik Effect (1927)
 People
remember uncompleted tasks
better than completed
 Weybrew (1984) used time estimation


People overestimate time on unfinished tasks
People underestimate time on completed
tasks
 Van
Bergen (1968) task has to be
engaging or more difficult to get the effect
UIST 2004
44
Spool (2001)
 Found
a strong correlation between
perceived download time and whether
users successfully completed their tasks
on a site
 When people accomplish task on a site,
they perceive that site to be fast, and vice
versa
 Goes against emphasis on fast download
times only, need to support user’s task
most
UIST 2004
45
MSN Study—SDA Better Predictor
of Difficulty than User Satisfaction
30
20
10
0
-10
-20
-30
-40
-50
67%
83%
100%
Task Success Rate
UIST 2004
46
So Are We in Good Shape?
 Obviously,
great work has come
out of theory in HCI
 Do we need more?
UIST 2004
47
Why I Think Tomorrow’s HCI Needs
Theory

We’ve come full circle


Theory is one of the most important things we
should be doing as a discipline


Many new tools used both singly and by multiple
people
Not everyone needs to do it, but some of us have to!
A way to guarantee progress

Invite new disciplines to work with us

Cognitive neuroscientists, biologists, ethicists?
UIST 2004
48
How to Do Theory?
 Whittaker,
Terveen & Nardi (2000)
Adopt a reference task agenda
 A set of well-defined “tasks” that candidate
systems could be evaluated against like
TREC
 Even skeptics (Landauer, Carroll et al., Bannon)
agree on the importance of having an
adequate descriptive base for HCI as a
prerequisite for more theory development

UIST 2004
49
Engestrom’s Activity System Model
(1987)

Most human activities are collective ones taking
place in rich social environments
 Model used to explain collective activities and
cooperative work, including cultural conventions
and established rules
UIST 2004
50
Activity Theory and HCI
 Transformation
of tools, rules and
divisions of labor is key
 Utilized more in CSCW community
 Exacty how Activity Theory is
operationalized in studies is a
common problem
 Can provide a framework to
reinterpret the concept of user needs
and society
UIST 2004
51
Challenges to Theory Building
 Advancing
technology
 Influence of individual differences
 Contextual effects
 Longitudinal effects
 New metrics needed

But paradigms exist that can be
leveraged
UIST 2004
52
It’s Good for Your Career!

Top 10 Most Frequently Cited CHI Authors










Stu Card (484 citations)
Bill Buxton (351)
Thomas Moran (344)
Ben Shneiderman (322)
Hiroshi Ishii (298)
Brad Myers (287)
Jakob Nielsen (286)
Allen Newell (222)
Jock Mackinlay (217)
George Robertson (215)
UIST 2004
53
Conclusion
 Benefits
of a theoretical
approach to HCI:





Move our community toward a
solid, scientific discipline
Identify gaps in the design space
Engage in a communal
discussion about design
Career advancement
Innovation
UIST 2004
Holy Grail for
HCI
54
Thank You!
UIST 2004
55

Tom Erickson’s Poem
Theory Theory: A Designer's View
Theory weary, theory leery, why can't I be theory cheery?
I often try out little bits, wheresoever they might fit.
(Affordances are very pliable, though what they add is quite
deniable.)
The sages call this bricolage, the promiscuous prefer menage...
A savage, I, my mind's pragmatic, I keep what's good, discard
dogmatic.
 //
 Oddly now, I'm theory cheery -- I find I have a theory theory!
Neither holy grail, nor deep disgrace, theory's useful in its
place,
(Framing, talking, predicting, bonding, evoking discourse-Others responding)
Like goals and methods, plans and actions, theory's situated,
not pure abstraction.
So make your theory a public way, where passers by may
pause and stay
UIST 2004
56