People, Bleeps, and Clicks
Cognitive Psychology,
Auditory Displays,
and Expertise Acquisition
S. Camille Peres, Ph.D.
University of Houston-Clear Lake
Goal of this Talk
To Accomplish This
• My background and training
• Generally describe my research
– Work done
– Work being done
– Planned projects
• Describe my goals as an educator and trainer
• Not
– a detailed description of experiments
– a specific proposal for future research
Background
• Cognitive Psychology at Rice University
– David Lane and Michael Byrne
– Human computer interaction
• Research
– Applying theories of information processing to the
human-machine interface
– Testing theories of information processing by observing
the human-machine interface
Two programs of research
Bleeps
Clicks
• Auditory Graphs
• Keyboard Shortcuts
• Auditory Progress Bars
• Expertise Acquisition
• Interactive Auditory
Displays
– How it happens now
– How to help it happen
• Time vs. muscle
efficiency
Bleeps
• Benefits of Auditory Displays:
– For those who are visually impaired
– When using a limited visual display
– For situations when the eyes are busy
• Applications of Auditory Displays
– Warnings, Monitoring, etc.
• Types of Auditory Displays
– Alarms, Auditory icons, Sonification
We know a lot about this
• We don’t know HOW we should make
auditory displays
– What dimensions of sound should be used?
– How to combine these dimensions?
– What impacts do aesthetics play?
• Theory of perception
– Integral versus Separable dimensions
– Can we apply this to Auditory Graphs
• Box plots
Visual Dimensions
• Separable
–
–
–
–
Shape
Color
Processed analytically
No benefits of
redundancy
• Integral
–
–
–
–
Brightness
Hue
Processed holistically
Redundancy gains
Auditory Dimensions
• Separable
– Pitch
– Timing
– Redundant
• Integral
– Pitch
– Loudness
– Redundant
Bleeps
• Auditory Graphs
Redundancy gain
with
Integral dimensions
For Identification
and
interpretation of box plots
• Auditory Progress Bars
• Interactive Auditory
Displays
Timing results in
best performance
Bleeps
• Auditory Graphs
• Auditory Progress Bars
In Pilot studies,
increasing pitch
resulted in best performance
– Auditory display for “On-hold”
environment
– Could people accurate estimate
time on hold?
People did NOT
like the sounds
natural
sounds
did not
about
using
– Does sound dimension impactUsingWhat
improve
subjective
ratings
natural
sounds?
performance?
– What about aesthetics?
– Prospective vs. retrospective
memory
• Interactive Auditory Displays
Using
principles
of
Enter
the
composer…
musicabout
composition
What
utilizing
didprinciples
improve of
subjective
ratings
composition
Clicks
• People plateau at sufficiency with software
– Why?
• Some people do become efficient/expert users
– Why?
• Use of the keyboard to issue commands is a
good proxy for efficiency with software
– Easy to learn, takes less than 1/2 the time, most
people know them
Predictor of efficiency
• Survey data
• Ethnographic study
• Experimental data
Peer Support in the
Working Environment
• Enter, “the Lean Move”
– Forward lean, “Let me show you how to do it.”
– Backward lean, “Hey, do you remember how to do it?”
Why do we care?
• We can utilize this for training
– Have people who are co-located trained as a cohort
• We can further our understanding of cognitive
processes.
– Why does observing someone change our “selection
rules”? (most people know about keyboard shortcuts, they just
don’t use them)
Human as Information Processor
Long Term Memory
Working Memory
Visual store
Auditory store
(iconic memory) (echoic memory)
Eyes
Ears
Perceptual
processor
Motoric
processor
Fingers, etc.
Cognitive
Processor
Modeling Human processing
• GOMS (Card, Moran, and Newell, 1983)
– Goals - what do you want to do?
– Operators - tools that human processor has available
(perception, cognition, motor response)
– Methods - sequence of steps that use operators to
accomplish goals
– Selection rules - algorithm for choosing which method
to use
• GOMS allow us to predict how long it will take
someone to do a task.
Informing a theory
• Selection rules not always robust
– Findings from this research will be used to test how
observation and social effects impact the methods
people select.
• Next steps
– Relation between time efficiency and muscle
efficiency
– Can we add a new parameter to the GOMS model
• Specifically, can we predict the amount of muscle
activity used to complete a task as well as the time?
Increasing the numbers…
• More jobs than people to
do them in STEM fields.
• Women and minorities
not in STEM.
– These folks want to “work
with people”
– HCI and Human Factors as
a bridge
Integrating this knowledge
• Have Human Factors
specialists do this work
• Train Engineers how to utilize
principles of Human Factors
• When do you do one?
• When do you do both?
• Which is better for business?
• Which is better for outcomes?
• Can we have these both be
the same?
Finally
• Human Computer Interaction/Human
Factors
– Not just about the display
• User Centered Design is foundation of all
good HCI/HF
– Is the device, interface, process designed so
the human can perform the task
• Easily, safely, & efficiently
• The goal of my research is to inform the
UCD process and improve the human
experience.
Collaborators
Magdy Akladios
Phil Kortum
Kurt Stallman
Michael Fleetwood
Frank Tamborello
Aniko Sandor
David Lane
Al Napier
Vickie Nguyen
Ashitra Megasri
April Amos
Now, let’s go
to the Moon!
Questions…