Use-inspired basic research:
Marrying humans & technology
Frank Durso
Georgia Tech
BASIC
APPLIED
Science and
application are
bound
together like fruit
and the tree that
bears it.
The opposite
of a profound
truth may
well be
another
profound
truth
Without
theory,
practice is but
routine born of
habit
Peterson?
Backoff
man, I’m a
scientist
If I find
10,000 ways
something
won’t work,
I haven’t
failed…every
wrong
attempt
discarded is
a step
forward
Pasteur in his quadrant
• Basic: tartaric vs. paratartaric acid
– all organic crystals rotate light
• Applied: Contamination in fermentation
– Crystals rotate light fermentation was organic (Germ
theory)
• Basic: Germ theory  experiments
– Swan necks & beef broth
• Applied: beer, wine, silkworms, pasteurization,
antiseptics
• Basic (Analogy): Fermentation::contagious disease
• Applied: Vaccinations
Wright brothers in the quadrant
• Applied: Glider tests
• Basic: Experimental data (lifting tables) of the
time were incorrect
• Basic: Experiments
– Built a wind tunnel to use for aircraft design
• Applied: The Flyer at Kitty Hawk
The life of Besse Cooper
The rise of the machines
TECHNOLOGY CHANGES
Happy Birthday Besse
Model T Ford - 1908
Buick - 2011
1908 Model T vs. 2011 Buick
•
•
•
•
•
Model-T Buick
Weight (lbs)
1,200
3,671
Horsepower
20
220
Max speed (mph)
30
120
Gas mileage (mpg)
25
23
Cost (2008 $)
20,000 30,000
ENIAC - 1945
PDP-1 Computer - 1960
Apple’s Ipad - 2011
ENIAC vs. PDP-1 vs. Ipad
(Nos. approx.)
ENIAC PDP-1
Date
1945
1960
Weight
27 tons 1,200 lbs
Space (sq ft)
680
30
Mem capacity
--10 KB
Speed (add/sec)
5K
100K
Cost (2011 $)
6,000,000 875,000
Cal/sec for $1k (2011 $) 0.8
115
Ipad
2011
1.3 lbs
<1
64 GB
1Ghz
700
1.4G
Putting the rate of change in information
technology in perspective
 “An ordinary notebook PC can run a database that
is more powerful and almost 100 times as large as
that of a major bank of the 1980s”
 If the cost of an automobile had changed at the
same rate as the cost of computing over 100 years,
one would be able to buy about 10 million 2011
Buicks for about 3 cents.
 If the speed of an automobile had changed at the
same rate as the speed of computers over 100
years, an automobile would have attained the
speed of light around 1965.
THE HUMAN-TECHNICAL SYSTEM:
AN IRRESISTIBLE ATTRACTION?
Technology is attractive
 Promise of faster, better, cheaper; makes life
easier; makes us safer
 Technology impacts everything
 Technology centric: Because we can
Have cold feet?
• Resistance to new technology
– Luddites
– Samurai
– Bicycle face
• What is being replaced?
– Tradeoffs
– Side effects
– Evolution
Marrying humans and technology
The solution to every major societal problem of the next 50 years will
require an understanding of how people think
WHY COGNITION?
The jobs are cognitive
The people are cognitive
The problems are cognitive
•
•
•
•
•
•
•
•
Situation awareness and hazard detection
Information overload in air traffic control
Low altitude military combat
Automation and out of the loop
Medication errors
Managing chronic disease
Training health care workers
Global warming
The solutions are cognitive . . .
Tower Simulator
Situation dimension principle
Dimensions of situation understanding
Ground Control
Local Control
Unclassified, 11
Time, 25
Unclassified, 16
Intentionality, 78
Environment, 3
Time, 25
Intentionality, 219
Controller Memory, 7
Space, 326
Controller Memory, 3
Space, 180
Motion, 21
Protagonist, 286
Protagonist, 455
Integrated display
Reverse engineering
• Chase & Simon
Some displays increase information
complexity, some do not
Mean Number of Glances
Mean Number of Glances by Condition
350
300
250
200
150
158.67
243.17
253.50
cpdlc
tmacpldc
178.67
100
50
0
standard
tma
Condition
Practice reduces information
complexity
Mean number of glances
12
10
8
STATIC
DYNAMIC
6
4
2
0
TRIAL 1
TRIAL 2
Threat & Error Strategery
•
•
•
•
Identify threat
What strategy would you use?
What cues suggest that strategy?
Participants
– Children’s intensive care nurses
– Airline pilots [automation]
– Air traffic controllers
– Locomotive engineers
PICU Nurses
AUGMENTED COGNITION
What is the state of the operator?
 Modeling the state of the operator
 Monitoring the state of the operator









Auditory P300  differential attention
EEG , heartrate variability  workload
Heart rate  arousal
Pupil diameter  cognitive load
Eyelid droop  fatigue
Posture  imminent action
Saliva  stress
Facial expression  emotion
fNIR  retrieval v. encoding
MB’s S-car
AugCog and loss of SA
• EMG
• But how can we make sure they’re confused
Watching a parade from the 40th floor
• The view was breathtaking. From the window one
could see the crowd below. Everything looked
extremely small from such a distance, but the colorful
costumes could still be seen. Everyone seemed to be
moving in one direction in an orderly fashion and there
seemed to be little children as well as adults. The
landing was gentle and luckily the atmosphere was
such that no special suits had to be worn. At first there
was a great deal of activity. Later, when the speeches
started, the crowd quieted down. The man with the
television camera took many shots of the setting and
the crowd. Everyone was very friendly and seemed
glad when the music started.
Participant Self-Report
Confused
Not Confused
Increase in amplitude
N=5
Right Corrugator supercilii
Left Corrugator supercili
Right Depressor anguli oris
No increase in amplitude
Electromyographic Activity
N = 11
N=4
N=4
Figure 4. Z-score transformed electromyograms for the experimental passage, classified by participant response and EMG activity. The x-axis of each graph is time in seconds and
the y-axis is standard deviations.
Leave the lab
Realize where you are and why you’re there
HOW TO PLAY IN PASTEUR’S
QUADRANT
Leaving “the lab”
• I was a control freak
The broader context
• I thought
cognitive
psychology
was all there
was
– Environment
– Socio-political
The QWERTY story
In which quadrant are you?
Never express
yourself more
clearly than
you are able
to think
No job is too
big, no fee is
too big
Chance favors
the prepared
mind
Hell, there
are no rules
here, we’re
trying to
accomplish
something
Thanks
Others in Pasteur’s Quadrant
•
•
•
•
•
Jim Staszewski mine sweeper
Frank Drews anesthesia
Penny Sanderson sonification
The Habers—low altitude military combat
Many more
Thanks
Help with abduction
• Looked for a rich literature to help me explain how operators in dynamic
environments understand their situations
• Reading comprehension
– Strengths
•
•
•
•
Detailed processing structures underlying comprehension in dynamic environments
Plethora of clever methodologies that could, in principle, be brought into dynamic realms
Measurement techniques and discussion of strengths and weaknesses
Context to classify different approaches to SA (e.g., cognitive vs. cognitive engineering)
– Limits
•
•
•
•
Dynamic
Input
Reason for understanding is different
Strategies
• Information needs & search
• Information complexity
• Detecting confusion