How do you know what’s going on in a noisy
and uncertain world?
When the only available information is
indirect, what mistakes might you make?
Safety-critical psychology
Part II
Airport security
Friendly Fire
Signal detection: A fundamental
problem
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Signal Detection Problem
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Medical lab work as Signal Detection
Indication of Condition
Test Result
More sensitive
Present
Present
Hit
Absent
Present
Miss
Condition
Hit
Absent
False
alarm
Miss
Disease
Absent
False
alarm
Correct
rejection
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Airport security checkpoint
More specific
More specific
Present
More sensitive
Absent
Correct
rejection
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Airport Security Signal Detection Problem
Indication of Condition
More sensitive
Present
Present
Hit
Absent
False
alarm
Absent
Miss
Condition
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
More specific
Is one of these people carrying a dangerous object?
Correct
rejection
Fatigue and lack of vigilance
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
1
Airport Security Signal Detection Solution (NOT!)
Inserting test false-alarms
Indication of Condition
Present
Absent
Hit
Miss
Present
More sensitive
Present
More specific
Present
Hit
Absent
False
alarm
Absent
Miss
Condition
Absent
False
alarm
Correct
rejection
Condition
Indication of Condition
More specific
Reaction to vigilance enhancing alarms
More sensitive
Zero tolerance for mistakes
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Correct
rejection
Vigilance enhancing alarms
Fatigue and lack of vigilance
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Battlefield Intel as Signal Detection
Sensor Indications
More sensitive
Present
Friendly fire in a military that is
trading tooth for tail
The case of the USS Vincennes
Some additional conceptual tools
Hit
Absent
Friendly
Fire
Correct
rejection
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
A key question for Human-Tech system
design
What information
goes where
when and
in what form?
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Get
Whupped
Present
Hostile
Enemy
More specific
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Absent
Measurement
Trajectories/pathways
Timing
Representation
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
2
This ship (USS Vincennes)….
On July 3, 1988….
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
… destroyed this civilian airliner.
How could this happen?
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Political context: 1988 the “tanker” war
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Political and Organizational context
US backs Iraq and Saddam Hussein
US Navy ships are assigned to escort oil
tankers through the Persian Gulf
Prior naval battle with Iranian forces
Vincennes is sent to the Persian Gulf
Rules of engagement (ROE) are changed
–  Preemptive strikes are authorized
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
3
Political and Organizational level factors set
the stage
Strait of Hormuz: strategic chokepoint
Tanker war
Tense international relations
Bandar Abbas
Joint use airport
Change in rules of engagement
Emphasis on protecting US
personnel
Dubai
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
AEGIS automated battle system
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Automation in warfare
“The system gives the ship an unusually clear
two-dimensional (range and bearing) picture
of all air activity out to a considerable
distance, and it automatically computes
engagement orders [commands to fire
weapons] for all the targets it tracks. That
makes it possible for the ship to engage any
of the targets with little delay, once the
target has been identified as
hostile.” (Friedman, 1989 Proceedings of the Naval
Review).
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Combat information Center on Vincinnes
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
4
Condensed chronology
1.  Vincennes and Montgomery engage small
boats in a gun battle
2.  Iran Air 655 takes off from Bandar
Abbas, 40 minutes behind schedule
3.  New air track identified, IFF ambiguous
(mode II/military or mode III/civilian)
4.  Vincennes issues a warning
5.  IDS announces “F-14”; AEGIS records
mode III/civilian
What happened?
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Condensed chronology II
6.  Forward gun jams, Vincennes makes a
sharp turn at high speed; things topple in
CIC
7.  Vincennes issues more warnings
8.  IDS reports target descending (AEGIS
records climbing)
9.  When Iran Air 655 was 10 miles from
Vincennes at an altitude of 13,500’,
Vincennes launched two surface to air
missiles which destroyed the airliner.
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
How can we use our growing
understanding of the cognitive
consequences of technology to
understand what happened that day in
July of 1988?
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Battlefield Intel as Signal Detection
Sensor Indications
More sensitive
Present
Present
Absent
Hit
Get
Killed
Hostile
Enemy
Absent
Friendly
Fire
Correct
rejection
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
1.  Vincennes and Montgomery
engage small boats in a
gun battle
2.  Iran Air 655 takes off
from Bandar Abbas, 40
minutes behind schedule
3.  New air track identified,
IFF ambiguous (mode II/
military or mode III/
civilian)
4.  Vincennes issues a warning
5.  IDS announces “F-14”;
AEGIS records mode III/
civilian
Fog of war
Cannot match published
schedule
Clumsy interface, poor action
specification, slips invited
Principle of least commitment
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
5
6.  Forward gun jams,
Vincennes makes a sharp
turn at high speed; things
topple in CIC
7.  Vincennes issues more
warnings
8.  IDS reports target
descending (AEGIS records
climbing)
9.  When Iran Air 655 was 10
miles from Vincennes at an
altitude of 13,500’,
Vincennes launched two
surface to air missiles
which destroyed the
airliner.
Unanticipated interactions
To access info on a track
Confirmation bias? Interface
design?
If cursor on track,
read CRO display
Otherwise,
put cursor on track
read CRO display
Tight coupling, complexity
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Cursor on Bandar Abbas,
Not on TN 4131.
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
A key question for Human-Tech system
design
Principle of least commitment
What information
goes where
when and
in what form?
CO
TAO
Measurement
Trajectories/pathways
Timing
Representation
TIC
IDS
AAWC
“F-14”
ARC
ACS
AIC
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Smart Weapons
Consequences of Smart Weapons
Trading tooth for tail.
First feedback loop - the value of personnel
Technology “frames” situations
–  Smart weapons require smart people to maintain and operate
them.
–  Smart people are too valuable to be put at risk near combat
zones.
–  The best way to keep people away from combat zones is to
make and use smart weapons.
Second feedback loop - the cost of personnel
–  Smart people are expensive to acquire and train. Military must
compete with other users of smart people, so the military
cannot afford so many of them.
–  Fewer people makes the ones you have more precious - can’t
afford to replace them.
–  The best way to protect the human assets is to keep them away
from combat.
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
–  What are the available categories of events?
–  How are events assigned to categories?
The Vincennes incident was an example of ‘failure of
representation’, not equipment failure and not human
error!
–  The inability to form an independent interpretation in an
environment created by the new equipment.
Tighter coupling increases the pressure to strike
first.
–  He who shoots first is the only one who gets a chance to shoot.
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
6
With Hi-Tech weapons
Techno War
Situations develop more quickly (tighter coupling).
Smart weapons shift the control of war from the
warrior to the automated system
Changing the nature of military work
–  There is little time for reflection.
The representations of situations provided by
equipment is complex and difficult to interpret
(complexity).
–  Confirmation bias is present in humans.
–  Technology frames the situation.
There is great pressure to act sooner rather than
later.
–  It may be necessary to act before an unambiguous
interpretation of the situation can be formed.
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
–  Moving the warriors away from the battlefield
–  Army without soldiers, navy without sailors, airforce without
pilots
Technology creates representations and
representations frame decisions
But
–  Complexity makes it difficult to know what is happening
–  Tight coupling and the efficiency of weapons increase pressure
to shoot first
Technology MUST provide the right representations
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
Techno War
Good news:
–  No more wars of attrition
Bad news:
–  loss of control over war process
–  the “fog of war” gets denser
–  fatal system accidents (e.g. Vincennes)
Except where otherwise noted, these slides are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license
7