Diploma in Aviation Medicine:
Human Performance
Revision
June 11
Objectives of Aviation Psychology
•
to enhance flight safety
•
to improve effectiveness
Why is There Interest in Human Performance in Aviation?
• Aviation is a safety-critical operation
•
•
•
Aircrew are subjected to many sources of stress
High levels of human performance must be achieved (e.g., fast jet pilots)
Human error is heavily implicated in aviation accidents
Main fatal accident causes in 1994
Accident cause
Air crew error
Controlled Flight into Terrain
Weather
Loss of Control
Engine Failure/Fire
Structural/System Failure
Accidents (%) Fatalities (%)
65
37
30
14
16
11
68
43
24
34
7
13
The Human in the Aviation System
Equipment
& tasks
Environment
Human
operator
Other
personnel
Content of Human Performance
Module
General Principles
• Introduction to Human Performance Module
• Fundamentals of Human Performance
• Individual Differences
• Social Psychology and Aviation
• A small amount of basic theory to help you to
interpret the practical studies
Content of Human Performance
Module
New this year!
• An early session on human error
• …to provide a context for the module
Content of Human Performance
Module
Personal & Environmental Factors
 Stress & Workload in Aviation I
 Stress & Workload in Aviation II
 Perceptual Issues in Aviation
 Situation Awareness
 Selection of Aviation Personnel
Content of Human Performance
Module
Training and Simulation
 Simulation and Training
 Fundamentals of CRM Training
 Practical Aspects of CRM & LOFT
Content of Human Performance
Module
Systems Factors
 Aviation Ergonomics I
 Aviation Ergonomics II
Content of Human Performance
Module
The Human Factor in Aviation Accidents
Seminar: Flight Safety
– Prof Peter Jorna, former head of division at NLR
Amsterdam
Also an accident module at Henlow, providing a context
for this module
See also lectures on Sleep, Fatigue and Shift-Working
The Human in the Aviation System: Relevance of Module Topics
Equipment
& tasks
Environment
Human
operator
Selection
Individual
Differences
Other
personnel
The Human in the Aviation System: Relevance of Module Topics
Ergonomics
Equipment
& tasks
Environment
Workload
Technical
Training
Human
operator
Other
personnel
The Human in the Aviation System: Relevance of Module Topics
Equipment
& tasks
Environment
Perception
Human
operator
Other
personnel
Stress
The Human in the Aviation System: Relevance of Module Topics
Equipment
& tasks
Environment
Human
operator
CRM Training
Other
personnel
The Human in the Aviation System: Relevance of Module Topics
Equipment
& tasks
Environment
‘Situation
Human
Awareness’
operator
and ‘Human
Error’ encompass
all these interactions
Other
personnel
Human Information Processing
Cognition
Processes involved in the input, storage, transformation, and output
of information by humans
Main topics:
• Memory
• Attention
• Skills
Memory
Three major memory systems:
•
Sensory memory
•
Short-term memory
•
Long-term memory
Summary of properties of memory systems (inferred from classic
experiments on memory) learn this!
Sensory
Short-term
Long-term
Capacity
High
72 chunks
no known
limit
Duration
1 sec
10-15 sec
permanent?
Type of storage
Physical
characteristics
acoustic
semantic
Nature of retrieval
parallel
serial
hierarchy?
Nature of forgetting
decay/masking
interference
failure of
retrieval
Attention
Selective attention:
attend to one of several competing sources of
information
Divided attention:
attempt to attend to more than one information source
or task at the same time
Shadowing task
Can detect physical changes on the unattended channel, but not semantic content
Dichotic listening task
Performance very poor
Subjects tended to organise their recall by ear, not by recency of presentation
Led Broadbent to propose Filter Theory. But, later shown that
• subjects tend to hear their own name on the unattended channel
• subjects tend to follow the message, even if it switches ears
• hence, there is semantic processing on the unattended channel
However, we can assume that recognition of unattended information is
less likely than recognition of attended material
Make sure that you understand this!
Divided attention
Key question:
Does man have a single information-processing channel
(all tasks compete for the same ‘resources’ or ‘capacity’)
or
specialised resources for particular types of activity?
(tasks performed concurrently compete only if they draw upon
the same resources)
Some support for the multiple resource theory: often, the degree of
task interference depends upon the similarity of the tasks
But: sometimes tasks that are dissimilar are found to interfere
Baddeley’s working memory model
is a compromise between extreme
single-channel and multiple-resource
views
Visuo-spatial
scratchpad
Central Executive
Articulatory loop
Skills (obviously relevant to training lectures!)
Characteristics
• typically a sequence of activities
• goal-directed behaviour
• use of feedback
Skill acquisition
Three phases are sometimes distinguished:
• Early or cognitive phase
• Intermediate or associative phase
• Final or autonomous phase
In the final phase, behaviour becomes automatic; delegated to the
control of ‘motor programs’ that do not require conscious attention
and do not place heavy demands for mental resources
Many everyday errors (actions not as planned) are associated with
overlearned behaviour
These errors involve well-practised behaviour, but are inappropriate
Some aircrew errors are of this type
Issues in skill acquisition
• Whole versus part learning
• Massed versus spaced learning
• Transfer of training very important aspect of simulator-based training
Individual Differences
Two major types of individual difference covered:

Intelligence/ability/aptitude

Personality
Factor Analysis:
make sure that you have a good intuitive
grasp of this: you don’t need to know the underlying
mathematics!


Basic psychometric criteria
Intelligence/ability/aptitude These are the key issues

Intelligence: Innate or learned?

Intelligence: How many abilities?
•
•
There is evidence for a general ability factor
However, specific abilities also appear to exist

Intelligence: The Intelligence Quotient (IQ)

Aptitudes

Test Fairness

Intelligence: Are IQ tests valid?
Personality key issues

Types of personality test:
 Interview (not reliable)
 Projective tests
 Personality questionnaires
 (discussed examples of each)

Is there a ‘pilot personality’?

Does personality influence success in flying training?

Is there an ‘accident-prone’ personality’?
Social Psychology
Types of social influence (can use this info for CRM questions)
Compliance: behaviour consistent with direct request
• foot-in-the-door phenomenon
• door-in-the-face phenomenon
Conformity: behaviour consistent with group norms
• size of group (up to about four)
• attractiveness and status of group members
• Informational influence (trusting others’ judgements) and normative
influence (seeking group acceptance)
Obedience to authority
• Milgram experiment
• 62.5% of the 40 subjects administered shocks to the highest level
• factors affecting obedience, such as status of experimenter,
proximity to ‘student’
Think about how this applies to small groups of interest to us,
such as flight crew or teams of maintenance engineers
Group Decision Making: Polarisation
•
Was thought that group decision making was more risky
than individual DM (‘risky shift’) — but became apparent
that there is a shift in the direction of the pole that, on
average, the group favours as individuals (polarisation)
•
Stoner’s experiments
•
Normative and informational influences produce group
polarisation
Group Decision Making: Groupthink
•
•
•
Work of Janis. Based on real-life examples such as Bay of Pigs (or, more
recently, UK MPs’ expenses!)
Desire for consensus overrides group members’ motivation to assess risk
and consider alternative courses of action
Groupthink occurs under the following conditions:
•
•
•
•
•
•
Symptoms include
•
•
•
•
•
•
High cohesiveness of the group
Uncertainty of approval
Insulation of the group
Directive leadership
High stress situations
Illusion of invulnerability
Stereotypes of out-group
‘Mindguards’
Direct pressure on dissenters
Collective rationalisation
Effects on decision making:
•
•
•
•
Incomplete survey of alternatives
Incomplete survey of objectives
Incomplete analysis of risks associated with course of action
No contingency plans
Aeronautical Decision Making (ADM)
•
Work of Jensen: decision error is cause of most fatal aviation accidents;
argued that decision making can be improved through training
•
Decisions have two components:
•
•
•
Hazardous attitudes:
•
•
•
•
•
•
Anti-authority
Resignation
Impulsivity
Invulnerability
Macho
ADM courses aim to provide:
•
•
•
•
Rational judgement (‘Headwork’)
Motivational judgement (‘Attitudes’)
Ability to recognise hazardous attitudes
Knowledge of effects of these attitudes
Skills to overcome the effects
Methods
•
•
•
Self assessment tools
Examination of case studies
Practical Exercises
Perceptual Issues in Aviation
Perception is the process of acquiring, selecting,
and organising sensory information
The most important perceptual processes for aviation
are those associated with vision and hearing
1
The ear and the auditory system
– balance and the vestibular system
– localisation of sound and identification of source
2 The visual system
– bottom-up processing
– top-down processing
– cues to depth perception
The ear and the auditory system
The ear serves two main functions:
 Balance. The vestibular system of the inner ear
detects angular and linear accelerations of the head
 Hearing. To detect sounds, to determine the location
of their sources and to recognise the identity of these
sources
Balance and the vestibular system
Practical implications
With regard to the otolith, the weight force in a climbing aircraft
operates similarly to the resultant force in an accelerating aircraft.
Without visual feedback, pilots can mistake acceleration for pitch.
Ascending
aircraft
Accelerating aircraft

Weight force
Inertia force
Resultant
The situation is aggravated if the pilot attempts to compensate for an
incorrect percept. Although feedback from the vestibular system can
be compelling, a pilot needs to learn to trust instrumentation.
Auditory perception
Localisation of sound
Interaural differences:
•Intensity. Most suited to localising
high frequencies
•Time/phase. Most suited to
localising low frequencies.
Sounds emanating from directly in front and behind
the head produce the same interaural differences.
Auditory perception
Practical implications
 Cockpit design
– The cockpit relies heavily on the presentation of
visual information. Adoption of auditory signals may
reduce the workload experienced by pilots in the
visual domain.
 Localisation of auditory warnings
– Similar sounding warnings emanating from similar
areas may cause confusion
– Adoption of white noise bursts within ambulance
sirens
Visual perception
What you see is what you get?
Visual modality is obviously extremely important in aviation. But can we
always trust our eyes?
The visual scene is captured by the eye as a poor quality, two-dimensional
representation
What is perceived is determined by:
 ‘Bottom-up’ processes. The percept of a stimulus is determined by
features of the stimulus as processed by the visual cortex
 ‘Top-down’ processes. The interpretation (consciously or not) of a
stimulus can be determined by our experience and knowledge
Important distinction!
Visual perception
Top-down processes
 If the percept is generated
deterministically (bottom-up
processing) from the visual
cortex . . .
Necker cube
Thirteen or ‘B’?
 . . . how can one distal (real
world) stimulus produce two
percepts?
 By a mental model: Our own
experience and expectations
help to determine what we see
(top-down processing)
Lincoln or women?
Old or young women?
Visual perception
Depth perception: learn this!
 Convergence
– of the eyes.
 Stereopsis
– disparity between the
two images.
 Accommodation
– of the lens.
 Retinal versus actual size
– for known objects.
 Overlap
– a near object will
occlude the view of a
far object.
 Position in visual field
– objects nearer the horizon
are farther away.
 Aerial Perspective
– clarity of objects is reduced
at distance.
 Relative motion
– angular velocity greater for
near objects.
All require both bottom-up & top-down
processing.
How we perceive depth
•
Position in visual field
•
•
Textual Gradient
•
•
Occlusion
•
•
Surfaces will have a finer texture with
distance
Retinal versus actual size
Relative motion
•
Binocular disparity between the two
images
a near object will occlude the view of a
far object
Perceptual constancy
•
•
Stereopsis
•
•
•
objects nearer the horizon are farther
away
angular velocity greater for near objects
Convergence
•
of the eyes
•
44
Know this
All require both bottom-up & top-down
processing.
Visual perception
Some perceptual problems
 Featureless surfaces, or those with textures of unknown
sizes, can produce inaccurate judgements of size.
– Sea.
– Beehives for caravans.
 Can produce an inaccurate mental model of the situation
which overrides the correct perception of the instruments.
– Top-down influences.
– Exacerbated by fatigue and workload.
Visual perception: know the practical implications (next few slides)
Practical implications: visual approach
 Pilots may have to visually judge the glide slope without
any cues other than those from the surface of the world.
 The ‘aspect’ (retinal shape) of the runway is not very useful.
 However, the visual touchdown point is a constant and
unchanging cue, relative to the horizon.
 If the horizon cannot be seen, its location must be implied,
– The runway’s sides meet at the horizon.
– The terrain’s texture gradients.
– The relative position of the aircraft’s canopy.
Visual perception
Practical implications: visual approach (2)
HORIZON
B
Visual
touchdown
point
A
Angle of Approach A = B
Visual perception
Practical implications: visual approach (3)
Visual
impact
point
Actual
touchdown
point
 = angle of approach


Visual perception
Practical implications: mid-air collisions
 Identification of a colliding aircraft is confounded by;
– Constant relative bearing.
• Unique characteristic.
• Periphery of retina detects sensitive to movement.
– Non-linear increase in retinal size.
• Retinal image doubles with each halving of closure
distance.
– Uneven visual acuity across the retina.
• Maximal acuity at the fovea.
• Detection only if pilot is looking directly at it.
• Implications for visual scanning to acquire proximal
image on the fovea.
Visual perception
Practical implications: mid-air collisions (2)
Impact


Aircraft A
Aircraft B
Visual perception
Practical implications: mid-air collisions (3)
3 secs / 0.5 degree
1.5 secs / 1 degree
0.1 secs / VERY BIG
Perception
Summary
 Bottom-up (information from our senses) and top-down (expectations
and experiences) processes affect the way we perceive the world.
 The resultant perception is often not a true reflection of the external
world.
 This can be advantageous when it is in our interest for differences
between features in the external world to be exaggerated but potentially
catastrophic when perceptual illusions lead us to take inappropriate
behaviour.
Ergonomics
Why ‘Ergonomics’? – Murrell
The HSI Framework – seven domains
–
–
–
–
Manpower
Personnel
Training
Human Factors Engineering (aka Ergonomics)
• Workplace design
• Anthropometry
• Critical Dimensions
– System Safety
– Health Hazards
– Social & Organisational
Some people adopt a strict
definition of ergonomics;
others treat all of HSI as
being within the scope of
ergonomics. You would not
be penalised for adopting
the latter definition!
 HSI often called Human Factors Integration (HFI) –
HFI is really the process by which HSI is applied to
equipment procurement
Tragic consequences
USS Vincennes
Kegworth
Chernobyl
Three Mile Island
Herald of Free Enterprise
HSI Domains KNOW THESE
Manpower: numbers of personnel required to operate, maintain, sustain, & train to
deliver capability (e.g. aircrew complement)
Personnel: cognitive/physical capabilities required to train for, operate, maintain, sustain
system
Training: instruction/education/ training to provide job skills, knowledge, values, and
attitudes (different methods summarised)
Human Factors Engineering (aka Ergonomics)
Workplace design
Anthropometry
Critical Dimensions
Systems Safety: applying HF expertise into programme Safety Management Process
Health hazards: conditions inherent in the system that may cause injury or reduce
performance or well-being
Social/organisational factors: applying techniques from organisational psychology,
social sciences, information science, and system of systems
Human Factors
Engineering
(aka Ergonomics)
focused on the integration of human characteristics
into system definition, design, development, and
evaluation to optimise human machine performance
under operational conditions.
Workplaces & interfaces
Cockpits
Workstations
Control rooms
Offices
Transport systems
Factories
Controls
Displays
Computer hardware
Computer software
Protective clothing
Other people
Physical workplace design
 Inputs required from
– EHFA
– Task analysis
– Link analysis
– Allocation of function
 Consider
– Operational and environmental context
– Human dimensions
– Biomechanics and physiology
Functional factors
 Task issues
– Procedures
– Critical elements
 Responsibilities of organisation and individuals
 Communications
– Verbal
– Non-verbal
 Visual issues, such as sight lines
 Flows of materials and personnel
 Access and clearance
– Normal
– Emergency
– Maintenance
 Protection
– Protective clothing & equipment
– Barriers & guards
Anthropometry
 Physical human dimensions
 Population specific
 Linear dimensions, for example:
– Stature
– Functional reach
– Sitting height
 Girth dimensions, for example:
– Waist
– Head circumference
 Each dimension is expressed in terms of percentile
Be careful with percentiles when applying
anthropometry
 Requirements often state …must accommodate
the 5th percentile and the 95th percentile
human…
 But, these people do NOT exist!
Critical dimensions
Choose dimensions relevant to the
workstation, posture, and task
– Sitting, standing, reach, fit, walking, crouching
5th percentile (smaller) dimensions
considered for:
– Seat adjustment, reach, vision, control movement,
foot rests
95th percentile (larger) dimensions
considered for:
– Seat adjustment, ingress, fit, access, clearance
Clothing
 Clothing increases most dimensions through the
addition of bulk
– e.g. stature, sitting height, chest depth, shoulder
breadth
But
 Decreases the reach dimensions due to restriction of
movement
– e.g. functional reach, vertical functional reach
When to integrate Human Factors
(Eurocontrol, 1999)
HSI “Designed” to Fit MoD’s Acquisition Operating
Framework (AOF) Policy and Good Practice
 CADMID cycle
 System Readiness Levels
(DEF STAN 00-250. May 2008; http://www.aof.mod.uk ; www.hfidtc.com )
 MoD JSP to be introduced later this year
Has now happened
Concept
Assessment
Initial Gate
Demonstration
Main Gate
Manufacture
In service
System Acceptance
Disposal
Summary
 HSI covers all aspects of applied human factors
 Human Factors Engineering is just one element that
needs to be integrated
 HSI comprises tools and processes that fit with
systems engineering
 HSI is widely applicable
 Early inclusion is so much better than late
intervention
Stress and Workload
Types of stress
• Life stress
less important than the others in this context, but be aware of it
• Environmental stress
• Cognitive stress
Life stress
Typically measured by questionnaire
Some correlation between questionnaire scores and illness
Some evidence that life stress is associated with accidents
Yerkes-Dodson law: Know this
Environmental
stress
Inverted U relation between arousal and performance
Performance declines as arousal increases or decreases
from the optimal level
Performance
The optimal arousal level is inversely related to task
difficulty
Difficult
task
Easy
task
Level of arousal
Know this
Fear
• disruption of manual dexterity
• disruption of secondary task performance
Noise
• greater effect on difficult tasks
• effect on error
• increased attentional selectivity
• effect on arousal (increases initially, then returns to normal)
Sleep loss
• periodic lapses
• decreased attentional selectivity
• greater decrement on ‘easy’ tasks
• decreased arousal
Hypoxia
• performance affected at over 10,000 ft
• some evidence that task learning is affected at only 8,000 ft
Combined stressors
Sleep loss and noise each impair performance in isolation
However, noise improves the performance of sleep-deprived individuals
Patterns of effects of stressors (from Hockey)
Stressor
Arousal
Selectivity
Speed
Accuracy
STM
Noise
Anxiety
Incentive
Stimulants
Heat
Alcohol
Sleep loss
Fatigue
Depressants
+
+
+
+
+
-
+
+
+
+
+
+
+
-
0
0
+
+
0
-
+
0
-
+
0
0
0
-
+
0
increase
decrease
no effect
no need to memorise all this, but know that each stressor has its own
pattern of effects (cannot be explained by Yerkes-Dodson law)
Personality and stress
Know this
Two major dimensions
of personality are:
neuroticism (trait anxiety)
introversion-extraversion
Introverts are chronically over-aroused
Extraverts are chronically under-aroused
An arousing stressor (caffeine) has different effects on these individuals
Trait anxiety comprises worry and emotionality
Worry appears to interfere with task performance
Performance of high-anxiety subjects impaired under high workload
Evidence that personality influences success in flying training
Is the Yerkes-Dodson law adequate? Know this
For:
• Can explain effects of combined stressors
• Can explain some effects of personality
• Can explain some effects of task difficulty (e.g. greater effect of sleep
loss on easy tasks)
Against:
• Does not explain specific patterns of effects of individual stressors
• Does not explain effects on attention
• Too flexible: does not lead to firm predictions
Factors influencing the effects of stressors know this
• Task difficulty
• Task duration
• Personality
• Intensity of the stressor
• Motivation
• Importance of the task component
• Presence of other stressors
Workload (cognitive stress) know this
Types of workload measure
Subjective
Example: NASA Task Load Index
• easy to obtain
• face valid
• unobtrusive
• subjects can readily quantify their
experience
• difficult to establish which questions to
ask (dimensions of workload)
• difficult to compare different types of task
• ratings may not be correlated with task
performance
Physiological
Example: heart rate variability
• do not disrupt performance
• often provide continuous record
• equipment may be physically intrusive
• only indirect indication of performance
Performance-based
Primary task or secondary task
(e.g. time estimation)
• provide direct measure of operator
performance
• operator may invest more effort to
maintain primary-task performance
• choice of secondary task is important
Effects of high workload
• operator is prone to actions not as planned: unable to monitor
activity fully
• increased attentional selectivity
• may respond quickly but inaccurately
• may shed some sub-tasks completely
Strategies for workload reduction [know this]
Change the task:
apply sound ergonomic principles
automate some functions
use new technologies
Change the operator
provide extensive training, to produce
motor programs (overlearning)
Personnel selection
for example, low trait anxiety may confer better
ability to cope with high task demands
Selection
Aims:
Deciding
• What to measure
• How to measure
• Effectiveness of measures
Stages in Selection System
Specify selection criteria
Specify assessment
methods
Evaluate
(After Hunter & Burke 1995)
The Systems Approach to developing selection
processes.
1.
Job / Competency analysis —
Identify Knowledge, Skills and
Attitudes (KSAs) required
2.
Use KSAs to identify appropriate
selection methods.
3.
Establish the reliability and fairness
of the process
4.
Validate
Job Analysis
Job
Requirements
Person
Requirements
Task
Competencies
Aptitudes
Prediction
Predictors
Person
Specification
(After Hunter & Burke 1995)
Criteria
Job
Description
Job Analysis
•
Aim — Identify critical competencies required
for successful job
performance
•
•
Outputs — What does the job holder do?
Inputs — what skills, knowledge, abilities does the job holder need?
•
Result is a competency framework identifying critical success factors
associated with successful performance
•
Why?
1.
2.
To achieve the best possible prediction of job performance (put the
right people in the job)
Legal requirement – test fairness
Types of Job Analysis
1. Hierarchical task analysis (Annett, Duncan et al 1971)
2. Functional Analysis (Fletcher 1991)
Techniques:
Critical incidence technique (CIT) (Flanagan 1954*)
•
•
•
•
•
Identify key roles and functions of job
Identify critical behaviours (related to success or failure)
Classify into similar behaviours
Summarise
Validate using other SME
Other techniques include Repertory grid
Classifying Aptitudes
Fleishman’s Taxonomy of Skills
Abilities were classified into:
• Cognitive:
Information processing and problem solving
• Perceptual/spatial:
Attention and spatial orientation
• Physical:
Flexibility, strength and stamina
• Psychomotor:
Coordination and reaction time
NATO Study: Aptitude dimensions for military fast-jet pilots (Bydorf 1993)
• Situational awareness: Perceptual closure + reaction time
• Spatial orientation
• Time sharing
• Aggressiveness
• Divided attention
• Psychomotor coordination
• Perceptual speed
• Selective attention
• Visualisation
Weighting Aptitudes
Determining priorities
• Need to identify relative importance of aptitudes in job performance
• DIF Analysis.
•
•
•
Ratings of:
Difficulty
Importance
Frequency
Choice of Measure
Personality Questionnaire
Group exercises
CV; Biodata
Interview
Ability
• Psychometric tests
• Work sample tests
• Physical tests
Life
Experiences
Motivation
Performance
Temperament
•Occupational Interest
Inventories;
•Measures of
personal values
•Interview
Types of aptitude measure
•
Paper & pencil measures
•
Computer-based testing: BARB (British Army); OASC (RAF); MicroPat
(AAC, RN, BA, Cathay); TASKOMAT (Commercial); BAT (USAF)
• Ease of administration
• Experimental testing
• Dynamic measures possible
• Measure processing capacity
• Multi-tasks
• Sophisticated measures such as response latency
•
Work sample
• RAF Flying Grading
•
Simulation based
•Advantages of CBT & work sample
• Lower costs
Example: Canadian Automated Pilot Selection System
•
Biodata
•
Personality measures see other lectures
Effectiveness of measures
Evaluating Selection: Reliability and Validity
Reliability
Accuracy and stability of the test
• Internal consistency reliability
• Split-half reliability
• Parallel forms
• Test-retest reliability
• Inter-rater reliability
Validity
Does the test really measure what it claims to measure?
• Construct validity
• Content validity
• Predictive validity
See other lectures as well!
Error in allocation
Performance score
Cut-off score
True
Positives
False
Negatives
‘Pass Mark’
True
Negatives
False
Positives
Predictor score
Performance score
Higher Correlation reduces error
TP
FN
FP
TN
Predictor score
EffectCut-off
of setting
Cut-off scores
score
Performance score
1
Cut-off score
2
‘Pass Mark’
Predictor score
Average Correlation between competency ratings and job performance
Great 8 Competency Factors
Leading and deciding
Supporting and co-operating
Interacting and presenting
Analysing and reporting
Creating and conceptulising
Organising and executing
Adapting and coping
Enterprising and performing
Overall job performance
0.35
0.23
0.30
0.35
0.42
0.45
0.25
0.37
Interpreting scores
• Norm referenced — most cognitive/ability tests
• Self referenced — Attitude/Personality measures
• Criterion-referenced — job skills
Validity of Different Methods
Selection Method
Mean Validity Co-efficient
Interview - Unstructured 1
.14
Interview - Structured 2
.35
Biodata 1
.37
References 3/1
.17 to .26
Cognitive ability testing 4/1
Personality testing 1/5
.25 to .53
.10 to .33
Work-sample tests 1
.54
Trainability tests 6
.46
Example: RAF Aircrew Selection
READY TO CHANGE
YOUR LIFE?
Your visit to OASC will take several days and
includes:
• an initial briefing;
• aptitude tests;
• an aptitude test review;
• the exercise phase;
• an interview;
• an occupational medical; and
• fitness assessments
.
(from OASC brochure)
Example: RAF Aircrew Selection
ADPO10369
EVOLUTION OF APTITUDE TESTING IN THE RAF
M. Bailey, RAF Cranwell
Before 1940: main method was
unstructured interview
About 50% pilot training failure rate at start of WWII
First set of Aircrew Selection Board tests included
• Essay writing
• Elementary maths
• General intelligence
•
Early developments
• need recognised for separate tests of skills and personality
• shift to testing for specific roles (e.g. electromechanical coordination)
• 1944: With help from USAAF, series of objectives measures (e.g. 24 aptitude tests
for six aircrew categories) – waste down from 48% to 25%
• use of specially trained staff
Example: RAF Aircrew Selection
1944–84
Many more tests created – but at the end of this period tests were not markedly different
Preliminary Flying School closed 1974 – selection then relied purely on aptitude tests. For
various reasons validities dropped; for example, to .14 for training results
Second generation selection tests:
• exploited increased computing capability:
• at first, computerised versions of existing tests
• later, new tests (based on abilities required, using Fleishman’s system) – Air
Traffic and Fighter Controller Test Battery produced
• Nine weighted test scores used
• Good predictive validity
Issues
• No formal job analysis
• Tests driven by theory and test availability
Hence 1990s:
• Shift to domain-centred framework
Example: RAF Aircrew Selection
:
Simulation and Training
Information from
• skill lecture
• simulation and training lecture (technical skills)
• CRM lecture (non-technical skills)
• Human error lecture
• etc
Key Issues
• Training needs analysis (organisational, occupational, individual) —
focus on Knowledge, Skills, Abilities/ Attitudes (KSAs)
Design of training programme
•
•
•
•
•
•
•
Develop Instruction by Objective
Select Instructional Strategy
Select/ organise element to be trained
Identify training aids
Organise materials/resources
Apply learning principles
Develop Evaluation Instruments
• Implementation (who, where, when)
• Evaluation (Reaction, Learning, Behaviour, Results)
• Note DIF analysis (difficulty, importance, frequency) — helps to decide whether not to train,
to train, or to over-train
Training issues (see also Human Information Processing lecture):
• Massed versus distributed practice
• Whole- versus part-task approach
• Phases of learning
• Feedback
Media and technology
• Simulation — very important in aviation, where the objective is to maximise the transfer of
learning from simulator to aircraft; fidelity is a key issue: do not need physical fidelity
(simulator does not need to resemble the aircraft), but functional fidelity is important
• Internet-based — increasingly important; can be accessed even in the field
• Traditional — still some role for classroom-based instruction
Situation Awareness
Topics
 Definitions
 Models
 Theory
 Metrics
 Applications
 Limitations
Why is Studying Situation Awareness
Important?
Hartel, Smith
and Prince
(1991)
Leading causal factor in a review of 175 aviation mishaps
Endsley
(1994)
Major causal factor in 88% of accidents associated with human error in a review of
major aircraft carrier accidents (1989-1992)
Woodhouse
and
Woodhouse
(1995)
Controlled Flight Into terrain (CFIT) accidents killed 5000 people between 1978 and
1992. 74% of these accidents were due to loss of flight crew SA
Historical Origins of SA
 SA popularised to describe the psychological
processes of:
–
–
–
–
–
Attention
Memory
Perception
Prediction
Pattern Matching
‘Mental Model’
of the situation
Definitions of SA 1
 Situation Awareness is . . .
“ . . . Knowledge of current and near-term disposition of both
friendly and enemy forces within a volume of airspace.” McMillan
(1994)
“. . . One’s ability to remain aware of everything that is happening at
the same time and to integrate that sense of awareness into what
one is doing at that moment.” Haines & Flateau (1992)
“. . . A pilot’s continuous perception of self and aircraft in
relation to the dynamic environment of flight, threats, and mission,
and the ability to forecast, then execute tasks based on that
perception.” Hamilton (1987)
A Working Definition of SA
 Situation Awareness is . . .
– “The perception of the elements in the environment within a
volume of time and space, the comprehension of their
meaning, and the projection of their status in the near future”
Endsley (1988)
– It is derived from the aircraft instrumentation, the out-thewindow view, and his or her senses
– The quality of an operator’s SA is moderated by individual
capabilities, training, experience, objectives, and the ability
to respond to task workload
– The term ‘SA’ should only ever be applied to dynamic
environments
Summary know this
PROJECTION OF
FUTURE STATUS
COMPREHENSION
OF CURRENT SITUATION
PERCEPTION OF
ENVIRONMENT
 When all is said and done we know that Situation
Awareness refers to an operator’s knowledge and
Understanding of the dynamic environment in
which he/she is operating
 It is knowledge of the ‘Big Picture’
 SA provides the basis for subsequent decision
making and performance in the operation of
complex, dynamic systems
A Model of SA
Level 3
Level 2
Level 1
PROJECTION OF
FUTURE STATUS
COMPREHENSION
OF CURRENT SITUATION
PERCEPTION OF
ENVIRONMENT
Endsley (1995)
‘Predict future
events / states
based on
understanding’
‘Integrate with
task goals’
‘Perceive
relevant
information’
Endsley’s Model





be aware of main elements
System Capability
Interface Design
Stress & Workload
Complexity
Automation
Task/System Factors
Feedback
SITUATION AWARENESS
State of the
Environment
Perception
of Elements
in Current
Situation
Comprehension
of Current
Situation
Projection
of Future
Status
Level 1
Level 2
Level 3
Performance of
Actions
Decision
Individual Factors
 Goals & Objectives
 Preconceptions
(Expectations)
Model of SA in dynamic decision
making (from Endsley, 2000)
Information Processing
Mechanisms
Long–term Memory
Stores
Automaticity
 Abilities
 Experience
 Training
E.g. IQ
Cognitive Abilities
Conscientiousness
Experience
Personality
Training
Risk Taking
QinetiQ’s Model of The SA Process
Operator Traits
E.g. Frightened
Confused
High Workload
Fatigued
Aggressive
E.g. Temperature
Time Pressures
Day/night
Noise
Lethality
Environmental
State
Operator State
Real World/System
Information
Goals
Mission Goals
System Goals
Personal Goals
SA
Salience of info
Availability of info
Info complexity
Info quantity
Automation
Quality of HMI
The SA Process
The SA ‘PROCESS’
 A series of complex cognitive
processes, including: Perception,
Working Memory, Pattern Matching,
Attention and Long Term Memory
 NOT ‘task’ or ‘individual’ specific
 Also referred to as Situation
Assessment (SAS)
 Will be influenced by a multitude of
‘SA Factors’
Factors Affecting the SA Process
Operator Traits
Environmental
State
Goals/Doctrine
/SOPs
Knowledge/
Information
E.g. IQ
Cognitive Abilities
Conscientiousness
Experience
Personality
Training
Risk Taking
E.g. Temperature
Time Pressures
Day/night
Noise
Lethality
Mission Goals
System Goals
Personal Goals
Directives
ROE
Commander Intent
Salience of info
Availability of info
Info complexity
Info quantity
Automation
Quality of HMI
 These factors WILL BE
‘task’ AND ‘individual’
specific
 Each factor will have
different weightings or
importance attached to it
for differing military
domains
 The number of such
factors is vast
Know the main headings!
SA as a Product
 The output of the SA
PROCESS will be a number of
‘Situation Models’ (or dynamic
mental models)
 These situation models are
essentially knowledge and
understanding
 The quality of a person’s SA is
defined by the match between
these situation models and
reality
Real World
The difference between
these represents the
quality of one’s SA
Situation Model
SA Elements
 The person will have a
situation model for
each of the relevant ‘SA
Information Domains’
associated with a
specific task or job
 Each SA information
domain will comprise a
number of ‘SA Elements’
 Example: Endsley
(2001) illustrates this
for the task/job of
piloting a civil aviation
aircraft
Geographical SA
•own aircraft
•other aircraft
•terrain features
•airports
System SA
•cities
•system status
•waypoints
•functioning and settings
•navigation fixes
•radio
•position relative to
•altimeter
Spatial/Temporal
SA
designated features
•transponders
•attitude
•path
to desired location
•flight modes and automation
•altitude
•runway
and taxiway assignments
•deviations
from correct settings
•heading
•climb/descent points
Environmental
SA
•ATC
communications
present
•velocity
•weather
formations and movement
•fuel
•vertical velocity
•temperature
•impact
of degrades and settings
•Gs
•icing
•flight •on
pathperformance
•ceilings
•time
and
distance
available
on fuel
•actual
values
relative
to assigned
•fog
•projected flight path
•Turbulence,
windstime
•projected landing
•sun
•visibility
•IFR/VFR conditions
•areas to avoid
•flight safety
•projected weather conditions
In Summary
 The development and
maintenance of SA occurs
within an individual’s head
 The SA process (or SAS) is a
generic continuous
process/cycle that is impacted
upon by many factors
 These factors will vary in their
importance and influence
depending upon the specific
task and the individual
undertaking that task
 An individual will continuously
cycle through the SA process
for each SA Information
Domain, developing a situation
model for each
 These situation models will be
task-specific
 All situation models will be
continually updated and
revised as new information
becomes available or as the
factors affecting the SA
process change in importance
or in state
Team SA
 SA can be applied to teams as
well as to individuals
 Caution needed here, as SA
cannot be shared (it resides
inside the individual’s head), but
information can be shared
 We could be talking about:
– 1) The overlap in SA for the team
– 2) The SA of the team as moderated by
the primary decision maker
– 3) The collective SA of the entire team
Measuring Situation Awareness
 SA has become a major design driver
– Developing operator interfaces to enhance SA
– Developing automated systems without resulting in a loss of
SA
– Training techniques are designed to develop better SA
 Development of SA metrics for evaluation purposes
– Development of metrics since the late 1980s
– Varying degrees of maturity / validation
– Various forms of metrics
• Subjective Vs Objective
• Self-report Vs Third-Party rating
• Simulator-based Vs Test flight
SA Metrics
 Crew SA
 SA Global Assessment technique (SAGAT)
 Snapshots
 SA Flight Training Evaluator (SAFTE)
 China Lake SA Scale (CLSA)
 SA Rating Technique (SART)
 SA Supervisory Rating Form (SASRF)
 Physiological Measures: Eye Activity
SA Metrics – Summary
SA
Level
Crew
SA
Perceive
Environment
Comprehend
Current
Situation
Project
Future
Status

SAGAT
Snap SAFTE
Shots
CLSA
SART











SASRF
Eye
POG




Know at least SAGAT/SART in a little detail, plus names of a few others
SA Metrics – Summary
 Most SA measures have been designed using a
particular SA definition, and with a specific application
in mind
– Keep this in mind when selecting an SA measure
 In practice, 2 of the SA measures outlined previously
are used far more than the others:
– SART (subjective)
– SAGAT (objective)
 This is probably due to the extensive validity data that
accompanies these measures (we ‘know’ they are
measuring SA)
So, of What Use is SA Research?
 There are three main military applications for SA
research:
– 1) System/interface design, development, assessment and
evaluation
• Operator interfaces designed to enhance SA
• Automated systems must switch without losing operator
SA
– 2) Training operators to have better SA
– 3) Selecting operators who are predisposed to having high SA
Limitations of SA
 Immature concept
– Still much debate over definitions and measures
 SA is a theoretical construct
– Practical difficulties in measuring and predicting SA
– For those who do not understand the theoretical basics of
what SA is all about, there can be an element of perceived
circularity
‘Why did the aircraft
crash?’
Because he
crashed
Because of lack
of pilot SA
‘How do we know there
was a lack of pilot SA?’
Crew Resource Management (CRM)
CRM Training
Introduction
 Need for effective interaction
 Aviation accidents: most have human error component
CRM Evolution
 Evolution of CRM to fifth-generation
CRM Training
Objectives of CRM
 Knowledge, skills, attitudes to promote safe, efficient
operations:
– Effective decision making
– Good crew communication
– Understanding/acceptance of role and responsibilities
 CRM focuses broadly on training transportable teamwork
skills
CRM Training
 Types of CRM course
 Foundation Course
– Wide range of topics covered
– Focus on discussion and video
 Continuation Courses
– In depth coverage of topic areas
– Skills practice (low fidelity)
 LOFT/MOST
–
–
Skills practice (high fidelity)
Crew-centred debrief
CRM Training
 Topics in typical CRM courses
– Human information processing
– Personality and attitudes
– Communications
– Teamwork structures
– Teamwork behaviours
– Leadership style
– Decision making
– Stress management
– Human error
– Situation awareness
– Automation on the flight deck
– Fatigue and workload
– Case studies & research findings
– Be able to list the main topics
CRM Training
Leadership issues
 Effects of captain’s attitudes
 Authority Gradient
CRM Training
 Communication
– US ASRS: most accidents involved failure of information
transfer
– Low-error crews demonstrate different patterns of comms
 Communication skills know them!
–
–
–
–
–
–
Inquiry
Advocacy
Listening
Conflict resolution
Critique
Feedback
 Barriers to communication
–
–
–
–
Physical; word usage; interpersonal; mental
Cultural and language barriers
Subordination problems
‘Power-distance’ barriers
CRM Training
 Core teamwork behaviours
– Monitoring
– Feedback
– Backing up
CRM Training
 Ad hoc teams
– Frequently arise in airline ops
– 73% of accidents occur on first day crew flying together
 Situation awareness
– Important topic in CRM
– See lecture on SA!
CRM Training
 CRM training resources
– Self-study
– Classroom awareness training
– Modelling
– Classroom skills training
– Skills practice in simulators
– Practice/coaching during flying
CRM Training
 LOFT
– Run in a high fidelity simulator
– Realistic sortie/real time
– Crew and facilitator ‘in role’
– Few failures
– Non-technical focus
– Focus on choice dilemmas
– Non jeopardy
– Crew-centred debrief using video
CRM Training
 CRM Issues
– What is ‘best practice’?
– Does it work?
– Those needing most help from CRM most resistant to
change
– May change attitudes but not behaviour
– Needs management commitment
– CRM skill fade occurs over time
– Cultural issues should be considered
Error and Accidents
(See Accident module)
The following may help you structure your
Knowledge of this topic – drawn from work
of John Chappelow
ENVIRONMENT
Disruptive factors
noise
heat
cold
vibration
threat
task demand
Be able to list/
describe the
main factors
SYSTEM
Enabling factors
OPERATOR
ergonomics
training
briefing
social context
Predispositions
personality
talent
fatigue
alcohol
overarousal
underarousal
Intention
Perception
Action
Task
Error type
Disruptive
factors
Enabling
factors
Predisposing
factors
Perception
errors
Stressors
Physical
Trait
Visual illusion
Disorientation
Physiological
Noise
Personality
Lack of talent
Undetected threat
Misinterpretation
Time pressure
Operational
pressure
High task demand
Ergonomics
Handling
characteristics
System logic
Inexperience
Excess zeal
Lack of airmanship
Intention errors
Low task demand
Sensory limitations
Rule violation
Threat
Inappropriate
model
Distraction
Cognitive
limitations
Motor limitations
Action errors
Organisational
State
Cognitive failure
Slow response
Precipitate response
Training
Briefing
Administrative
support
Social context
Alcohol
Fatigue
Hypoglycaemia
Disorganised
response
Mishandling
Life stress
Low morale
Underarousal
Overarousal
Social factors
Summary:Error
majortype
factors
Disruptive
factors
Enabling
factors
Predisposing
factors
Perception
errors
Stressors
Physical
Trait
Visual illusion
Disorientation
Physiological
Noise
Personality
Lack of talent
Undetected threat
Misinterpretation
Time pressure
Operational
pressure
High task demand
Ergonomics
Handling
characteristics
System logic
Intention errors
Low task demand
Rule violation
Threat
Inappropriate
model
Distraction
Action errors
Cognitive failure
Cognitive
failure
Slow response
Precipitate response
Disorganised
Disorganised
response
response
Mishandling
17%
26%
Ergonomics
Training
Briefing
Admin.
support
Personality
21%
Inexperience
23%
Inexperience
Excess zeal
Lack of airmanship
Sensory limitations
Cognitive
limitations
Motor limitations
Organisational
State
Training
Briefing
Administrative
support
Social context
Alcohol
Fatigue
Hypoglycaemia
~ 40%
Social context
Life stress
Low morale
Underarousal
Overarousal
Social factors
9%
Social factors
11%
You’ve seen this before...
Anxious
Neurotic
Impulsive
Extraverted
Introverted
Stable
Major causal factors: Human factors
In e x p e rie n c e
H ig h t a s k d e m a n d
A d m in is t ra t ive s u p p o rt
O ve ra ro u s a l
D is t ra c t io n
S o c ia l fa c t o rs
S e n s o ry lim it a t io n s
P e rs o n a lit y
L a c k o f a irm a n s h ip
E rg o n o m ic s
0
Expanded data set
5
10
15
20
N u m b e r o f m i sh a p s
25
30
35
Sensitivity: Human factors
Social factors
Distraction
This graph shows
benefit of eliminating
the factor, and cost
of an increase in its
severity
High task demand
Inexperience
Administrative support
Briefing
Social factors are
seen to be more
important when we
conduct sensitivity
analysis: these problems
are soluble
Lack of airmanship
Supervision
Sensory limitations
Social context
-15
-10
Expanded data set
-5
0
5
10
15
Error: Examples of Possible Remedies know this
Social factors
• Personnel selection
• CRM training
Distraction
• May be able to select individuals less prone to distraction
High task demand
• Selection (e.g., some personality types cope better with high workload)
• Training creates more ‘spare capacity’
• The system can be modified to reduce workload (automation, better
ergonomics etc)
Inexperience
• More (or improved) training
Administrative support
• Organisational interventions