Operational Considerations in the
Certification of Technically Advanced
Aircraft (TAA)
Robert Barnes, Robert B Barnes Associates, Inc. (USA)
Charles Adam, Federal Aviation Administration (USA)
Operational Considerations in the Certification of TAA
Slide 1
Today’s topics
• The need to consider the user
• What we’ve seen during flight test
• Current policy
• Current design practice
• Importance of real user trials
• Our suggestions
Operational Considerations in the Certification of TAA
Slide 2
“General aviation today is experiencing the
most rapid pace of innovation since the late
1940s …
… These advances offer general aviation
pilots the promise of increased levels of
safety and performance …
… Unfortunately, these increased levels of
safety have not materialized.”
-- Littlefield, Glass Cockpit Flying, 2010
Operational Considerations in the Certification of TAA
Slide 3
“Why has the promise of greater levels of safety
for glass cockpit airplanes not been realized?
Because, in order to realize these benefits, general
aviation pilots must learn a new way of flying.
Unfortunately, general aviation pilots and training
providers have not yet evolved the way they train
and fly to catch up with the advances in glass
cockpit technology.”
-- LinkedIn Aviation Training Group Discussion
Operational Considerations in the Certification of TAA
Slide 4
As seen during flight test
• Design may be
functionally
acceptable to
engineers
• Is the design
acceptable to properly
qualified pilots?
• Can a crew safely
operate new design?
Operational Considerations in the Certification of TAA
Slide 5
Some personal findings
• Engineers tend to design systems which
may seem logical and intuitive to them (an
engineering skill)
– Sequences of steps
– Multifunction control logic
• These same systems may be far from
logical or intuitive to the typical pilot (a pilot
skill)
– Recognizing operation of dedicated controls
Operational Considerations in the Certification of TAA
Slide 6
Effectively demonstrating design
acceptability requires:
Involvement of qualified pilots …
Representing a cross-section of the
target pilot population …
Who are operating the new design
under realistic conditions.
Operational Considerations in the Certification of TAA
Slide 7
Human Factors in Part 23 Aircraft
Certification
• FAA Certification Policy on Human Factors (PS-ACE1002001-004, Aug 2002)
- Response to industry requests for guidance
- Advocates early FAA involvement during development
- Goal: To aid in the timely identification and resolution of human
factors-related issues during the aircraft certification process
- Guidance only … not a regulatory requirement
• “General Aviation fatalities (part 91 operations) occur at
higher rates than part 121 and part 135 operations.”
Operational Considerations in the Certification of TAA
Slide 8
Operational Considerations
• Pilot characteristics
• Operational considerations
• System operation procedures
• Training requirements
Operational Considerations in the Certification of TAA
Slide 9
FAA Policy on Human Factors
• Not a regulatory requirement but FAA ACO “should
attempt” to follow
• Increased effort being made to address designrelated human error
• FAA rulemaking effort (25.1302) already being
applied by EASA
• Human factors issues are open to broad
interpretation
– Component level
– Operational level
Operational Considerations in the Certification of TAA
Slide 10
Human Factors Design Practice
• Design and Operational
factors affect workload
• There are many ways to
evaluate these affects
• 500 pages of methods
described
• User trials are on page
475
Operational Considerations in the Certification of TAA
Slide 11
Some user trial advantages
• Simplistic and flexible approach to
usability evaluation
• Evaluates the system based upon the
potential end users’ performance
• Gives the designers a powerful
insight into how the system will be
used
Operational Considerations in the Certification of TAA
Slide 12
Some user trial disadvantages
• Time consuming to conduct
• Large amounts of data are collected,
ensuring a lengthy analysis process
• It may be difficult to gain access to
the required personnel or end users
• End users may be biased towards the
old system or procedure
Operational Considerations in the Certification of TAA
Slide 13
Applying User Trials in U.S.
• Limited form now used periodically by FAA
for avionic component evaluations
• Multiple Pilot System Usability Evaluation
(MPSUE)
• FAA pilots not typical end users
Does this adequately evaluate operation by
the typical user?
Operational Considerations in the Certification of TAA
Slide 14
TAA certification user trial
• Removing restrictions required a human
factors workload evaluation
• Applicant chose to use Modified Pilot
Subjective Evaluation (MPSE)
• Two typical user pilots were qualified
• Certification was successful but many
lessons were learned by manufacturer,
training provider, and regulator
Operational Considerations in the Certification of TAA
Slide 15
Conclusions
• Human factors evaluation is subjective
• Level of pilot competence required to
operate design is critical
• Using experienced test pilots is not
enough
“Is the aircraft capable of being operated safely
by the manufacturer’s target customer with
the type of training proposed?”
Operational Considerations in the Certification of TAA
Slide 16
Questions/Comments?
For a copy of our
papers, please leave
me your card
Operational Considerations in the Certification of TAA
Slide 17