“Flight Test Is
As Flight Test Does”
Guide to Safe and Effective Flight Test
George Cusimano
Flight Test Safety Workshop
5 May 2010
1
Mamma always said:
“Stupid is as stupid does”
Forrest Gump circa 1994
• Meaning – You are what you do
• Applies to flight test as well as life in general
2
Safe & Effective Flight Test Is . . .
• Desired perceptions
–
–
–
–
–
–
Culture of safety
Manage risks
Credible and unbiased
Efficient and effective
Legitimate results
Requirements validation
3
. . . As Flight Test Does
• Real actions do not always reflect good intentions
– Program pressure – test is an obstacle to progress
• Cost
• Schedule
– “We can do more with less” attitude
– Poor risk assessments
• Inadequate
• Incomplete
– Poor systems knowledge
4
Guide to Positive ACTIONS
• Truths for Safe and Effective Flight Test
– Common sense
– Historical basis
– Mindful of safety
• “Stupidity” prevention
5
Guide to Positive ACTIONS
•
•
•
•
•
•
All models are wrong
Believe the data until it is proven wrong
Don’t be like a box of chocolates
Always challenge
When in doubt – Don’t
Testing the wrong thing right is as costly as testing
the right thing wrong
• If at first you don’t succeed – check your plan
6
All Models are Wrong!
• Model: Representation of reality
– Bounded by assumptions and simplifications
– Valuable to the development process
• Flight Test: Reality
– Validates models
– If predictions are incorrect – fix the models!
7
Believe the Data . . . Until . .
• Flight Test is truth . . . IF
–
–
–
–
Good instrumentation
Test configuration
On the correct condition
Applying the correct technique, correctly
• If the results are different than expectation . . .
– Slow down
– Stop
. . . until the differences are resolved
8
Believe the Data
(Example)
• A tracking algorithm was supposed to run at 100 Hz.
• During test the control room observed 10 Hz
– Assumed to be a TM anomaly
• Post flight analysis revealed that 10 Hz was real.
9
Box of Chocolates
Mamma always said:
“Life is like a box of chocolates,
you never know what you are going to get”
Forrest Gump circa 1994
• Flight test should NEVER be like a box of chocolates;
we should always know what to expect!
10
Set Expectations
• Predict flight test results
– For every test point
– Based on models and/or analyses
• If you do not know what to expect, then you do not
understand the system
• If you do not understand the system, then you may
not be testing the right thing right
• If you are not testing the right thing right, then you
should not be conducting the test.
11
Set Expectations
(Example)
• Objective: Expand forward CG limit
– 12” forward of existing limit
• Incremental approach
– First flight planned @ 1” in front of baseline limit
• Difficulty rotating with full aft stick; nearly departed
• Did not understand why – decided to land . . .
12
Always Challenge
• What if something unexpected happens?
– What will we do?
– Have a plan
• Why did the system behave as it did?
– Systems knowledge
– Significance of error
• What next?
– Do nothing – proceed
– Suspend test point, but continue the mission
– Stop the mission
13
Always Challenge
(Example)
• Designed to test fighter thrust vectoring technology
• High AOA test point
• Pilot: “Something is not right. I show 280 KIAS and
20 alpha.”
• Pilot did not persist in knowing why
• Control room did not suspend test while they
investigated
• Result: Lost aircraft
14
When in Doubt – Don’t!
• If test results do not meet expectation
–
–
–
–
Ponder
Wonder
Debate
Decide
• Listen to your instincts
• The obligation to stop a test resides with every
member of the test team
Proceeding without knowledge = disaster
15
When in Doubt
(Example)
• Objective: Expand forward CG limit
– 12” forward of existing limit
• Incremental approach
– First flight planned @ 1” in front of baseline limit
• Difficulty rotating with full aft stick; nearly departed
• Did not understand why – decided to land . . . .
& try again
• Repeated problem on 2nd takeoff
• Post flight – CG 13” in front of existing forward limit
16
Testing the Wrong Thing Right;
Testing the Right Thing Wrong
• Both are costly and inefficient
• How we get trapped:
– Not having well defined test objectives
– Not understanding the system under test
17
Testing Right
(Example)
• RADAR test at maximum slant range
• Weather forced the test to higher altitude and
closer horizontal range
• Four hours of data collected
• Result: Invalid test
– Test had to be repeated
18
If at First You Don’t Succeed -Check Your Plan
• Based on well defined requirements
–
–
–
–
Relevant test objectives
Data required to meet the objectives
Methods needed to collect that data
How the collected data will be analyzed to show
compliance
– Must include resources
19
Check Your Plan (cont.)
• Without a good test plan you do not know:
–
–
–
–
Why you are testing
How you will test
When you are finished testing
What to do with the collected data
• Plans change
– Configuration control is required
20
Check Your Plan
(Example)
•
•
•
•
F-117
Night
Navigation triangles with autopilot
Pilot decided to hand fly one turn
– Disengaged autopilot
– Black hole phenomenon
• Almost departed the aircraft.
21
Configuration . . .
• Test article
– Testing the right thing
• Test plans
– Testing requirements
• Instrumentation
– Valid data
• Test methods and conditions
– Testing right
Know What You’re Testing
22
Common Element
Test
Article
Test
Plan
Instrument
Methods &
Conditions
Models are
Wrong
Not real
Limited by
simulation
Not part of
simulation
Limited by
simulation
Believe the
Data




Set
Expectations




Always
Challenge




When in
Doubt




23
Common Element
Test
Article
Test
Plan
Instrument
Methods &
Conditions
Test the
Right Thing




Planning is
Everything




24
Summary
• “Stupid is as stupid does”
– Testing without knowledge is stupid
• Do NOT be like a box of chocolates
– Always know what to expect
Know What You are Testing
25
“That’s all I have to say about that !”
Forrest Gump circa 1994
26