An Introduction to Reliability
testing
Dr Jane Marshall
Product Excellence using 6 Sigma
Module
PEUSS 2011/2012
Reliability testing
Page 1
Objectives of this session
•
•
•
•
•
•
Reliability testing
Accelerated life testing
RET testing
Reliability growth testing
Test, analyse and fix
FRACAS
PEUSS 2011/2012
Reliability testing
Page 2
1
Product life cycle
Design
FMECA, FTA, PoF,RBD
FE,accelerated life test
Development
Development Test
Use
FRACAS
Field data analysis
Manufacture
Test
SPC
ESS, Burn-in
PEUSS 2011/2012
Reliability testing
Page 3
What is a test ?
• A test has been defined as
– "a subjection to conditions that show the real
character of the thing”
• However, testing is a continuing operation to
provide information throughout the complete
evolution of the system.
PEUSS 2011/2012
Reliability testing
Page 4
2
Purposes of testing
• The purposes of testing are numerous.
– A test may be performed to see whether a certain configuration or item is
feasible.
– A test may be used to determine which of several configurations is the
optimum with respect to performance, reliability, cost, modes of behaviour
under varying conditions, etc.
– A test maybe used to make more sensitive comparisons to further
improve economy, maintenance, use of standard parts, and so on.
– A test may be used to demonstrate whether the item is adequate to meet
the requirements of performance and reliability.
– A test can be used as thorough investigation of the latent capabilities of
the item under severer or more diverse conditions than those immediately
anticipated.
PEUSS 2011/2012
Reliability testing
Page 5
What should we be testing ?
Goal: To minimize the number of tests required.


Define exactly what requires testing - bearing in mind that the
objective is to minimize the number of tests required for cost
reasons
The test types may be categorized as follows:  Mandatory
 Testing based on pedigree or experience
 Data driven test requirements.
PEUSS 2011/2012
Reliability testing
Page 6
3
What should we be testing ?

Mandatory Testing
 Tests based on regulatory requirements - e.g. Road worthiness
 Tests based on safety requirements
 Tests based on customer requirements
 Tests based on competitor benchmark
PEUSS 2011/2012
Reliability testing
Page 7
What should we be testing ?

Testing based on pedigree or experience:  A historical product with similar physical and performance attributes
and similar testing may be referred to as a guide.
 Manufacturing and development engineers may have a ‘feel’ for the
type of testing required based on previous experience.
PEUSS 2011/2012
Reliability testing
Page 8
4
Frequency of testing
Goal: To test units as infrequently as possible.

Need to determine the frequency of testing.
 Each and every product
 Each and every product for a limited period
 Periodic testing
 Regulatory requirement testing at a specified interval/frequency
PEUSS 2011/2012
Reliability testing
Page 9
Time based testing
•
•
•
•
when an item fails,
which part in a component causes failure
which component in a system fails.
the mode or modes of failure as exemplified by
performance drift, erratic performance,
catastrophic failure, etc.,
• the mechanism of failure that is, the reasons for
failure caused by poor design, part misapplication,
etc.
PEUSS 2011/2012
Reliability testing
Page 10
5
Event testing
• In this case samples of the equipment are tested
repeatedly through their cycle of operation until failure.
• This testing, which is analogous to time-to-failure
testing, becomes the more meaningful test when
starting and stopping operations are more destructive
than the mere accumulation of time.
• The important parameter in this form of testing is mean
number of cycles to failure
PEUSS 2011/2012
Reliability testing
Page 11
Environmental testing
• As its name implies, this form of testing
represents a survey of the reaction of the item to
the various environments.
• It is usually required in qualification tests and is
frequently introduced in the development stage,
usually at less numerous or less severe
environmental levels.
PEUSS 2011/2012
Reliability testing
Page 12
6
Tests used during design
and development
•
•
•
•
Accelerated life tests
Qualification tests
Software testing
Development testing – RET, Growth testing,
Test, Analyse and fix
– Test to cause failures not to demonstrate successful
achievement
PEUSS 2011/2012
Reliability testing
Page 13
Testing for reliability and
durability
• Test to cause failures
• Not test to demonstrate successful achievement
• Use concepts of load and strength to determine
test conditions
PEUSS 2011/2012
Reliability testing
14
7
Types of Accelerated Test
• High Usage Rate.
• run the product faster than normal,
• increase the products duty cycle.
• Care should be taken that the increased usage
rate does not increase the other stresses on the
product too much, so that the failure mechanism
produced are the same as those seen under
normal usage rates.
PEUSS 2011/2012
Reliability testing
Page 15
Types of Accelerated Test
• Over-stress testing
– This consists of running a product at a higher than
normal level of some accelerating stress or stresses
to shorten product life or to degrade product
performance factor.
– Typical accelerating stresses are temperature,
voltage, mechanical load, thermal cycling, humidity
and vibration.
– Over-stress testing is the most common form of
accelerated testing.
PEUSS 2011/2012
Reliability testing
Page 16
8
Types of Accelerated Test
•
Censoring
– This is accelerated testing through a statistical ‘trick’, where the tests are terminated
before all the specimens have failed.
– Statistical techniques are then used to predict what the outcome of the test would have
been had it run to completion.
– This allows the tests to be shortened considerably and has become common practise.
•
Degradation
– Accelerated degradation testing involves over-stress testing but instead of life, product
performance is observed as it degrades over time.
– Similar statistical methods to those used for the censoring tests are used to calculate
the reliability.
•
Specimen Design
– The life of some products can be accelerated through the size, geometry and finish of
specimens.
– For instance large specimens fail sooner than small ones, e.g. high capacitance
capacitors fail sooner than lower capacitance ones of the same design, this is because
the large capacitors have more dielectric area.
PEUSS 2011/2012
Reliability testing
Page 17
Constant stress
• The most common form
• Each specimen is run at a constant stress level.
• Mimics the stresses seen in actual use in many
cases and can be simple.
• The advantages are
– A constant stress level is easy to maintain
– Models for acceleration are well developed
and tested.
PEUSS 2011/2012
Reliability testing
Page 18
9
Step stress tests
• Progressively increasing stresses are applied to
the same sample for constant time intervals
while the sample does not fail.
• The fundamental assumption in this kind of test
is that the failure probability at each level is
independent of previous levels, regardless of
previous history.
• Generate data on the product reliability more
quickly, but with less precision.
• The step stress test are normally carried out as
comparison tests.
PEUSS 2011/2012
Reliability testing
Page 19
Methods of applying stress
•
Progressive Stress Tests
– In progressive stress testing, a specimen undergoes a continuously increasing level of
stress.
– The method has similar advantages and disadvantages to the step stress test.
•
Cyclic Stress
– In use some products repeatedly undergo a cyclic stress loading, for example all AC
electrical products see a sinusoidal stress.
– A cyclic stress test for a product repeatedly loads a specimen with the same stress
pattern at high stress levels.
– For many products this cycle is sinusoidal, for others the duty cycle repeats but is not
sinusoidal.
•
Random Stress
– In a random stress test, a random stress loading is applied to the specimen for a
random time and then removed another random stress for some other random time
and so on.
– This type of test simulates the real use of a number of products.
PEUSS 2011/2012
Reliability testing
Page 20
10
‘Elephant’ or proof tests
• Such tests are often called
– torture tests,
– shake and bake,
– killer tests,
– design margin tests
– design limit tests.
• If the product survives the test then it passes and is judged to be a
‘good’ product.
• If the product fails it is usually redesigned to remove the cause of
failure.
• Elephant tests are normally performed on a few specimens which
are subject to a single severe stress level for a fixed time.
• An elephant test may not produce certain important failures that are
seen in practice and so a series of different elephant tests may
have to be used.
PEUSS 2011/2012
Reliability testing
Page 21
Accelerated Degradation tests
• Accelerated degradation is concerned with models and
data analysis or product performance over time at
overstress and design conditions.
• These form of tests have an advantages over accelerated
life tests since performance degradation data can be
analysed earlier before any specimens fail.
• This is done by extrapolating performance degradation to
estimate a time where performance reaches a failure level.
PEUSS 2011/2012
Reliability testing
Page 22
11
Reliability Enhancement Tests
• Tests that are carried out in the design and
development phase of a product to remove
defects.
• The standard approach is
– TEST
– ANALYSE
– FIX
• Various methods exist for analysis of the data
PEUSS 2011/2012
Reliability testing
Page 23
RET
•
•
•
•
•
•
Purpose is to improve design and increase durability
Step stress testing – combination of stresses
Do not simulate environment
Stresses in excess of those seen in service
HALT – highly accelerated life testing
Also helps with helps optimise stress screens
PEUSS 2011/2012
Reliability testing
Page 24
12
What stress limits?
• Increase stress to cause failures but limit by:
– Technological limits
– Test capability
• Use unrepresentative stresses because:
–
–
–
–
Future causes of failure uncertain
Probabilities to failure uncertain
Time spent on testing expensive
Finding causes of failure during development less expensive in
the long run
PEUSS 2011/2012
Reliability testing
Page 25
Tests used prior to shipment
to the customer
• Environmental Stress Screening – ESS
• Demonstration testing
PEUSS 2011/2012
Reliability testing
Page 26
13
Environmental stress
screening (ESS)
– This involves accelerated testing of products under a
combination of random vibration and thermal cycling
and shock - shake and bake.
– It has two major purposes,
• as an elephant test during development
• as an accelerated burn in to improve reliability.
– It is widely used for military, industrial and consumer
electronics.
PEUSS 2011/2012
Reliability testing
Page 27
Burn in
– Burn-in consists of running items under design or
accelerated conditions for a suitable length of time.
– Burn-in is a manufacturing operation that is intended
to fail short-lived units (defectives, sometimes called
freaks).
– If burn-in is successful then the production units will
have few early failures.
– Burn-in is primarily used for electronic components
PEUSS 2011/2012
Reliability testing
Page 28
14
FRACAS
• Failure reporting and corrective action system
• Closed-loop system for identifying, assessing,
and correcting failure-related problems in a
timely manner
• Start of project
• Used by all personnel
• Review team
PEUSS 2011/2012
Reliability testing
Page 29
Summary
• Introduction to reliability testing
– Design and development
• Accelerated life testing
• RET
• Reliability Growth
– Pre-delivery testing
• ESS
• Demonstration
• FRACAS
PEUSS 2011/2012
Reliability testing
30
15
The generic “test” process
Inputs
Outputs
Product
Spec
external
& internal
customer
requirem
ents
Top Level Process
Define tests to
be performed
Verificatio
n reports
Plan
resources
Complete
Testing
Analyse Test
Results
Draw
Conclusions
Write test
plan
Airworthi
ness
requirem
ents
Declaratio
n of
design
and
performan
ce
Complian
ce
analysis
Tech.
Risk
analysis
Additional
testing
requireme
nts
Safety &
reliability
requirem
ents
PEUSS 2011/2012
Define tests to
be performed
Reliability testing
Plan
resources
Complete
Testing
Analyse Test
Results
Page 31
Draw
Conclusions
Write test
plan
Inputs
Outputs
Sub process - Define tests to be performed
Product
Spec
external
& internal
customer
requirem
ents
List all
tests
required
Airworthi
ness
requirem
ents
Tech.
Risk
analysis
Safety &
reliability
requirem
ents
PEUSS 2011/2012
Group tests
which can be
done together
Define test goal
Define test
method e.g.
mechanical
environmental
performance
reliability
Identify
potential
relationships
between inputs
and outputs
Identify test
inputs
Identify levels
for test inputs
Take account
of experience
data
Identify test
outputs
Reliability testing
Identify
constraints
budget
people
time
Define verification
test plan taking into
account variation
of inputs
List of
grouped
tests with
definition of
test method
and design
Define Sample size
and repetitions
required
Page 32
16
Plan
resources
Define tests to
be performed
Complete
Testing
Analyse Test
Results
Draw
Conclusions
Write test
plan
Outputs
Inputs
Sub process - Plan resources
Define hardware
requirements
List of
grouped
tests with
definition of
test method
and
procedure
Define test
equipment
requirements
Allocate
budget
Allocate people
resource
Procure test
hardware
Verify
Calibration is
up to date for
test equipment
Procure test
equipment
Allocate test
equipment
Completed
test plan with
resources
including
verified test
equipment
Write test plan - include information from
test method
test design
lessons learned
standard formats
Sub process - Write test plan
PEUSS 2011/2012
Define tests to
be performed
Reliability testing
Plan
resources
Complete
Testing
Analyse Test
Results
Page 33
Draw
Conclusions
Write test
plan
Inputs
Outputs
Sub process - Complete Testing
Completed
test plan
with
resources
including
verified test
equipment
PEUSS 2011/2012
Record Test Results
Run test as planned
Record unusual
events or
observations
Reliability testing
Test
Results
Page 34
17
Define tests to
be performed
Plan
resources
Complete
Testing
Analyse Test
Results
Draw
Conclusions
Write test
plan
Inputs
Outputs
Analyse Test Results
Produce test report
Use engineering analytical
tools
Test
Report
Complete/update
compliance analysis
Test
Results
Complianc
e Analysis
Use engineering knowledge
Update lessons
leaned database
PEUSS 2011/2012
Reliability testing
Define tests to
be performed
Plan
resources
Complete
Testing
Analyse Test
Results
Page 35
Draw
Conclusions
Write test
plan
Inputs
Outputs
Sub process - Draw Conclusions
Verificatio
n reports
Test
Report
Complianc
e Analysis
Decide if test was successful or
not
Define actions to be taken based on
analysis
Decide what additional testing
is required
Conclude further
reporting
Declaratio
n of
design
and
performan
ce
Complian
ce
analysis
Additional
testing
requireme
nts
PEUSS 2011/2012
Reliability testing
Page 36
18