MPD 575
Design for Testability
Jonathan Weaver
1
Development History
• This material was prepared by Cohort 3
students in the Fall of 2002:
– Ron Anger
– Jim Gregoire
– Guillermo Jimenez
– Bob Ognjanovski
– Rob Spinks
2
Need for Testing
•
•
•
•
High Complexity
Mass Production
High cost of replacement in the field
“The earlier the faulty part is rejected,
the cheaper it is”
• Testing is no longer viewed as a “no
value add” or “hard to justify” expense
3
Need for Testing (Cont.)
• Testing is viewed as an integral part of
the manufacturing process
• Customer expectations of “0” PPM
• Increase in customer chargeback to
recover all costs associated with “faulty”
components
4
Problems with Testing
• Testing can comprise as much as 30% of the
cost of building a product
• Testing is difficult and time consuming due to
the large number of test steps, that must be
applied
• Testing is boring and considered not creative
• Designs are completed without testing in
mind
“ Testing is painful, Not Testing is suicidal.”
5
Why Testing:
Relative cost of finding and fixing errors
Cost vs. Error Type
20
18
16
14
12
Cost ($) 10
8
6
4
2
0
Requirements
Design
Coding
Unit Test
Acceptance
Maintenance
Error Type
6
Design for Testability
• Introduction to DFT
•
•
•
•
•
•
•
•
•
Key Principles in DFT
DFT Considerations
DFT Process
DFT in Hardware development
DFT in Software development
DFT, Reliability, and Robustness
Examples of DFT techniques
Heuristics
References
7
Related DFXs
•
•
•
•
•
•
•
Design for Inspectability
Design for Dimensional control
Design for Serviceability
Design for Diagnostics
Design for Modularity
Design for Reliability
Design for Robustness
8
Testability
The IEEE Standard Glossary of Software Engineering
Terminology (1990) defines testability as:
"(1) the degree to which a system or
component facilitates the establishment of
test criteria and the performance of tests to
determine whether those criteria have been
met, and (2) the degree to which a
requirement is stated in terms that permit
establishment of test criteria and
performance of tests to determine whether
those criteria have been met."
9
10
Definitions of DFT
– Ability to generate, to evaluate and to
apply tests that improve quality and
minimizes time-to-profit
– Extent to which a design can be tested
for the presence of manufacturing, base
component, system, and/or field defects
– Measure of how easy it is to generate
test sets that have a high fault coverage
11
Design for Testability
– An initiative in the computer hardware industry
in the 1980’s
– Objectives:
•
•
•
Lowers the cost of manufacturing
Minimizes the design engineer's involvement in
production set-up
Improves cross-functional communication and
cooperation among design, engineering, and
manufacturing
12
Design for Testability (Cont.)
• Testing is more expensive in the short-term
but cheaper in the longer-term
• Lowers both production and life-cycle costs
• Decreases test times and virtually
eliminates harrowing production delays
• Guarantees more efficient diagnosis and
repair in the field
• Improves fault coverage
13
Design for Testability (Cont.)
• Testability must be engineered into the
product at the design stage itself, such that
optimal compromise is archived between
system maintainability and performance.
• To maximize its impacts, DFT must be
performed at all stages of the design –from
schematics –to design of subsystems – to
system integration
14
Test Coverage vs. DFT
Test Coverage (%)
100%
A
B
C
Number of Test Steps
A=Design done with testability in mind
B= Design made without Testability
in mind by a good fault coverage due
to large effort in making test steps
C=Design very difficult to test
15
Motivation, Goals, and
Benefits of DFT
•
•
•
•
•
Better fault coverage and fault isolation
Shorter testing time
Higher quality product
Shorter time-to-market
Lower life-cycle cost
16
Design for Testability
• Introduction to DFT
• Key Principles in DFT
•
•
•
•
•
•
•
•
DFT Considerations
DFT Process
DFT in Hardware development
DFT in Software development
DFT, Reliability, and Robustness
Examples of DFT techniques
Heuristics
References
17
Key Principles of DFT
• Interfaces that are standard, common, and
simple
• Accessible points
• Automated
• Self-test with onboard sensors
• Integrated (testing multiple components at the
same time)
• Testing in parallel (sweep gauges at the same
time)
• Testing one thing verifies many (Traction
control switch checks switch, MUX, cluster,…)
18
Key Principles of DFT
•
•
•
•
•
•
•
•
•
Identification of opportunities
Standardization
Simplification of interfaces
Adjustable
Tunable
Diagnostics
Indicators
Procedures
Location
19
Key Principles of DFT
•
•
•
•
•
•
•
•
•
Accessibility
Obstruction
Orientation
Visibility
Intuitive
Tools (not specialized)
Ergonomic
Non-destructive
Models/CAE
20
Design for Testability
• Introduction to DFT
• Key Principles in DFT
• DFT Considerations
•
•
•
•
•
•
•
DFT Process
DFT in Hardware development
DFT in Software development
DFT, Reliability, and Robustness
Examples of DFT techniques
Heuristics
References
21
Perspective of DFT
• Keywords in Testability:
– Understandability (The more information we have,
the smarter we test)
– Predictability
– Observability and Traceability (What we see is
what we test )
– Controllability (The better we can control it , the more
the testing can be optimized)
22
Perspective of DFT
• Keywords in Testability:
– Understandability (The more information we have,
the smarter we test)
– Predictability
– Observability and Traceability (What we see is
what we test )
– Controllability (The better we can control it , the more
the testing can be optimized)
23
Perspective of DFT (Cont.)
• DFT involves modifying the design in
such way that maximum controllability
and observability are attained.
• DFT is an approach in which the
component (SW or HW) is designed
from the start such that testing problems
do not arise during the product life-cycle
24
Evaluation of Component
Testing Capability
Four Levels of testing
• Level 1: Initial
– Constructed with ad-hoc testing
mechanism, testing format, and testing
functions
– More time in understanding behaviors,
debugging, and testing
25
Evaluation of Component
Testing Capability (Cont.)
• Level 2: Standardize
– Built to support pre-defined testing
mechanism & testing format
– Reduces cost of debugging and testing
– Extra programming overhead
26
Evaluation of Component
Testing Capability (Cont.)
• Level 3: Systematic
– Design with a set of systematic testing
mechanics
– Easy to monitor and to test the
components
– Reduce programming overhead
27
Evaluation of Component
Testing Capability (Cont.)
• Level 4: Customizable
– Design to facilitate the support of the
testing functions & customization
– Help to set-up testing for components
based software
28
Mechanisms to Increase
Component Testability
•
Framework-based Testing facility
– Well-defined framework (such as class
library) to add test code
– Simple and flexible to use
– Need component source code
29
Mechanisms to Increase
Component Testability (Cont.)
•
Built-in testing
– Need well-defined built-in mechanisms to
add test code
– High programming overhead during
component development
– No external support needed
30
Mechanisms to Increase
Component Testability (Cont.)
•
Automatic component wrapping for
testing
– Component wrapped inside program for
testing
– Low programming overhead
– Well-defined testing framework to interact
with testing tools
31
Design for Testability
• Introduction to DFT
• Key Principles in DFT
• DFT Considerations
• DFT Process
•
•
•
•
•
•
DFT in Hardware development
DFT in Software development
DFT, Reliability, and Robustness
Examples
Heuristics
References
32
DFT Process
1. Evaluate testability of system
architecture
2. Define testability requirements and
targets
3. Describe testability context
4. Perform testability reviews
5. Define required design changes
33
DFT Process (cont.)
6. Collect experience
7. Define General testing strategy and
standards
8. The design is not finished until final
testing requirements are defined and
accounted for
34
35
Design for Testability
•
•
•
•
Introduction to DFT
Key Principles in DFT
DFT Considerations
DFT Process
• DFT in Hardware development
•
•
•
•
•
DFT in Software development
DFT, Reliability, and Robustness
Examples
Heuristics
References
36
DFT in Hardware
Development
• Test generation for large circuits is
very time consuming. One way to
get around this problem is to
constrain or to modify the design
in order to make test generation
easier.
37
DFT in Hardware
Development
• Most DFT techniques are targeted
to sequential circuits where test
generation is usually a difficult
problem
38
DFT in Hardware
Development
• If testing is not considered during
the design phase, then very low
fault coverage and high test
generation times can result.
39
DFT in Hardware
Development
• The objective of DFT is to improve
the controllability and observability
of internal circuit nodes so that the
circuit can be tested more
efficiently and effectively
40
DFT in Hardware
Development
Controllability:
• Ability to set or to reset internal nodes
from the primary inputs
Observability:
• Ability to observe the value of an
internal node at the primary outputs
41
DFT in Hardware
Development
• DFT attempts to improve circuit
testability by making the internal
nodes more controllable and
observable
42
DFT in Hardware
Development
• Benefits in implementing DFT in
HW development:
–
–
–
–
Shorter time-to-market
Reduced test time
Less expensive testing equipment
Yield learning, which is often overlooked
43
DFT in Hardware
Development
• Sacrifices in implementing DFT in
HW development:
–
–
–
–
Increased area of components
More pins on printed circuit boards(PCB)
Increased PCB area
Degraded performance on the circuits
44
Design for Testability
•
•
•
•
•
Introduction to DFT
Key Principles in DFT
DFT Considerations
DFT Process
DFT in Hardware development
• DFT in Software development
•
•
•
•
DFT, Reliability, and Robustness
Examples of DFT techniques
Heuristics
References
45
DFT in Software development
•
•
Most complex modern systems are a
blend of Software and Hardware
Testability analysis of a system is
incomplete without adequately
accounting for the effect of software
46
Challenging Problems in
Software Testing
– Software is usually much more
complicated than hardware
– Typically, about 40 to 50% of the overall
development budget is spent on testing
– Absence of “Known good” response
– Lack of testing models, adequate testing
criteria, and testing methods
– Software flaws are design flaws
47
Software Verification
The IEEE Standard Glossary of Software
Engineering Terminology (1990) defines software
verification to be the
“Process of evaluating a system or
component to determine whether
the products of a given
development phase satisfy the
conditions imposed at the start of
that phase."
48
Software Testability
Software testability can be defined
as “the probability that a piece of
software will fail on its next
execution during testing (with a
particular assumed input
distribution) if the software
includes a fault.”
49
True Reliability
Software
Testing
Software Code
Software
Testability
Formal
Verification
50
Design for Testability
•
•
•
•
•
•
Introduction to DFT
Key Principles in DFT
DFT Considerations
DFT Process
DFT in Hardware development
DFT in Software development
• DFT, Reliability, and Robustness
• Examples of DFT Techniques
• Heuristics
• References
51
52
53
DFT, Reliability, and
Robustness
•
Testability: “A design characteristic
allowing the following to be
determined with a given confidence, in
specified time and condition (noise):
location of any faults, whether an item
is inoperable, is operable but
degraded, and/or is operable”.
54
DFT, Reliability, and
Robustness
•
•
Reliability - the probability that the
System will perform its intended
function over time under specific
operating conditions
Reliability - Targets may be set on the
commodity or by specific tests used to
age the commodity and account for
the noise factors.
55
DFT, Reliability, and
Robustness
•
•
Key Life Testing – A method to
demonstrate Reliability and
Robustness by combining the primary
stresses into one test or a series of
tests on the same System.
Noise Factors – All noise factors
should be accounted for in the
appropriate testing (ex. DVP)
56
DFT, Reliability, and
Robustness
•
•
•
The component/subsystem/system MUST
consistently perform its ideal function in the
presence of uncontrollable influences (NOISE
FACTORS).
Noise factors MUST be included in testing plans
used to demonstrate testability and reliability
Noise Factors must be identified and linked to
Potential Failure Modes and Design Verification
Testing Plans to achieve an appropriate robustness
using reliability metric(s) to assess the consistent
performance of the System design.
57
DFT, Reliability, and
Robustness
•
•
Functional performance targets should
be established during the development
of program-specific System Design
Specification, P-diagrams, and FMEAs
Where individual component targets
are not available or appropriate, the
subsystem or system target will be
referenced
58
DFT, Reliability, and
Robustness
Design Validation
•
Targets for both the SOFT and the HARD reliability
failures are to be established and to be documented
in the component Design Verification Plan (DVP).
These testing targets and criteria are to reflect
customer expectations for the useful life of the
component/subsystem/ system ideal function(s). Use
of generic “failure levels” are not acceptable, as they
may not sufficiently represent the customer
expectations for product reliability
SOFT (degraded performance to an unacceptable level)
HARD (product function ceases)
59
DFT, Reliability, and
Robustness
Design Validation

Any product testing plan MUST include:


The range of critical Noise Factors that the
component/subsystem/system will be exposed to
during the System Useful Life
Compounded noise factors to create worst-case
noise scenarios (i.e. min/max levels of part
tolerance (dimension, strength, smoothness)
against an extreme set of external noises (i.e.
temperature, humidity, user conditions).
60
DFT, Reliability, and
Robustness

Testing Matrix
Testing metrics must include component,
subsystem, and system test samples in either:





Key Life Testing (KLT)
Test-to-failure (Weibull analysis)
Signal-to-Noise Ratio (Taguchi methods for
robustness)
Comparative testing (testing against either a
competitors’ or surrogate component)
Component, subsystem, and system level testing
(a weak test)
61
Design for Testability
•
•
•
•
•
•
•
Introduction to DFT
Key Principles in DFT
DFT Considerations
DFT Process
DFT in Hardware development
DFT in Software development
DFT, Reliability, and Robustness and Robustness
• Examples of DFT Techniques
• Heuristics
• References
62
Example of DFT Technique:
In-circuit Test
•
•
•
•
Highly cost-effective test approach
Testing access made with bed-of-nail-fixture
Highly automated
Total nodal access (Test Points) through
devices (i.e. pins, test pads, connectors or
vias)
• Verifies the electrical characteristics of each
component
63
Example of DFT Technique:
Built-In Self-Test (BIST)
• Implementation of a different kind of
logic in the design so it can test itself
• BIST can be categorized in:
– Online BIST: Testing is done while the
system is in normal operation or during idle
mode
– Offline BIST: System is brought into a
testing mode at predetermined regular
intervals
64
Example of DFT Technique:
Logical Built-In Self-Test (LBIST)
• Used to test standard cell logic
• A State machine is used to drive
pseudo-random vectors into scan
chains and then the output of the chain
is compressed into a signature value to
be scanned out at the end of test
65
Example of DFT Technique:
Boundary-Scan
• New practical testability tool.
• Initiated by Joint European Test Action Group
• Provides the ability to develop a test to
exercise all devices pins with a limited
amount of effort
• Extra control lines must be added to the
device to support the boundary-scan function
• It is intended to check for shorts or open
connections between ICs mounted on a
circuit board
66
Example of DFT Technique:
Automatic Test Pattern Generation (ATPG)
• Reduce the volume of data needed to test
each device to the highest possible coverage
• Unlike functional test vector, ATPG
specifically targets structural defects or faults
• Includes Advance Pattern Compression and
optimization techniques
67
Example of DFT Technique:
ATPG (Cont.)
• Advance Pattern Compression: 1) Static
Compression Technique (eliminate redundant
test from a given pattern-it does not detect
new faults) and 2) Dynamic Compression
Technique (multiple faults are targeted during
test pattern generation itself)
• Pattern Optimization Capability: Order pattern
sets from the most effective (highest test
coverage) to least effective pattern
68
Example of DFT Technique:
Full-Scan Design
• All circuits are placed in a Scan chain,
and values are scanned before and
after each test vector
• Straightforward ATPG problems
• Guarantee high coverage
• High-speed testing
69
Example of DFT Technique:
Full-Scan Design (Cont.)
• Disadvantages:
– Some designs are not able to abide by
design rules in all cases
– Area overhead (10-20% additional area
dedicated to testing), routing difficulties
– Timing impact
– Many testing cycles required on testers
70
Example of DFT Technique:
Static Fault Analysis
• Used as a rapid means to assess the
inherent testability of a system
• Identifies undetectable faults, ambiguity
groups, and redundant tests
• Identifies the topological testability
limitations of the system, and makes
DFT recommendations to overcome
them
71
Example of DFT Technique:
Testability Engineering and Maintenance System (TEAMS)
• Graphical software tools for diagnostic
model development and analysis
• Integrates a unique multi-signal flow
graph modeling methodology
• Integrate various analysis techniques
for performing testability analysis and
design for testability
72
Example of DFT Technique:
TEAMS (Cont.)
• Examples of problems that TEAMS can
solve:
– With a given set of tests, can all failures be
detected?
– What testing should be used and where
should they be located, so all the faults can
be isolated in minimal time and/or cost?
73
Example of DFT Technique:
TEAMS (Cont.)
• Examples of problems that TEAMS can
solve (Cont.):
– What is the most efficient sequence of
testing that will isolate all the failures?
– What percent of modules, pulled as
“faulty”, are actually OK?
– Are all the components within the system
reliable enough to survive the entire
mission?
74
Design for Testability
•
•
•
•
•
•
•
•
Introduction to DFT
Key Principles in DFT
DFT Considerations
DFT Process
DFT in Hardware development
DFT in Software development
DFT, Reliability, and Robustness and Robustness
Examples of DFT Techniques
• Heuristics
• References
75
Heuristics
• Prototype designs work, the problems
show up later.
• Diagnostics are highly efficient in finding
solved problems.
• Murphy’s law applies 95% of the time.
The other 5% we are on coffee breaks.
• When all but one wire, in a group,
switches, the one will switch too.
76
Heuristics
• Worst Case tolerances never add – but
when they do they are in our best
customer’s machine
• Map your testing strategy and your
design approach with respect to
inheritance hierarchies
• Make control structures explicit
• Don’t squeeze the code
77
Heuristics
• The percent of errors (bugs) left after
software validation is proportional to the
percent of errors found during validation
78
Testability Challenges... the
Management Issue
• Because DFT is essentially a
management issue and not a
technology issue, any testability effort
must have management's full
commitment and support if it is to
succeed
79
The Testability Challenge
• Regardless of the trends in system
testing capability, the basic challenge
for test engineers is not to change the
design, but rather to make the designer
a believer in testability.
80
Design for Testability
•
•
•
•
•
•
•
•
•
Introduction to DFT
Key Principles in DFT
DFT Considerations
DFT Process
DFT in Hardware development
DFT in Software development
DFT, Reliability, and Robustness and Robustness
Examples of DFT Techniques
Heuristics
• References
81
References
http://www.teamqsi.com/: Qualtech System Inc WEB Page.
Electronic News: Decreasing the Cost of Testing with Automatic Test
Pattern Generation
Integrated Diagnostics Toolset. IEEE Autotest Conference (1997).
Integrated Process for Fault Diagnosis. IEEE Aerospace Conference
(1999).
82
References
S. Deb, K.R. Pattipati, V. Raghavan, M. Shakeri, and R. Shrestha, “Multisignal Flow Graphs: A Novel Approach for System Testability Analysis
and Fault Diagnosis,” IEEE Aerospace and Electronics Magazine, May
1995, pp. 14-25. (Winner of the Best Technical Paper Award at the
1994 IEEE AUTOTEST Conference, Anaheim, CA, September 1994).
IEEE Standard Glossary of Software Engineering Terminology (1990)
http://www.cigitallabs.com/resources/definitions/testability.html
http://www.ate.agilent.com/emt/industry/testabilityguidelines/index.shtml
83
References
Phillips, Jeffery C., “Essential Testability Guidelines for current Technology.” IEEE
computer society press reprint, Los Alamitos, CA 90720 1993
Pettichord, Bret., “Design for Testability”,
http://www.io.com/~wazmo/papers/design_for_testability_PNSQC.pdf 2002
Illlman, Richard., “Design for testability: separating the myths from reality.
http://www.eetimes.com/in_focus/silicon_engineering/OEG20020718S002
18-July-2002
Olausson, Mikeal, and Wiklund, Daniel. “Introduction to Design for Testability.”
http://www.ida.liu.se/~zebpe/teaching/test/lec6.pdf
2001
84
5
References
Gao, Jerry. “Component Testability and Component Testing Challenges”,
Technical Report
http://www.sei.cmu.edu/pacc/cbse2000/papers/18/18.pdf in San Jose
State University, in 2000.
Neal, Bob. “Test for Designability.”
Technical Report Agilent Technology
15 February 2003
http://www.home.agilent.com/upload/cmc_upload/All/Bneal_dft_dfd.pdf
85