PRESENTATION ON
DFR ROI: Calculating ROI
When Implementing a DFR
Program
BY
Mike Silverman, CRE, Ops A La Carte LLC
Agenda
• Background/Introduction
5 min
• Reliability Program Assessment
15 min
• DFR ROI Calculator
15 min
• Example
10 min
• Summary
5 min
• Questions
10 min
© Ops A La Carte LLC 2012
Introduction
Ops A La Carte
– Founded in 2001
– Named top 10 fastest growing, privately-held
companies in the Silicon Valley in 2006 and
2009 by the San Jose Business Journal.
– Over 1500 projects completed in 11 years
– Over 500 Customers in over 30 countries
– Over 100 different industries, 6 main verticals
• CleanTech, MedTech, Telecom, Defense, Oil/Gas,
Consumer
– We run FREE monthly webinars.
© Ops A La Carte LLC 2012
Upcoming Events we are at
Accelerated Stress Test and Reliability
(ASTR) Workshop
Oct 17-19, 2012, Toronto
Ops A La Carte's Mike Silverman will be giving a tutorial on “30 Years of HALT – What Have We Learned".
• Medical Device/Mfg Conference (MD&M)
Oct 31 - Nov 2, Minneapolis
Ops A La Carte's Mike Silverman will be giving a day tutorial on Medical Reliability Testing.
© Ops A La Carte LLC 2012
Other Reliability Events
Certified Quality Engineer Preparation Class
Date(s): Oct 16 to Nov 27, 2012
Time: 6pm-10pm one night a week, 7 weeks
Location: San Jose, CA and Webinar
http://www.opsalacarte.com/Pages/education/edu_10cqe.htm
We have been teaching this class for 9 years and our pass rate is
50% higher than for those that did not take our prep class.
© Ops A La Carte LLC 2012
Upcoming Reliability Webinars
Advances in HALT Calculator - New MTBF and
ROI Calculator
Date: Oct 3, 2012, 11:30am
Location: Webinar
This is our next webinar in our free webinar series
See link from our website www.opsalacarte.com
Since we introduced our HALT Calculator 3 years ago, we have made some important
improvements
1) Calculator now resides on the cloud
2) We have developed methods for using calculator as input to HALT plan
3) We have added an ROI Calculator
4) We have added an MTBF estimator
5) We have added several new factors, including duty cycle and variable confidence intervals
All of these will be explained and demonstrated in this 1 hour webinar
© Ops A La Carte LLC 2012
PRESENTATION ON
DFR ROI: Calculating ROI
When Implementing a DFR
Program
BY
Mike Silverman, CRE, Ops A La Carte LLC
© Ops A La Carte LLC 2012
Background
• Last year we presented a paper on Design for
Reliability (DFR), reviewing the benefits of a
good DFR program and included some of the
essential building blocks of DfR along with
pointing out some erroneous practices that
people today are using today.
© Ops A La Carte LLC 2012
Background (cont’d)
• We discussed a good DFR Program having the
following attributes:
– Setting Goals at the beginning of the program and
then developing a plan to meet the goals.
– Having the reliability goals being driven by the
design team with the reliability team acting as
mentors.
– Providing metrics so that you have checkpoints on
where you are against your goals.
– Writing a Reliability Plan (not only a test plan) to
drive your program.
© Ops A La Carte LLC 2012
Background (cont’d)
• A Good DFR Program must choose the best
tools from each area of the product life cycle
– Identify
– Design
– Analyze
– Verify
– Validate
– Monitor and Control
• The DFR Program must then integrate the tools
together effectively
© Ops A La Carte LLC 2012
Introduction
• Today we will discuss a method we have
developed to calculate the Return on
Investment (ROI) from a Design for
Reliability (DFR) program, also known as
the DFR ROI.

The DFR ROI is the cornerstone of our
new Ounce of Prevention Solution
This is the OPS in Ops A La Carte!
© Ops A La Carte LLC 2012
Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012
Calculating Improvement in
Warranty
• To calculate your Improved Warranty Rate,
you will need to determine both your
current Reliability Maturity Level as well as
your desired Reliability Maturity Level.
• To determine these, you will need to
conduct a Reliability Program Assessment
(RPA).
© Ops A La Carte LLC 2012
RELIABILITY
PROGRAM
ASSESSMENT
© Ops A La Carte LLC 2012
Reliability Program Assessment
© Ops A La Carte LLC 2012
Assessment Motivation
• Identify systemic changes that impact
reliability
– Tie into culture and product
– Both enjoy benefits
• Provides roadmap for activities that
achieve results
– Matching of capabilities and expectations
– Cooperative approach
• Save Money (ROI)
© Ops A La Carte LLC 2012
When to perform an
Assessment?
•
•
•
•
•
Entering into a new market
Product reliability is below target level
New personnel
New technology
Product design > 50% different than
previous.
© Ops A La Carte LLC 2012
Steps Involved in an
Assessment
•
•
•
•
•
Selecting People to Survey
Selecting Survey Topics (from IEEE1624)
Develop a Scoring System
Results and Meaning
Data Analysis/Reliability Maturity Matrix
(our new Matrix is aligned with IEEE1624)
© Ops A La Carte LLC 2012
Selecting People to Survey
•
•
•
•
•
•
•
•
Hardware Engineers
Software Engineers
Test Engineers
Manufacturing Engineers
Quality/Reliability Engineers
Program Management
Procurement
others?
© Ops A La Carte LLC 2012
Survey Topics
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Do you have a Reliability Plan? Reliability Goals?
Is reliability an integral part of strategic business planning?
Do reliability requirements exist for all engineering projects?
Is there a reliability plan for each engineering project?
What training relevant to reliability is provided to employees?
What reliability analysis is conducted?
Are design rules used effectively to avoid known failures?
How effective is testing?
How are suppliers managed
How is parts selection managed?
How is parts supply managed?
How effective is failure data tracking and analysis?
© Ops A La Carte LLC 2012
Develop a Scoring System
Scoring:
4 = 100%, top priority, always done
3 = >75%, use normally, expected
2 = 25% - 75%, variable use
1 = <25%, only occasional use
0 = not done or discontinued
- = not visible, no comment
© Ops A La Carte LLC 2012
Example Survey Topic/Score
To what extent is FMEA used?

Design Engineer Response: Used only as a
troubleshooting tool


Manufacturing Engineer Response: Commonly used on
critical design elements


Score = 1
Score = 3
Reliability Engineer Response: Always used on all
products

Score = 4
Results: Score 2.6
•Comment: Clearly a disconnect between
reliability and design engineering – indicative
of a problem with the tool.
•
© Ops A La Carte LLC 2012
Results & Meaning
• Looking for trends, gaps in process, skill
mismatches, over analysis, under analysis.
• Looking for differences across the
organization, pockets of excellence, areas
with good results
• Process provides snapshot of current system
• No one tool make an entire reliability
program. The tools need to match the needs
of the products and the culture.
• Check step is critical before moving to
recommendation around improvement plan
© Ops A La Carte LLC 2012
Summary of Past Assessments
• What Companies Are
Weak at
–
–
–
–
Goal setting/Planning
Repair/warranty invisible
Lessons learned capture
Single owner of product
reliability
– Multiple defect tracking
systems
– Reliability Integration
– Statistics
© Ops A La Carte LLC 2012
Reliability Maturity Matrix
• 5 levels of maturity
• Based on IEEE 1624: “IEEE Standard for
Organizational Reliability Capability”
• Similar to Crosby’s Quality Maturity
• On the following page is a matrix based on
Crosby’s as an example.
• Read across each row and find the statement
that seems most true for your organization.
• The center of mass of the levels is the
organization’s overall level.
© Ops A La Carte LLC 2012
Reliability Maturity Matrix
ATTRIBUTES
Stage1:
Uncertainty
4. Feedback
Process
3. Engineering
2. Product 1. Management
Reqts.
1.1Understanding No comprehension of
reliability as a
&Attitude
management tool. Tend
1.2Status
to blame engineering
for ‘reliability problems’
Reliability is hidden in
manufacturing or
engineering
departments. Reliability
testing probably not
done. Emphasis on
initial product
functionality.
Not done other than
anecdotally
1.3MeasuredCost
ofUnreliability
Informal or nonexistent
2.1Requirement
&Planning
Stage2:Awakening Stage3:
Enlightenment
Stage4:Wisdom
Stage5:Certainty
Recognizing that reliability
management may be of value but not
willing to provide money or time to
make it happen.
Still learning more about reliability
management. Becoming supportive
and helpful.
Participating. Understand absolutes of
reliability management. Recognize their
personal role in continuing emphasis.
Consider reliability management an essentia
part of company system.
A stronger reliability leader appointed,
yet main emphasis is still on an audit
of initial product functionality.
Reliability testing still not performed.
Reliability manager reports to top
management, with role in management
of division.
Reliability manager is an officer of
company; effective status reporting and
preventive action. Involved with consumer
affairs.
Reliability manager is on board of directors.
Prevention is main concern. Reliability is a
thought leader.
Direct warranty expenses only
Warranty, corrective action materials
and engineering costs monitored
Customer and lifecycle unreliability costs
determined and tracked
Lifecycle cost reduction done through produc
reliability improvements
Basic requirements based on
customer requirements or standards.
Plans have required activities.
Requirements include environment and
use profiles. Some apportionment.
Plans have more details with regular
reviews.
Training for engineering community for
key reliability related processes.
Managers trained on reliability and
lifecycle impact.
Formal use of FMEA. Field data analysis
of similar products used to adjust
predictions. Design changes cause reevaluation of product reliability
Detailed reliability test plan with sample
size and confidence limits. Results used
for design changes & vendor evals.
Assessments and audit results used to
update AVL. Field data and failure
analysis related to specific vendors.
Plans are tailored for each project and
projected risks. Use of distributions for
environmental and use conditions.
Contingency planning occurs. Decisions
based on business or market considerations
Part of strategic business plan.
Reliability and statistics courses tailored
for design and manufacturing engineers.
Senior managers trained on reliability
impact on business.
Predictions are expressed as distributions
and include confidence limits.
Environmental and use conditions used for
simulation and testing.
Accelerated tests and supporting models
used. Testing to failure or destruct limits
conducted
Vendor selection includes analysis of
vendor’s reliability data. Suppliers conduct
assessments and audit of their suppliers.
Root cause analysis used to update
AVL and prediction models. Summary
of analysis results disseminated.
Focus is on failure mechanisms. Failure
distribution models updated based on
failure data
New technologies and reliability tools tracked
and training adjusted to accommodate.
Reliability training actively supported by top
management.
Lifecycle cost considered during design.
Stress and damage models created and
used. Extensive risk analysis for new
technologies.
Test results used to update component stres
and damage models. New technologies
characterized.
Changes in environment, use profile, or
design, trigger vendor reliability assessment
Component parameters and reliability
monitored for stability
Customer satisfaction relationship to product
failures is understood. Use of prognostic
methods to forestall failure.
Internal reviews of reliability processes
and tools. Failure mechanisms regularly
monitored and used to update models and
test methods
Identified failure mechanisms addressed
in all products. Advanced modeling
techniques explored and adopted. Formal
and effective lessons learned process
exists.
“Failure prevention is a routine part of our
operation.”
2.2Training&
Development
Informally available to
some, if requested
Select individuals trained in concepts
and data analysis. Available training
for design engineers
3.1Analysis
Nonexistent or solely
based on
manufacturing issues
3.2Testing
Primarily functional
3.3SupplyChain
Management
Selection based on
function and price
4.1FailureData
Tracking&
Analysis
4.2Validation&
Verification
Failures during function
testing may be
addressed
Point estimates and reliance on
handbook parts count methods. Basic
identification and listing of failure
modes and impact
Generic test plan exists with reliability
testing only to meet customer or
standards specifications
Approved vendor list maintained.
Audits based on issues or with critical
parts. Qualification primarily based on
vendor datasheets.
Pareto analysis of field return and
internal testing. Failure analysis relies
on vendor support.
Informal and based on
individuals rather than
process
Basic verification that plans are
followed. Field failure data regularly
reported.
Supplier agreements around reliability
monitored. Failure modes regularly
monitored.
Design and process change followed.
Corrective action process includes
internal and vendor engagement.
Effectiveness of corrective actions
tracked over time. Identified failure
modes addressed in other product.
Improvement opportunities identified as
environment and use profiles change.
“Through commitment and reliability
improvement we are identifying and
resolving our problems.”
Nonexistent or informal
4.3Reliability
Improvements
5.Prevailing
Sentiment
“We don’t know why we
have problems with
reliability”
“Is it absolutely necessary to always
have problems with reliability?”
Reliability predictions match observed field
reliability.
New technologies evaluated and adopted to
improve reliability. Design rules updated
based on field failure analysis.
“We know why we do not have problems with
reliability.”
Reliability Maturity Matrix
(cont’d)
• Lets look at one row to get a better understanding
ATTRIBUTE Stage1:
1.1Under‐
standing
&
Attitude
Stage2:
Uncertainty
Awakening
No
comprehension of
reliability as a
management
tool. Tend to
blame
engineering
for ‘reliability
problems’
Recognizing
that reliability
management
may be of
value but not
willing to
provide
money or
time to make
it happen.
Stage3:
Enlighten‐
ment
Still learning
more about
reliability
management
Becoming
supportive
and helpful.
© Ops A La Carte LLC 2012
Stage4:
Wisdom
Stage5:
Certainty
Participating.
Understand
absolutes of
reliability
management
Recognize
their
personal role
in continuing
emphasis.
Consider
reliability
management an
essential
part of
company
system.
Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012
Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012
Warranty vs. Maturity by
Industry
% of Revenue
SECTOR
Automotive Manufacturer
Auto Parts Suppliers
Aerospace and Marine
Computers
Telecom
Semiconductor
Consumer Electronics
Medical and Scientific
Data Storage
PC Peripherals
Appliances and HVAC
Homebuilders
Building Materials
Power Generation
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
9.2
4.6
2.3
1.15
0.58
4.8
2.4
1.2
0.6
0.3
2.8
1.4
0.7
0.35
0.18
Some of above data courtesy of Eric Arnum, Warranty Week Magazine
© Ops A La Carte LLC 2012
Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012
Calculating the Cost of Repair
• When calculating, include the following
processes:
– Call center processes (even if you outsource)
– Support organization processes (even if you outsource
this function)
• On-site repair (service engineers/authorized service suppliers)
– Supply chain processes
• Spare parts management (usually as an overhead)
• Spare parts logistics
– Manufacturing/factory resources expensed against above
processes (usually as overhead expenses)
• Return parts testing process
– Reliability/Quality engineering resources
© Ops A La Carte LLC 2012
Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012
Cost of New Reliability Program
•
•
•
•
•
•
•
Extra engineering support for tools
Consulting/test lab time for extra tests
Test equipment rental
Cost of units under test
Cost of troubleshooting/failure analysis
Cost of product changes/ECNs
Training
© Ops A La Carte LLC 2012
Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012
Savings from Losing
Fewer Customers
• A. Calculate # of customers lost over the past year
due to reliability/quality issues
• B. Calculate # of customers lost over the past year for
unexplained reasons
–
–
–
–
–
If Stage 1= Multiply by 80%
If Stage 2 = Multiply by 60%
If Stage 3 = Multiply by 40%
If Stage 4 = Multiply by 20%
If Stage 5 = Multiply by 0%
• Calculate Number of Fewer Customers Lost
• Determine average amount of revenue per customer
• Multiply Difference in # of customers lost x revenue
per customer
© Ops A La Carte LLC 2012
DFR ROI
CALCULATION
© Ops A La Carte LLC 2012
DFR ROI Calculation
1. Perform Assessment/Determine Maturity Level
2. Determine maturity level desired
3. Determine your industry sector
4. Subtract Warranty % between two levels
5. Calculate Average Cost of Repair
6. Calculate Cost of New Reliability Program
7. Calculate Savings from Fewer Lost Customers
8. Determine Volume of Shipments
9. Calculate Repair Cost Savings
10.Calculate ROI
© Ops A La Carte LLC 2012
ROI EXAMPLE:
MEDICAL PRODUCT
© Ops A La Carte LLC 2012
DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
© Ops A La Carte LLC 2012
Reliability Maturity Matrix - start
ATTRIBUTES
Stage1:
Uncertainty
4. Feedback
Process
3. Engineering
2. Product 1. Management
Reqts.
1.1Understanding No comprehension of
reliability as a
&Attitude
management tool. Tend
1.2Status
to blame engineering
for ‘reliability problems’
Reliability is hidden in
manufacturing or
engineering
departments. Reliability
testing probably not
done. Emphasis on
initial product
functionality.
Not done other than
anecdotally
1.3MeasuredCost
ofUnreliability
Informal or nonexistent
2.1Requirement
&Planning
Stage2:Awakening Stage3:
Enlightenment
Stage4:Wisdom
Stage5:Certainty
Recognizing that reliability
management may be of value but not
willing to provide money or time to
make it happen.
Still learning more about reliability
management. Becoming supportive
and helpful.
Participating. Understand absolutes of
reliability management. Recognize their
personal role in continuing emphasis.
Consider reliability management an essentia
part of company system.
A stronger reliability leader appointed,
yet main emphasis is still on an audit
of initial product functionality.
Reliability testing still not performed.
Reliability manager reports to top
management, with role in management
of division.
Reliability manager is an officer of
company; effective status reporting and
preventive action. Involved with consumer
affairs.
Reliability manager is on board of directors.
Prevention is main concern. Reliability is a
thought leader.
Direct warranty expenses only
Warranty, corrective action materials
and engineering costs monitored
Customer and lifecycle unreliability costs
determined and tracked
Lifecycle cost reduction done through produc
reliability improvements
Basic requirements based on
customer requirements or standards.
Plans have required activities.
Requirements include environment and
use profiles. Some apportionment.
Plans have more details with regular
reviews.
Training for engineering community for
key reliability related processes.
Managers trained on reliability and
lifecycle impact.
Formal use of FMEA. Field data analysis
of similar products used to adjust
predictions. Design changes cause reevaluation of product reliability
Detailed reliability test plan with sample
size and confidence limits. Results used
for design changes & vendor evals.
Assessments and audit results used to
update AVL. Field data and failure
analysis related to specific vendors.
Plans are tailored for each project and
projected risks. Use of distributions for
environmental and use conditions.
Contingency planning occurs. Decisions
based on business or market considerations
Part of strategic business plan.
Reliability and statistics courses tailored
for design and manufacturing engineers.
Senior managers trained on reliability
impact on business.
Predictions are expressed as distributions
and include confidence limits.
Environmental and use conditions used for
simulation and testing.
Accelerated tests and supporting models
used. Testing to failure or destruct limits
conducted
Vendor selection includes analysis of
vendor’s reliability data. Suppliers conduct
assessments and audit of their suppliers.
Root cause analysis used to update
AVL and prediction models. Summary
of analysis results disseminated.
Focus is on failure mechanisms. Failure
distribution models updated based on
failure data
New technologies and reliability tools tracked
and training adjusted to accommodate.
Reliability training actively supported by top
management.
Lifecycle cost considered during design.
Stress and damage models created and
used. Extensive risk analysis for new
technologies.
Test results used to update component stres
and damage models. New technologies
characterized.
Changes in environment, use profile, or
design, trigger vendor reliability assessment
Component parameters and reliability
monitored for stability
Customer satisfaction relationship to product
failures is understood. Use of prognostic
methods to forestall failure.
Internal reviews of reliability processes
and tools. Failure mechanisms regularly
monitored and used to update models and
test methods
Identified failure mechanisms addressed
in all products. Advanced modeling
techniques explored and adopted. Formal
and effective lessons learned process
exists.
“Failure prevention is a routine part of our
operation.”
2.2Training&
Development
Informally available to
some, if requested
Select individuals trained in concepts
and data analysis. Available training
for design engineers
3.1Analysis
Nonexistent or solely
based on
manufacturing issues
3.2Testing
Primarily functional
3.3SupplyChain
Management
Selection based on
function and price
4.1FailureData
Tracking&
Analysis
4.2Validation&
Verification
Failures during function
testing may be
addressed
Point estimates and reliance on
handbook parts count methods. Basic
identification and listing of failure
modes and impact
Generic test plan exists with reliability
testing only to meet customer or
standards specifications
Approved vendor list maintained.
Audits based on issues or with critical
parts. Qualification primarily based on
vendor datasheets.
Pareto analysis of field return and
internal testing. Failure analysis relies
on vendor support.
Informal and based on
individuals rather than
process
Basic verification that plans are
followed. Field failure data regularly
reported.
Supplier agreements around reliability
monitored. Failure modes regularly
monitored.
Design and process change followed.
Corrective action process includes
internal and vendor engagement.
Effectiveness of corrective actions
tracked over time. Identified failure
modes addressed in other product.
Improvement opportunities identified as
environment and use profiles change.
“Through commitment and reliability
improvement we are identifying and
resolving our problems.”
Nonexistent or informal
4.3Reliability
Improvements
5.Prevailing
Sentiment
“We don’t know why we
have problems with
reliability”
“Is it absolutely necessary to always
have problems with reliability?”
Reliability predictions match observed field
reliability.
New technologies evaluated and adopted to
improve reliability. Design rules updated
based on field failure analysis.
“We know why we do not have problems with
reliability.”
Reliability Maturity Matrix - desired
ATTRIBUTES
Stage1:
Uncertainty
4. Feedback
Process
3. Engineering
2. Product 1. Management
Reqts.
1.1Understanding No comprehension of
reliability as a
&Attitude
management tool. Tend
1.2Status
to blame engineering
for ‘reliability problems’
Reliability is hidden in
manufacturing or
engineering
departments. Reliability
testing probably not
done. Emphasis on
initial product
functionality.
Not done other than
anecdotally
1.3MeasuredCost
ofUnreliability
Informal or nonexistent
2.1Requirement
&Planning
Stage2:Awakening Stage3:
Enlightenment
Stage4:Wisdom
Stage5:Certainty
Recognizing that reliability
management may be of value but not
willing to provide money or time to
make it happen.
Still learning more about reliability
management. Becoming supportive
and helpful.
Participating. Understand absolutes of
reliability management. Recognize their
personal role in continuing emphasis.
Consider reliability management an essentia
part of company system.
A stronger reliability leader appointed,
yet main emphasis is still on an audit
of initial product functionality.
Reliability testing still not performed.
Reliability manager reports to top
management, with role in management
of division.
Reliability manager is an officer of
company; effective status reporting and
preventive action. Involved with consumer
affairs.
Reliability manager is on board of directors.
Prevention is main concern. Reliability is a
thought leader.
Direct warranty expenses only
Warranty, corrective action materials
and engineering costs monitored
Customer and lifecycle unreliability costs
determined and tracked
Lifecycle cost reduction done through produc
reliability improvements
Basic requirements based on
customer requirements or standards.
Plans have required activities.
Requirements include environment and
use profiles. Some apportionment.
Plans have more details with regular
reviews.
Training for engineering community for
key reliability related processes.
Managers trained on reliability and
lifecycle impact.
Formal use of FMEA. Field data analysis
of similar products used to adjust
predictions. Design changes cause reevaluation of product reliability
Detailed reliability test plan with sample
size and confidence limits. Results used
for design changes & vendor evals.
Assessments and audit results used to
update AVL. Field data and failure
analysis related to specific vendors.
Plans are tailored for each project and
projected risks. Use of distributions for
environmental and use conditions.
Contingency planning occurs. Decisions
based on business or market considerations
Part of strategic business plan.
Reliability and statistics courses tailored
for design and manufacturing engineers.
Senior managers trained on reliability
impact on business.
Predictions are expressed as distributions
and include confidence limits.
Environmental and use conditions used for
simulation and testing.
Accelerated tests and supporting models
used. Testing to failure or destruct limits
conducted
Vendor selection includes analysis of
vendor’s reliability data. Suppliers conduct
assessments and audit of their suppliers.
Root cause analysis used to update
AVL and prediction models. Summary
of analysis results disseminated.
Focus is on failure mechanisms. Failure
distribution models updated based on
failure data
New technologies and reliability tools tracked
and training adjusted to accommodate.
Reliability training actively supported by top
management.
Lifecycle cost considered during design.
Stress and damage models created and
used. Extensive risk analysis for new
technologies.
Test results used to update component stres
and damage models. New technologies
characterized.
Changes in environment, use profile, or
design, trigger vendor reliability assessment
Component parameters and reliability
monitored for stability
Customer satisfaction relationship to product
failures is understood. Use of prognostic
methods to forestall failure.
Internal reviews of reliability processes
and tools. Failure mechanisms regularly
monitored and used to update models and
test methods
Identified failure mechanisms addressed
in all products. Advanced modeling
techniques explored and adopted. Formal
and effective lessons learned process
exists.
“Failure prevention is a routine part of our
operation.”
2.2Training&
Development
Informally available to
some, if requested
Select individuals trained in concepts
and data analysis. Available training
for design engineers
3.1Analysis
Nonexistent or solely
based on
manufacturing issues
3.2Testing
Primarily functional
3.3SupplyChain
Management
Selection based on
function and price
4.1FailureData
Tracking&
Analysis
4.2Validation&
Verification
Failures during function
testing may be
addressed
Point estimates and reliance on
handbook parts count methods. Basic
identification and listing of failure
modes and impact
Generic test plan exists with reliability
testing only to meet customer or
standards specifications
Approved vendor list maintained.
Audits based on issues or with critical
parts. Qualification primarily based on
vendor datasheets.
Pareto analysis of field return and
internal testing. Failure analysis relies
on vendor support.
Informal and based on
individuals rather than
process
Basic verification that plans are
followed. Field failure data regularly
reported.
Supplier agreements around reliability
monitored. Failure modes regularly
monitored.
Design and process change followed.
Corrective action process includes
internal and vendor engagement.
Effectiveness of corrective actions
tracked over time. Identified failure
modes addressed in other product.
Improvement opportunities identified as
environment and use profiles change.
“Through commitment and reliability
improvement we are identifying and
resolving our problems.”
Nonexistent or informal
4.3Reliability
Improvements
5.Prevailing
Sentiment
“We don’t know why we
have problems with
reliability”
“Is it absolutely necessary to always
have problems with reliability?”
Reliability predictions match observed field
reliability.
New technologies evaluated and adopted to
improve reliability. Design rules updated
based on field failure analysis.
“We know why we do not have problems with
reliability.”
DFR ROI Calculation
1.
2.
3.
4.
Starting maturity level = 2
Desired maturity level = 3
Industry Sector = Medical and Scientific
Subtract Warranty % between two levels
© Ops A La Carte LLC 2012
Warranty vs. Maturity by Industry
% of Revenue
SECTOR
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
Automotive Manufacturer
Auto Parts Suppliers
Aerospace and Marine
Computers
Telecom
Semiconductor
Consumer Electronics
Medical and Scientific
Data Storage
PC Peripherals
Appliances and HVAC
Homebuilders
Building Materials
Power Generation
9.2
4.6
2.3
1.15
0.58
4.8
2.4
1.2
0.6
0.3
2.8
1.4
0.7
0.35
0.18
© Ops A La Carte LLC 2012
DFR ROI Calculation
1.
2.
3.
4.
Starting maturity level = 2
Desired maturity level = 3
Industry Sector = Medical and Scientific
Subtract Warranty % between two levels
– Warranty % Level 2 = 1.4%
– Warranty % Level 3 = 0.7%
– Difference = 0.7%
© Ops A La Carte LLC 2012
DFR ROI Calculation
1.
2.
3.
4.
5.
Starting maturity level = 2
Desired maturity level = 3
Industry Sector = Medical and Scientific
Warranty % Difference = 0.7%
Calculate Average Cost of Repair
© Ops A La Carte LLC 2012
Calculating the Cost of Repair
• Call center processes = $200
• Support organization processes (even if you
outsource this function) = $200
• Supply chain processes = $1000
• Manufacturing/factory resources expensed
against above processes = $300
• Reliability/Quality engineering resources = $300
TOTAL = $2K
© Ops A La Carte LLC 2012
DFR ROI Calculation
1.
2.
3.
4.
5.
6.
Starting maturity level = 2
Desired maturity level = 3
Industry Sector = Medical and Scientific
Warranty % Difference = 0.7%
Average Cost of Repair = $2K
Calculate Cost of New Reliability Program
© Ops A La Carte LLC 2012
Cost of New Reliability Program
•
•
•
•
•
•
•
Extra engineering support for tools = $10K
Consulting/test lab time for extra tests = $10K
Test equipment rental = $5K
Cost of units under test = $10K
Cost of troubleshooting/failure analysis = $10K
Cost of product changes/ECNs = $20K
Training = $10K
TOTAL = $75K
© Ops A La Carte LLC 2012
DFR ROI Calculation
1.
2.
3.
4.
5.
6.
7.
Starting maturity level = 2
Desired maturity level = 3
Industry Sector = Medical and Scientific
Warranty % Difference = 0.7%
Average Cost of Repair = $2K
Cost of New Reliability Program = $75K
Savings from Fewer Lost Customers
© Ops A La Carte LLC 2012
Savings from Losing
Fewer Customers
• A. Calculate # of customers lost over the past year
due to reliability/quality issues = 3
• B. Calculate # of customers lost over the past year for
unexplained reasons = 10
– If Stage 2 = Multiply by 60%
– If Stage 3 = Multiply by 40%
• Calculate Number of Fewer Customers Lost
–
–
–
Stage 2 = 3 + (10*0.6) = 9
Stage 3 = 3 + (10*0.4) = 7
Difference = 2
• Average amount of revenue per customer = $100K
• Multiply Difference in # of customers lost x revenue
per customer
$200K savings from fewer lost customers
© Ops A La Carte LLC 2012
DFR ROI Calculation
1.
2.
3.
4.
5.
6.
7.
8.
Starting maturity level = 2
Desired maturity level = 3
Industry Sector = Medical and Scientific
Warranty % Difference = 0.7%
Average Cost of Repair = $2K
Cost of New Reliability Program = $75K
Savings from Fewer Lost Customers = $200K
Volume of Shipments = 5,000/year
© Ops A La Carte LLC 2012
DFR ROI Calculation
1.
2.
3.
4.
5.
6.
7.
8.
9.
Starting maturity level = 2
Desired maturity level = 3
Industry Sector = Medical and Scientific
Warranty % Difference = 0.7%
Average Cost of Repair = $2K
Cost of New Reliability Program = $75K
Savings from Fewer Lost Customers = $200K
Volume of Shipments = 5,000/year
Repair Cost Savings = Volume * Warranty  *
Cost of Repair = 5000*0.7%*$2K = $70K
© Ops A La Carte LLC 2012
DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
4. Warranty % Difference = 0.7%
5. Average Cost of Repair = $2K
6. Cost of New Reliability Program = $75K
7. Savings from Fewer Lost Customers = $200K
8. Volume of Shipments = 5,000/year
9. Repair Cost Savings = $70K
10.Calculate ROI
© Ops A La Carte LLC 2012
Calculating ROI
ROI = Repair Savings
+ Fewer Lost Customers
- Reliability Prog Cost
ROI = $70K + $200K - $75K
ROI = $195K / year
© Ops A La Carte LLC 2012
Conclusion
• Today we have shown you a powerful new
tool, the DFR ROI Calculator.
• With this tool, you can
– Justify using new techniques
– Quantify how much improvement you will make
– Balance cost savings with improvements
© Ops A La Carte LLC 2012
Q&A
© Ops A La Carte LLC 2012
Contact Information
Ops A La Carte, LLC
Mike Silverman
Managing Partner
(408) 472-3889
mikes@opsalacarte.com
www.opsalacarte.com
© Ops A La Carte LLC 2012