Six Sigma and CMMI :
Showdown in Software
Process Engineering
Jonathan Beckham
Process Improvement Goals
• Process Goals
– Improve schedule and budget predictability
– Improve cycle time / time to market
– Increase productivity
– Improve quality (as measured by defects)
– Increase customer satisfaction
– Improve employee morale
– Decrease cost of quality
Process Improvement Goals
• Companies want to increase Return On Investment
What affects software?
People
Process
Technology
Six Sigma: What is it?
• According to David Card, (IEEE Software 2000),
Six sigma is “A generic quantitative approach to
improvement that applies to any process.”
• “Six Sigma is a disciplined, data-driven approach
and methodology for eliminating defects (driving
towards six standard deviations between the
mean and the nearest specification limit) in any
process -- from manufacturing to transactional
and from product to service.” –isixsigma.com
What’s the missing piece?
• In essence, six sigma is an approach to
finding the cause of business problems and
solving them making an impact on the
bottom line of a corporation and decreasing
variation for products.
• Six Sigma is more
of a methodology
or a philosophy.
Six Sigma Levels
• Six Sigma can be identified at three levels:
– Metric
– Methodology
– Philosophy
Metric
• 3.4 Defects Per Million
Opportunities (DPMO)
• DPMO allows you take
complexity of product/process
into account
• Refers to statistics. The +-6
standard deviations (sigmas)
away from the standard mean
• Most companies are within 3
standard deviations of the
mean
Metric
• The higher the
sigma value, the
better the
process is
capable of
producing defect
free results.
• To be in this six sigma range you must
have a quality product 99.9999998% of
the time.
Philosophy
• Reduce variation in your business and
make customer-focused, data driven
decisions
• Translate customer
needs into operational
measurements
Methodology
• Structured problem solving techniques and
roadmap
• Two primary sub-methodologies in Six
Sigma
– DMAIC
– DMADV
DMAIC
• Define-Measure-Analyze-Improve-Control
• Tool for incremental process improvement
of existing processes within an
organization that fall below required
specification
DMAIC cont.
• Define
– Define the problem or process to improve upon related to the customer
and goals
• Measure
– How can you measure this process in a systematic way?
• Analyze
– Analyze the process or problem and identify the way in which it can be
improved. What are the root causes of problems within the process?
• Improve
– Once you know the causes of the problems, present solutions for them
and implement them
• Control
– Utilize Statistical Process Control to continuously measure your results
and ensure you are improving
– Several Software Packages available to assist in measuring yield, defects
per million opportunities, etc.
DMADV
• Define-Measure-Analyze-Design-Verify
• Methodology for producing new processes
that meet the Six Sigma Quality levels
desired
• Similar to DMAIC, however, we have a
design stage here
• DFSS
– Design-For-Six-Sigma
– Using models or prototypes to create designs
and ensure they are effective in meeting goals
DMAIC VS. DMADV
Six Sigma Personnel
• Champion
– Manager/Director/Executive who
makes sure the resources are in
place for a Six Sigma project
• Master Black Belts
– Quality experts in an organization
– Responsible for strategic implementation
– Teach/Mentor other Black and Green Belts
• Black Belts
– Six Sigma team leaders responsible for implementing process
improvement projects within the business
• Green Belts
– Employee of an organization that has some training in Six Sigma
and may lead a Six Sigma project, but only as part of their job
CMM: What is it?
• “A Capability Maturity Model (CMM) is a
reference model of mature practices in a
specified discipline, used to improve and
appraise a group’s capability to perform
that discipline” -SEI
• CMM contains a list of key process areas
that includes software engineering, project
management, and process improvement
divided into different levels
CMMI
• CMM sunset
– No more SEI CMM training since 2003
• Some vendors may still offer it
– CMM Appraisals will end this year
• SEI states other vendors may offer this but questions value
• Version 1.1 CMMI released in December 2001
– Adds requirements development, validation
– Identifies strategy for verification and validation or
products (testing)
– Adds relevant stakeholders and level of involvement
(Identify key people)
– Less focus on “documented procedures” (Only when
critical to a process)
CMMI Models
• Capability Maturity Model Integration brings
together four disciplines
– Systems Engineering (SE)
– Software Engineering (SW)
– Integrated Product and Process Development
(IPPD) (Level 3 add on)
– Supplier Sourcing (SS) or Acquisition (Level 3
add on)
CMMI Maturity Levels
Maturity Levels of
Organizations 9/28/04
SW-CMMI Strengths
• Engineering focused which assists in
integration and interaction among
engineering departments
• Recognized Internationally
• DOD contracts require certain levels for
bids
• Utilizes best practices in the industry
Six Sigma Strengths
• Generic Process Improvement Strategy
can be applied to any process
• Trained personnel working on process
improvements for organization on
continuous basis
SW-CMMI Weaknesses
• Statistical methods often not used at levels 4 and
5 when they could ease data analysis
– Six Sigma embraces statistical methods
– Improvement in CMMI
• CMMI goals are not always aligned with business
and customer goals
– Generally Software Specific, but improving with CMMI
• CMMI assessors are certified, but implementers
are not so they may not understand underlying
concepts (statistical process control, etc)
– Six Sigma has competence certifications for relevant
subject matter
Six Sigma Weaknesses
• Derived from manufacturing which has a real tangible
process that is easy to measure and apparent, as
opposed to software engineering
• No formal connection to ISO 9000
– ISO 9000 and CMM have milestones and certifications
• Six Sigma relies on training personnel (black belts) that
look for areas that need improvement which may not be
the best way to go about this
– Costs also involved in training for Six Sigma
• Focuses on Internal process experience and doesn’t
consider external technology or best practices
CMMI Case Studies
• Lockheed Martin M&DS
– SW CMM ML 2 (1993)
– SW CMM ML 3 (1996)
– CMMI SW ML5 (2002)
– Increased software productivity by 30%
– Decreased unit software cost by 20%
– Decreased defect find and fix costs by 15%
CMMI Case Studies
• Bosch Gasoline Systems
– CMM Level 4
– Predictability – internal on-time delivery
improved by 15%
– Less Rework – first pass yield improved by
10%
– Product Quality – reduction in error cases in
company by one order of magnitude
Six Sigma Case Study
• >$3 billion energy company
• Customer dissatisfaction with software
deliverables
• Customer satisfaction improved by 2 points
on their “report card”
• Product quality was improved by 30%
• Reduction of rework by 50% was achieved
• Project savings of $150 K was achieved
So which one is better?
•
•
•
•
“The is no silver bullet” – Fred Brooks
Neither one is necessarily “better”
Both have weaknesses
Both have multiple case studies and large
corporations that have realized real savings
and process improvements utilizing their
techniques
• Utilizing both is possible
CMM and Six Sigma Similarities
• Both emphasize reducing defects as their
main process improvement goal
• Both focus on quantitative decision making
that relies on measurement
– Six Sigma and CMM level 4 and 5
• Both utilize similar basic statistical process
control techniques
– As mentioned, Six Sigma has a better
foundation in this area
Using Six Sigma and CMM
• Six sigma provides a foundation to define,
measure, analyze, improve, and control the
process
– Utilize process mapping
• Can work in conjunction with CMM to map out a
process and identify gaps within software processes
– Failure Modes and Effects Analysis
• Detects how a process can fail and assigns severity
to it to prioritize actions
Using Six Sigma and CMM
• CMM level 4 and 5 and Six Sigma are often seen
as similar:
– Six Sigma training can help to deal with CMM’s
Quantitative Process Management and Defect
Prevention requirements
– CMM can clarify how to use Six Sigma techniques for
software process
– Important to note that organizations can achieve level
4 and 5 without using Six Sigma techniques
• Many utilize their own statistical methods and analysis
• Sometimes ad-hoc, as mentioned before CMM does not tell
you necessarily how to accomplish a level, just what need to
be accomplished
Benefits from
SW CMM and Six Sigma
• Study of utilizing them together:
– Tata Consultancy Service development center reduced cost of
process failures from 5 to 1 percent, thus reducing total quality
cost
– Improved cycle time and productivity
– Scheduled slippage was reduced from a range of +-20 percent to
+-2 percent saving the company roughly $700k over 3 years
– Amazingly, at the time of the article, clients had not reported
any defects in their software products!
Summary
• Six Sigma is a process methodology that focuses on
quantitatively measuring a process in order to control and
improve upon it
• Six Sigma strives for 3.4 Defects Per Million Opportunities
(DPMO) so that it is within six standard deviations of the
mean
• SW CMMI is a process improvement model for software
engineering that is divided into levels and contains a list of
key process areas
Summary
• Six Sigma and CMMI can be utilized together to
improve the software engineering process
• Six Sigma’s more advanced statistical methods
can be applied to assist in quantitatively
analyzing data within CMMI’s level 4 and 5
• Six Sigma can also complement any individual
process within CMMI by utilizing techniques such
as process mapping
Conclusion
• Lastly, the effects of implementing Six
Sigma and CMMI can highly improve the
quality of the product developed and
individual processes which can directly
impact a business's bottom line and
customer satisfaction
References
• David N. Card, “Sorting out Six Sigma and the CMM”, IEEE Software,
May June 2000, pp. 11-13
• Jack Horgan, “Six Sigma and CMM”, September 13 - 17, 2004,
http://www10.edacafe.com/nbc/articles/view_weekly.php?articleid=20
9200
• M. Murugappan and G. Keeni, “Blending CMM and Six Sigma to Meet
Business Goals”, IEEE Software, March/April 2003
• “IT Perspective: Balancing Six Sigma and the Capability Maturity
Model (CMM ®/CMMI SM )”,
http://www.gartner.com/4_decision_tools/measurement/measure_it_a
rticles/2002_10/six_sig.jsp
• SEI Website, http://www.sei.cmm.edu/
• Six Sigma, http://www.isixsigma.com/
• Thomas Pyzdek, "DMAIC and DMADV", Pyzdek Consulting, Inc. Six
Sigma Handbook, http://www.pyzdek.com/DMAICDMADV.htm
Questions
Or
Comments?