CMMI vs. ISO
David S. Craft CIRM, PMP
11 April 2007
Agenda
Process
ISO
CMMI
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
The Process Management Premise
The quality of a system is influenced by the quality of the process
used to acquire, develop, and maintain it, the analysis and forethought
that goes into an architecture that supports business goals and
requirements, and the training provided to teams involved in the
project. Using proven methods for process and product quality,
software success is predictable and achievable, and failure is
avoidable.
Once coding starts, teams trained in mature software engineering
processes can remove defects early, when defect removal is 10 to 100
times less costly than it is during test. This dramatically reduces test
costs and only marginally increases costs upstream
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Process
To Develop Software and Systems You Need A Process
So what is a process:
1.
2.
3.
A systematic series of actions directed to some end
A continuous action, operation or series of changes taking place
in a definite manner
A series of actions, changes or functions bringing about a result
4.
A series of operations performed in the making or treatment of a
product
5.
Process or processing typically describes the action of taking
something through an established and usually routine set of
procedures or steps to convert it from one form to another (such
as processing paperwork to grant a loan, processing milk into
cheese, converting computer data from one form to another, etc.
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Common Misconceptions
I don’t need defined processes I have:
–Really good people
–Advanced Technology
–An experienced manager
Defined Processes:
–Interfere with creativity
–Equals bureaucracy + regimentation
–Isn’t needed when building prototypes
–Is only useful on large projects
–Hinders agility in fast moving projects
–Costs too much
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Symptoms of Process Failure
Commitments consistently missed
• Late deliveries
• Last minutes crunches
• Spiraling costs
No management visibility into progress
•
You’re always being surprised
Quality Problems
• Too much rework
• Functions do not work correctly
• Customer complaints after delivery
Poor Moral
• People frustrated
• Is anyone in charge?
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Why We Need Structured Processes
Estimating (History)
•
•
•
•
Scope
Cost
Time
Tools
Deliver the Product to Estimate (Visibility)
•
•
•
Time
Cost
Quality
Handling/Controlling Changes
•
•
•
Planned
Unplanned
Scope Creep
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Why We Need Standard Processes
• Organizations and governments worldwide will spend about $1 trillion
this year on IT projects. Recent data suggested only about 35 percent of
those projects are likely to be completed on time and on budget with all
their originally specified features and functions. Many projects, perhaps
20 percent, will be abandoned, often after multimillion-dollar
investments—and the biggest projects will fail most often.
• One well-documented $170 million software failure was blamed on a
lack of defined requirements in the original contract; a lack of software
engineering, program, and contract management skills; and
underestimates of the complexity of interfacing the new system with
legacy systems, addressing security needs, and establishing an enterprise
architecture.
• Other software-development failures have brought down entire
companies, such as the $5 billion drug-distribution firm in Texas that
declared bankruptcy as a result of a poorly implemented resource
11 April 2007
planning system.
CMMI vs. ISO, Sarbanes Oxley
How to Achieve Quality Processes
ISO Standards
CMMI Models
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
ISO
International Standards Organization
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Meet The International Organization for
Standardization (ISO)
• ISO – a nongovernmental organization – is a network of the
national standards bodies of some 160 countries, one per country,
from all regions of the world, including developed, developing and
transitional economies.
• ISO is a global network that identifies what International Standards
are required by business, government and society, develops them in
partnership with the sectors that will put them to use, adopts them
by transparent procedures based on national input and delivers
them to be implemented worldwide.
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
What are standards?
Standards are documented agreements containing technical
specifications or other precise criteria to be used consistently as
rules, guidelines, or definitions of characteristics, to ensure that
materials, products, processes and services are fit for their purpose.
For example, the format of the credit cards, phone cards, and
"smart" cards that have become commonplace is derived from an
ISO International Standard. Adhering to the standard, which defines
such features as an optimal thickness (0,76 mm), means that the
cards can be used worldwide.
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
ISO 9000 - Quality management
• The ISO 9000 family addresses various aspects of quality management
and contains some of ISO’s best known standards. The standards
provide guidance and tools for companies and organizations who want
to ensure that their products and services consistently meet customer’s
requirements, and that quality is consistently improved.
• There are many standards in the ISO 9000 family, including:
•
ISO 9001:2008 - sets out the requirements of a quality management system
•
ISO 9000:2005 - covers the basic concepts and language
•
ISO 9004:2009 - focuses on how to make a quality management system more
efficient and effective
•
ISO 19011:2011 - sets out guidance on internal and external audits of quality
management systems
The ISO 9000:2008 standard has been implemented by over 1,000,000
organizations in 176 countries
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
ISO 9000:2008 Key Principles
• Customer Focus
• Leadership
• Involvement of People
• Process Approach
• System Approach to Management
• Continual Improvement
• Factual Approach to Decision Making
• Mutually Beneficial Supplier Relationships
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Quality System Documentation
Level 1
Quality
Manual
Defines
Approach and
Responsibility
Level 2
Procedures
Defines
Who, What, When
Level 3
Work/Job
Instructions
Answers
How
Level 4
Records/Documentation
CMMI vs. ISO, Sarbanes Oxley
11 April 2007
Results: shows that
the system is
operating
ISO 9001:2000 Structure
4.
5.
6.
Quality Management System
4.1 General requirements
4.2 Document requirements
Management
Responsibility
5.1 Management
commitment
5.2 Customer focus
5.3 Quality policy
5.4 Planning
5.5 Responsibility, authority,
communication
5.6 Management review
Resource Management
6.1 Provision of resources
6.2 Human resources
6.3 Infrastructure
6.4 Work environment
CMMI vs. ISO, Sarbanes Oxley
7.
Product realization
7.1 Planning of product realization
7.2 Customer-related processes
7.3 Design and development
7.4 Purchasing
7.5 Production and service provision
7.6 Control of monitoring and
measuring devices
8.
Measurement, Analysis &
Improvement
8.1 General
8.2 Monitoring and measurement
8.3 Control of nonconforming product
8.4 Analysis of data
8.5
Improvement
11 April
2007
Standard Examples
5.2 Customer Focus
“Top management shall ensure that customer requirements are determined
and are met with the aim of enhancing customer satisfaction.”
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Advantages of ISO 9000
• Increased marketability.
• Reduced operational expenses
• Better management control
• Increased customer satisfaction
• Improved internal communication
• Improved customer service
• Reduction of product-liability risks
• Attractiveness to investors
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Disadvantages of ISO Implementation
• Owners and managers do not have an adequate understanding
of the ISO 9000 certification process or of the quality standards
themselves
• Funding for establishing the quality system is inadequate
• Heavy emphasis on documentation
• Length of the process
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
ISO’s Impact In The Global Economy
ISO 9001:2000 is now firmly established as the globally accepted
standard for providing assurance about the quality of goods and services
in supplier-customer relations.
The positive roles played in globalization by ISO’s standards for quality
and environmental management systems include the following:
•
a unifying base for global businesses and supply chains – such as
the automotive and oil and gas sectors
•
a technical support for regulation – as, for example, in the medical
devices sector
•
a tool for major new economic players to increase their
participation in global supply chains, in export trade and in
business process outsourcing;
•
a tool for regional integration – as shown by their adoption by new
or potential members of the European Union
In the rise of services in the global economy – nearly 33 % of ISO
11 April 2007
9001:2000 certificates in 2005 went to organizations
in the service
sectors.
CMMI vs. ISO, Sarbanes Oxley
CMMI
Capability Maturity Model Integrated
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Software Engineering Institute (SEI)
• SEI is a federally funded research and development center sponsored by
the U.S. Department of Defense and operated by Carnegie Mellon
University in Pittsburgh, Pa.
• SEI helps advance software engineering principles and practices and
serves as a national resource in software engineering, computer security,
and process improvement.
• SEI works closely with defense and government organizations, industry,
and academia to continually improve software-intensive systems.
• Its core purpose is to help organizations improve their software
engineering capabilities and develop or acquire the right software, defect
free, within budget and on time, every time.
• SEI transitions its technologies to the global software engineering
community through its public courses, conferences, technical reports, and
Partner Network.
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Meet CMMI
CMMI® (Capability Maturity Model® Integration) models are
collections of best practices that help organizations to improve their
processes. These models provides a comprehensive integrated set of
guidelines for developing products and services. The SEI’s body of
work in technical and management practices is focused on developing
software right the first time, which results not only in higher quality,
but also predicable and improved schedule and cost
There are three CMMI models
• CMMI-DEV – Systems and Software Engineering
• CMMI -ACQ – Acquiring Products, Services or Outsourcing
• CMMI -SVC – For service organizations
CMMI helps you to meet your organizations business objectives and
improve performance.
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
What is a Maturity Model
A structured collection of elements describing characteristics of
effective processes
A maturity model provides:
• A place to start
• The benefits of companies prior experience
• A common language and shared vision
• A framework for prioritizing actions
• A way to define what improvement means for your organization
The model can be used to assess where your organization is against
other organizations
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
CMMI Organization
CMMI is organized as a process framework clustering related
practices into process areas that, when performed collectively, satisfy
a set of goals. It requires that you define specific practices to meet
specific goals but does not define how they are to be implemented.
The CMMI provides two representations – staged and continuous,
each containing 25 Process Areas (PA).
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Process Areas
Requirements Management
Organizational Process Definition
Project Planning
Organizational Training
Project Monitoring & Control
Integrated Project Management
Supplier Agreement Management
Risk Management
Measurement & Analysis
Integrated Teaming
Process & Product Quality Assurance
Integrated Supplier Management
Configuration Management
Decision Analysis & Resolution
Requirements Development
Organizational Environment for Integration
Technical Solution
Organizational Process Performance
Product Integration
Quantitative Project Management
Verification
Organizational Innovation & Deployment
Validation
Causal Analysis & Resolution
Organizational Process Focus
CMMI vs. ISO, Sarbanes Oxley
11 April 2007
CMMI Standard Example
SP 1.2 Specify Measures
Specify measures to address measurement objectives. Measurement objectives are refined into
precise, quantifiable measures. Measurement of project and organizational work can typically be
traced to one or more measurement information categories. These categories include the following:
• schedule and progress
• effort and cost
• size and stability
• quality.
Measures can be either base or derived. Data for base measures are obtained by direct measurement.
Data for derived measures come from other data, typically by combining two or more base measures.
Examples of commonly used base measures include the following:
• Estimates and actual measures of work product size (e.g., number of pages)
• Estimates and actual measures of effort and cost (e.g., number of person hours)
• Quality measures (e.g., number of defects by severity)
• Information security measures (e.g., number of system vulnerabilities identified)
• Customer satisfaction survey scores
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
CMMI Standard Example con’t
Examples of commonly used derived measures include the following:
• Earned value
• Schedule performance index
• Defect density
• Peer review coverage
• Test or verification coverage
• Reliability measures (e.g., mean time to failure)
• Quality measures (e.g., number of defects by severity/total number of defects)
• Information security measures (e.g., percentage of system vulnerabilities mitigated)
• Customer satisfaction trends
Derived measures typically are expressed as ratios, composite indices, or other aggregate
summary measures. They are often more quantitatively CMMI for Development, Version 1.3
Measurement and Analysis (MA) 180 reliable and meaningfully interpretable than the base
measures used to generate them.
There are direct relationships among information needs, measurement objectives, measurement
categories, base measures, and derived measures. This direct relationship is depicted using some
common examples in Table MA.1.
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Capability and Maturity Models
Continuous View
Staged View
Capability Levels
A well defined evolutionary plateau
describing the organization’s capability
relative to a particular process area
There are six capability levels
Maturity Levels
A well defined evolutionary plateau of
process improvement
There are five maturity levels
Each level is a layer in the foundation for Each level is a layer in the foundation for
continuous process improvement. Thus, continuous process improvement using a
capability levels are cumulative (i.e., a
proven sequence of improvements,
higher capability level includes the
beginning with basic management
attributes of the lower levels).
practices and progressing through a
predefined and proven path of successive
levels
Enables comparison across and among Provides a single rating that summarizes
organizations on a process-are-by
appraisal results and permits comparisons
process–area basis
across and among organizations
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Capability and Maturity Levels
Level
5
4
3
2
Continuous View Staged View
Focus on continuous
process improvement
Process measured and
controled
Process characterized for
the organization and is
proactive
Process characterized for
projects and is often
reactive
1
0
Capability Levels
Optimizing
Maturity Levels
Optimizing
Qualitatively Managed
Quantitatively
Defined
Defined
Managed
Managed
Performed
Incomplete
Initial
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Evaluation & Ratngs
• This is not a certification model, but ratings may be announced and
published.
• The SEI publishes ratings provided the company gives it
permission.
• Formal appraisals are typically 5 – 10 days and led by SEIauthorized internal or external lead appraisers, using trained teams
and a formal methods. The method is named SCAMPI (Standard
CMMI Appraisal Method for Process Improvement).
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
Benefits of CMMI
• Decreased Costs
• General Dynamics Advanced Information Suystems reduced maintenance staff
costs by 64% while doubling the size of the organization
• TATA consultancy Service saved $4.6 million across all development centers
• Improved On-Time Delivery
• Tufts Associated Health Plans achieved 100% on time delivery of major IT
projects in a full year
• GM improved the percent of milestones met from 50% to 85%
• Improved Quality
• Siemens Information Systems LTD. Reduced defect density an average of 71%
in three technical areas
• IBM Australia Application Management Services closed 95% of problems
within the customer-specified time frame
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
ISO – CMMI Differences
ISO9001:2008
CMMI-DEV
An audit standard
A process model
A certification tool that certifies businesses
whose processes conform to the laid down
standards
A set of related “best practices” derived
from industry leaders and relates to product
engineering and software development
Flexible and applicable to all manufacturing Rigid and only extends to businesses
industries
developing software intensive systems
Specific to conformance and remains
oblivious as to whether conformance is of
strategic value or not
Requires ingraining processes into business
needs so that processes become part of the
corporate culture and do not break down
under the pressure of deadlines
Provides generic guidelines for risk
management
Approaches risk management as an
organized and technical discipline
Customer satisfaction is an important part
of the requirements
Focuses on linkage of processes to business
goals, customer satisfaction is not a factor
in the ranking
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
ISO – CMMI Differences
ISO9001:2008
CMMI-DEV
Customer satisfaction is and important
part of ISO requirements
Focuses on linkage of processes to
business goals, customer satisfaction is
not a factor in ranking
Flexible, wider in scope and not
directly linked to business objectives
More focused, complex and aligned
with business objectives
Registration Document
No documentation
Certification audit for a 50 employee
Certification audit for a 50 employee
organization will be executed by 1 -12 organization will be executed by 4
auditors during one day
auditors during 4-5 days
11 April 2007
Netta Dotan, Quality Assurance & project management, Ronkal Office Technologies
CMMI vs. ISO, Sarbanes Oxley
Steps to Implement ISO (and CMMI)
• Decide to improve your internal processes
• Determine method for improvement
• Plan for ISO (CMMI) and gain commitment of people, particularly
upper management.
• Assign the responsibility of the implementation process to someone
(internal or external).
• Train all personal in ISO requirements
• Perform assessment of current processes and find the gaps
• Fill the gap by revising, adding or improving the current processes
and documentation to meet ISO requirements.
• Perform internal audit(s)
• External audit
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
ISO – CMMI Similarities
Both require the organization be explicit about what their processes
and quality systems are
Say what you do; do what you say
The organization records and tracks data for objective analysis
Require strong management support to succeed
Provide a structured and measured approach to quality improvement
Require an outside audit for “certification”
Both are refined/improved over time
11 April 2007
CMMI vs. ISO, Sarbanes Oxley
So What
Why Should You Care
11 April 2007
CMMI vs. ISO, Sarbanes Oxley