CMM vs. ISO
David S. Craft CIRM, PMP
11 April 2007
Agenda
Who Am I
Software Systems Development
ISO
CMM
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Who Am I
Managing Consultant, Engineering and Manufacturing Services
Inventory Control Manager
Shift Supervisor
Internal ISO Auditor
Team Leader
Industrial Engineer
Consultant
Materials Manager
VISTA Volunteer
Manager Production Planning & Control
Chief Industrial Engineer
CMM vs. ISO, Sarbanes Oxley
Project Manager
11 April 2007
Process, people and technology are the major determinants of project cost,
quality and schedule.
CMM vs. ISO, Sarbanes Oxley
11 April 2007
Process
To Develop Software and Systems You Need A Process
So what is a process:
1.
A systematic series of actions directed to some end
2.
A continuous action, operation or series of changes taking place in a definite
manner
3.
A series of actions, changes or functions bringing about a result
4.
A series of operations performed in the making or treatment of a product
5.
Process or processing typically describes the action of taking something
through an established and usually routine set of procedures or steps to
convert it from one form to another (such as processing paperwork to grant a
loan, processing milk into cheese, converting computer data from one form to
11 April 2007
another, etc.)
CMM vs. ISO, Sarbanes Oxley
Process
Type of processes
• Anything goes
• Defined
• Structured
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Common Misconceptions
I don’t need defined processes I have:
–Really good people
–Advanced Technology
–An experienced manager
Defined Processes:
–Interfere with creativity
–Equals bureaucracy + regimentation
–Isn’t needed when building prototypes
–Is only useful on large projects
–Hinders agility in fast moving projects
–Costs too much
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Why We Need Structured Processes
Estimating (History)
•
Scope
•
Cost
•
Time
•
Tools
Deliver the Product to Estimate (Visibility)
•
Time
•
Cost
•
Quality
Handling/Controlling Changes
CMM vs. ISO, Sarbanes Oxley
•
Planned
•
Unplanned
•
Scope Creep
11 April 2007
Why We Need Standard Processes
Recent data suggested only about 35 percent IT projects are likely to be
completed on time and on budget, with all their originally specified
features and functions. Many projects, perhaps 20 percent, will be
abandoned, often after multimillion-dollar investments and the biggest
projects will fail most often.
One well-documented $170 million software failure was blamed on a lack
of defined requirements in the original contract; a lack of software
engineering, program, and contract management skills; and underestimates
of the complexity of interfacing the new system with legacy systems,
addressing security needs, and establishing an enterprise architecture.
From SEI Web
CMM vs. ISO, Sarbanes Oxley
11 April 2007
How to Achieve Quality Processes
ISO
CMM
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Meet The International Organization for
Standardization (ISO)
• A worldwide federation of national standards bodies from some 162
countries
• Representing approximately 95% of worldwide production.
• The world's largest developer and publisher of International Standards.
• A non-governmental organization established in 1947
• Promotes the development of standardization and related activities with a
view to facilitating international exchange of goods and services and
development of cooperation in the spheres of intellectual, scientific,
technological and economic activity
• Many of its member institutes are part of the governmental structure of
their countries, or are mandated by their government. On the other hand,
other members have their roots uniquely in the private sector, having been
set up by national partnerships of industry associations. Therefore, ISO
enables a consensus to be reached on solutions that meet both the
requirements of business and the broader needs of society.
11 April 2007
CMM vs. ISO, Sarbanes Oxley
What are standards?
Standards are documented agreements containing technical specifications
or other precise criteria to be used consistently as rules, guidelines, or
definitions of characteristics, to ensure that materials, products, processes
and services are fit for their purpose.
For example, the format of the credit cards, phone cards, and "smart" cards
that have become commonplace is derived from an ISO International
Standard. Adhering to the standard, which defines such features as an
optimal thickness (0,76 mm), means that the cards can be used worldwide.
International Standards thus contribute to making life simpler, and to
increasing the reliability and effectiveness of the goods and services we
use.
Last modified 2002-07-17
11 April 2007
CMM vs. ISO, Sarbanes Oxley
What ISO Standards Do
•
Make the development, manufacturing and supply of products and
services more efficient, safer and cleaner
•
Facilitate trade between countries and make it fairer
•
Provide governments with a technical base for health, safety and
environmental legislation, and conformity assessment
•
Share technological advances and good management practice
•
Disseminate innovation
•
Safeguard consumers, and users in general, of products and services
•
Make life simpler by providing solutions to common problems
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Where are the Standards (12/2010)
Sector
18,536 Standards
Generalities, Infrastructure and Sciences
Health, Safety and Environment
762,653 Pages
Engineering Technologies
Electronics, Information Technology and
Telecommunications
Transport and Distribution of Goods
Agriculture and Food Technology
Materials Technology
Construction
Special Technologies
11 April 2007
CMM vs. ISO, Sarbanes Oxley
ISO 9000:2008 Quality Management Systems
The ISO 9000:2008 standard provides a tried and tested framework
for taking a systematic approach to managing the organizations
processes so that they consistently turn out product that satisfies
customers expectations.
ISO 9000:2008 lays down what requirements your quality system
must meet, but does not dictate how they should be met in any
particular organization.
The ISO 9000:2008 standard has been implemented by over
1,000,000 organizations in 176 countries
11 April 2007
CMM vs. ISO, Sarbanes Oxley
ISO 9000:2008 Key Principles
• Customer Focus
• Leadership
• Involvement of People
• Process Approach
• System Approach to Management
• Continual Improvement
• Factual Approach to Decision Making
• Mutually Beneficial Supplier Relationships
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Quality System Documentation
Level 1
Quality
Manual
Defines
Approach and
Responsibility
Level 2
Procedures
Defines
Who, What, When
Level 3
Work/Job
Instructions
Answers
How
Level 4
Records/Documentation
CMM vs. ISO, Sarbanes Oxley
11 April 2007
Results: shows that
the system is
operating
ISO 9001:2000 Structure
4.
5.
6.
Quality Management System
4.1 General requirements
4.2 Document requirements
Management
Responsibility
5.1 Management
commitment
5.2 Customer focus
5.3 Quality policy
5.4 Planning
5.5 Responsibility, authority,
communication
5.6 Management review
Resource Management
6.1 Provision of resources
6.2 Human resources
6.3 Infrastructure
6.4 Work environment
CMM vs. ISO, Sarbanes Oxley
7.
Product realization
7.1 Planning of product realization
7.2 Customer-related processes
7.3 Design and development
7.4 Purchasing
7.5 Production and service provision
7.6 Control of monitoring and
measuring devices
8.
Measurement, Analysis &
Improvement
8.1 General
8.2 Monitoring and measurement
8.3 Control of nonconforming product
8.4 Analysis of data
8.5
Improvement
11 April
2007
Evaluation
• ISO is a certification model.
• Typically, an internal quality system assessment (audit) is
performed, repairs made and the organization may then submit to a
formal system audit lasting for several days performed by one of
the ISO certification Bodies.
• The certificate usually is valid for three years and also requires that
a system of Quality Management be in place, including
performance of regular internal audits and intermediate external
audits.
11 April 2007
CMM vs. ISO, Sarbanes Oxley
ISO’s Impact
In the global economy
ISO 9001:2000 and ISO 14001:2004 have become thoroughly integrated with
the world economy.
ISO 9001:2000 is now firmly established as the globally accepted standard for
providing assurance about the quality of goods and services in suppliercustomer relations.
The positive roles played in globalization by ISO’s standards for quality and
environmental management systems include the following:
•
a unifying base for global businesses and supply chains – such as the
automotive and oil and gas sectors
•
a technical support for regulation – as, for example, in the medical
devices sector
•
a tool for major new economic players to increase their participation in
global supply chains, in export trade and in business process
outsourcing;
•
a tool for regional integration – as shown by their adoption by new or
potential members of the European Union
In the rise of services in the global economy – nearly 33 % of ISO 9001:2000
certificates in 2005 went to organizations in the service sectors.
11 April 2007
CMM vs. ISO, Sarbanes Oxley
CMM
11 April 2007
CMM vs. ISO, Sarbanes Oxley
CMM History
• Active development of the model by the US Department of Defense Software
Engineering Institute (SEI) began in 1986 when Watts Humphrey joined the Software
Engineering Institute located at Carnegie Mellon University after retiring from IBM.
• At the request of the U.S. Air Force he began formalizing his Process Maturity
Framework to aid the U.S. Department of Defense in evaluating the capability of
software contractors as part of awarding contracts.
• The result this study was a model for the military to use as an objective evaluation of
software subcontractors' process capability maturity.
• Humphrey based this framework on the earlier Quality Management Maturity Grid
developed by Philip B. Crosby in his book "Quality is Free".
• Humphrey's approach differed because of his unique insight that organizations mature
their processes in stages based on solving process problems in a specific order.
Humphrey based his approach on the staged evolution of a system of software
development practices within an organization, rather than measuring the maturity of
each separate development process independently.
• The CMM has thus been used by different organizations
as a general and powerful tool
11 April 2007
for understanding and then improving general business process performance.
CMM vs. ISO, Sarbanes Oxley
Meet CMMI
CMMI® (Capability Maturity Model® Integration) models are
collections of best practices that help organizations to improve their
processes. These models are developed by product teams with
members from industry, government, and the Software Engineering
Institute (SEI). These models provides a comprehensive integrated set
of guidelines for developing products and services.
The CMMI-DEV model provides guidance for applying CMMI best
practices in a development organization. Best practices in the model
focus on activities for developing quality products and services to meet
the needs of customers and end users.
Other CMMI models:
•
•
•
Acquisition
Services
People
CMM vs. ISO, Sarbanes Oxley
11 April 2007
Scope of CMMI
The SEI’s body of work in technical and management practices is focused on
developing software right the first time, which results not only in higher
quality, but also predicable and improved schedule and cost.
CMMI helps you to meet your organizations business objectives and improve
performance.
CMMI is a process improvement approach that provide organizations with the
essential elements of effective processes, which will improve their
performance
11 April 2007
CMM vs. ISO, Sarbanes Oxley
CMMI Organization
CMMI is organized as a process framework that cluster related
practices into process areas that, when performed collectively, satisfy
a set of goals. It requires that you define specific practices to meet
specific goals but does not define how they are to be implemented.
The CMMI provides two representations – staged and continuous,
each containing 22 Process Areas (PA). The staged view provides
five maturity levels: Initial, Managed, Defined, Quantitatively
Managed, and Optimizing. The PAs at each maturity level build on
the previous level. Alternatively, continuous representation is used to
focus on a process capability in a desired functional area (project
management, process management, engineering and support) rather
that maturity levels
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Process Areas
Requirements Management
Organizational Process Definition
Project Planning
Organizational Training
Project Monitoring & Control
Integrated Project Management
Supplier Agreement Management
Risk Management
Measurement & Analysis
Integrated Teaming
Process & Product Quality
Assurance
Integrated Supplier Management
Configuration Management
Decision Analysis & Resolution
Requirements Development
Organizational Environment for
Integration
Technical Solution
Organizational Process Performance
Product Integration
Quantitative Project Management
Verification
Organizational Innovation & Deployment
Validation
Causal Analysis
11 April 2007& Resolution
Organizational Process Focus
CMM vs. ISO, Sarbanes Oxley
11 April 2007
EIA – Electronic Industries Alliance Interim Standard
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Capability and Maturity Levels
Level
5 Focus on continuous
process emprovement
4 Process measured and
controled
3 Process characterized for
the organization and is
proactive
2 Process characterized for
projects and is often
reactive
1
0
Capability Levels
Optimizing
Maturity Levels
Optimizing
Qualitatively
Managed
Defined
Quantitatively
Managed
Managed
Performed
Incomplete
Initial
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Defined
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Evaluation
• This is not a certification model, but ratings may be announced and
published.
• The SEI publishes ratings provided the company gives it permission.
• Formal appraisals are typically 5 – 10 days and led by SEI-authorized
internal or external lead appraisers, using trained teams and a formal
methods. The method is named SCAMPI (Standard CMMI Appraisal
Method for Process Improvement).
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Examples of CMMI Impact: ROI
5:1 ROI for quality activities (Accenture)
13:1 ROI calculated as defects avoided per hour spent in training and
defect prevention (Northrop Grumman Defense Enterprise Systems)
Avoided $3.72 M in costs due to better cost performance (Raytheon
North Texas Software Engineering) as the organization improved
from SW-CMM level 4 to CMMI level 5
2:1 ROI over 3 years (Siemens Information Systems Ltd, India)
2.5:1 ROI over 12st year, with benefits amortized over less than 6
months (reported under non disclosure)
(reported by the American Society for Quality)
11 April 2007
CMM vs. ISO, Sarbanes Oxley
ISO – CMM Differences
ISO9001:2000
CMMI-DEV
International standard, applies to all
types of organizations, supports both
product and service oriented
organizations
Written specifically for software
development companies
A brief document – about 25 pages
long, identifying the minimal
requirements for a quality system
A detailed document – over 500 pages
long
Emphasizes on a management of
continuous improvement process,
based on the PDCA (Plan-Do-CheckAct) model
Emphasizes on achieving “maturity”
and improving its process continuously
One level of standard. The standard is
based on recommendation
Defines 5 maturity levels of the
organization, covering 25 process
areas (PAs)
11 April 2007
Netta Dotan, Quality Assurance & project management, Ronkal Office Technologies
CMM vs. ISO, Sarbanes Oxley
ISO – CMM Differences – My View
ISO 9000
SW-CMMI
Outwardly focused
Inwardly focused
Minimum requirements with
implied continuous
improvements
Explicit continuous quality
improvement
Registration Document
No documentation
Certification audit for a 50
employee organization will be
executed by 1 -12 auditors
during one day
Certification audit for a 50
employee organization will be
executed by 4 auditors during
4-5 days
11 April 2007
Netta Dotan, Quality Assurance & project management, Ronkal Office Technologies
CMM vs. ISO, Sarbanes Oxley
ISO – CMM Similarities
Both require the organization be explicit about what their processes and
quality systems are
Say what you do; do what you say
The organization records and tracks data for objective analysis
Require strong management support to succeed
Provide a structured and measured approach to quality improvement
Require an outside audit for “certification”
Both are refined/improved over time
11 April 2007
CMM vs. ISO, Sarbanes Oxley
So What
Why Should You Care
11 April 2007
CMM vs. ISO, Sarbanes Oxley