1.264 Lecture 5
System Process: CMMI, ISO
Next class: Read UML chapters 1 (skim), 2 (skim), 4, 8, 9
Make sure you have Visual Paradigm installed. We start using it Monday in class.
Homework 1 solutions posted Monday
1
Discussion: Case studies
• Chemical inventory case study
– List the errors made
– Outline resource estimation steps that should have been
used
– Outline process steps that should have been followed
• RFID case study
– What lifecycle model would you use?
– How would you give a time estimate?
– List major project risks and how to mitigate them
2
Capability Maturity Model Integration
• Developed at Software Engineering Institute (SEI),
Carnegie-Mellon University (www.sei.cmu.edu)
• Method for process assessment and improvement
• Five level model of maturity and capability
–
–
–
–
–
1: Initial/Performed
2: Managed
3: Defined
4: Quantitatively Managed
5: Optimizing
• Predictability, effectiveness and control of software
processes improve as organization moves up these levels
• One old reading on CMM/CMMI is on course Web site
• Current material is at http://www.sei.cmu.edu/cmmi/
3
CMMI Motivation
• CMMI covers three interlocking areas:
– Development
– Services
– Acquisition
• CMMI Development:
– 20 years of unfulfilled promises about productivity and
quality gains from new software technology
• Databases, object oriented programming, Internet, …
– Organizations realized fundamental problem is the
inability to manage the software process
– CMMI provides guidance on how to evolve toward a
culture of software engineering and rational
management
4
CMMI Maturity Level 1: Initial
• Ad hoc, occasionally chaotic
• Few processes defined
• Success depends on individual effort and
heroics
– This is what you’re taught to do in school…
5
CMMI Maturity Level 2: Managed
• Basic project management processes established to track
cost, schedule, functionality
• Discipline in place to repeat earlier successes on projects
with similar applications
• Key processes focus on basic project management
controls
– Requirements management: initial spec, change control
– Software project planning: resource estimation based on past
performance
– Software project tracking and oversight
– Software subcontract management
– Software quality assurance: code reviews, test plans, tracking
– Software configuration management
• At level 2, you can measure what’s going on, and that helps
understand future projects
– A project or group can assess at level 2
6
CMMI Maturity Level 3: Defined
• Software process for management and development is
documented, standardized and integrated into an overall
process for the organization
• All projects use approved, tailored version of standard
process
• Key process areas focus on institutionalizing effective process
–
–
–
–
–
–
Organization process: spiral model, tracking, control
Training program: managers, analysts, developers
Integrated software management: lines of code, defects, time,…
Software product engineering: alternative designs, scopes…
Intergroup coordination: Web, database, analyst, coder, customer
Peer reviews: initiate at requirements, design, QA. Continue code
reviews
• At level 3, you begin to have some control; you can actually
predict times/costs for new projects and make some choices
on how to approach them (fastest, most efficient, nominal…)
– The entire organization must assess at level 3; you can’t do it for an
individual project or group
7
CMMI Level 4: Quantitatively Managed
• Detailed measures of software process and product quality
are collected, in addition to product measures
• Process and products are quantitatively understood and
controlled
• Key processes focus on quantitative understanding of
process
– Quantitative process management: measure and manage time
at all stages: requirements, design, code, QA
– Software quality management: reviews at all stages, tests and
traceability at all stages
• At level 4 you have real control: you can measure and
manage all aspects of the project
8
CMMI Level 5: Optimizing
• Continuous process improvement through quantitative
feedback
• Piloting innovative technology and ideas
• Key process areas focus on continual process
improvement
– Defect prevention (‘extreme programming’, refactoring,
reviews)
– Technology change management (new tools, training, pilots,
tests, knowledge sharing)
– Process change management
• At level 5, you not only have control but are efficient
9
PROCESS CAPABILITY AS INDICATED BY MATURITY LEVEL
Target N
1
Schedule and cost targets are typically
overrun by Level 1 organizations
Time/$/....
2
Target N+a
Plans based on past performance are
more realistic in Level 2 organizations
Probability
Time/$/....
3
Target N-x
With well-defined processes,
performance improves in
Level 3 organizations
Time/$/....
4
Target N-y
Based on quantitative understanding
of process and product, performance
continues to improve in Level 4
organizations
Time/$/....
5
Target N-z
Performance continuously
improves in Level 5 organizations
Image by MIT OpenCourseWare.
10
Process Maturity Profile by All
Reporting Organizations
3600
3400
3200
Number of Organizations
3000
2800
2600
2400
2200
2000
1800
1600
1400
1200
1000
800
600
400
200
0
Not Given
Initial
Managed
Defined
Quantitatively
Managed
Optimizing
Based on March 2011 appraisal of 5346 organizations
Image by MIT OpenCourseWare, adapted from Carnegie Mellon University, Software Engineering Institute.
11
Maturity Profile by Organization Size
Based on the total number of employees within the area of the organization that was appraised
100%
Managed
Quantitatively Managed
Initial
Defined
Optimizing
38.8%
51.8%
51.6%
55.4%
47.8%
25 or fewer
26-50
51-75
76-100
27.4%
23.9%
201-300
301-500
1001-2000
1.2%
0.8%
7.1%
12.1%
501-1000
24%
5.6%
5.2%
0.8%
8.0%
14.3%
16.0%
4.8%
5.3%
10.4%
5.4%
0.3%
9.4%
9.0%
101-200
1.6%
3.6%
3.3%
5.9%
7.9%
0.9%
6.4%
2.4%
3.1%
4.9%
0.7%
1.4%
1.1%
4.6%
0.5%
0%
0.4%
0.7%
10%
5.0%
0.8%
20%
17.0%
20.7%
20.9%
21.6%
30%
25.8%
27.3%
40%
1.2%
50%
47.3%
60%
52.9%
57.1%
62.8%
70%
63.1%
65.1%
80%
45.5%
% of organizations by size
90%
Not Given
2001 +
Based on 5301 organizations reporting size data as of March 2011
Image by MIT OpenCourseWare, adapted from Carnegie Mellon University, Software Engineering Institute.
12
40
28.5
28
30
75th Percentile
Median
20
Number of months to move to
next maturity level
Time to Move Up
25th Percentile
20
10
Time period of
initial appraisal
Level
Orgs
5
2006 to March 2011
1-2
2-3
3-4
3-5
7
227
50
46
Image by MIT OpenCourseWare, adapted from Carnegie Mellon University, Software Engineering Institute.
13
Exercise
• Briefly assess the pros and cons of using
spreadsheets (Excel) as the major analysis
method in running a business or managing a
project
14
Solution
• Pros:
– Spreadsheets are very quick to develop
– Most staff know how to write and use them
• Cons:
– Incorporate decision making that is not known to central
organization
– Hard to pass on spreadsheet to successor or other user
– Spreadsheets rarely documented or tested well
– Spreadsheets are rarely completely correct
– No business rules, like a database
• Heavy use of spreadsheets indicates a low
process maturity organization
15
ISO 9001:2008
• International Standards Organization (ISO)
– National standards bodies from 100+ countries
• ISO 9001:2008
– First major modification of ISO standard in 20 years occurred
in 2000, and was updated in 2008
– No longer primarily a manufacturing standard, with military
origins, implemented as books of procedures
– New standard addresses all products and services in all
industry segments
• The new standard requires 8 principles be implemented:
–
–
–
–
–
–
–
–
–
Customer focus
Leadership
Involvement of people
Process approach
System approach to management
Continual improvement
Factual approach to decision-making
Mutually beneficial supplier relationships
(The old standard didn’t really require anything)
16
ISO 9001:2008 cont
• ISO 9001 certification important to do business in
Europe, and becoming so in Pacific Rim and
eventually Americas
• General approach
–
–
–
–
Create a quality steering team in management
Create project plan for ISO 9001 certification
Perform gap analysis (good process vs current process)
Write needed system development, quality procedures
• Often takes 6-12 months
– Train employees in new procedures
– Use new materials, retain records
– 3 certification options: self, clients or independent body
17
Exercise
• Steelmaker: current software
–
–
–
–
–
Steelmaking process control software: custom
Substantial raw material and outbound logistics: custom
Enterprise resource planning software: custom and SAP
Customer relationship mgt software: software-as-service
Demand forecasting: best-of-breed from small vendor
• Proposed project:
– Integrate demand forecasting and logistics
• Current organization large, not integrated (silos)
– Steelmaking process is ISO 9000, nothing else is ISO/CMMI
• What process suggestions do you make?
–
–
–
–
Which basis: CMMI or ISO?
At what CMMI level is the organization likely to be now?
List a set of initial steps to take
How long will it take to see substantive process change?
18
Solution (one of many)
• What process suggestions do you make?
– Which basis: CMMI or ISO?
• Could justify either
• ISO 9001: involves major parts of enterprise, not heavy on
system development.
• CMMI: choose to focus on systems aspect
– At what CMMI level is the organization likely to be now?
• Level 2, since there is ISO 9001 certification
• Perhaps level 1, since most parts are not certified
– List a set of initial steps to take
• ISO 8 principles, or CMMI level 2 steps, to ensure process
• Then do requirements, design, etc. in a spiral
– How long will it take to see substantive process change?
• ISO often 6-12 months
• CMMI time to move up from level 1 to level 2 is 5-10 months
and to level 3 is 20-24 months
19
Intelligent Transportation Systems (ITS)
project management process
Public domain (US Federal Highway Administration and Caltrans).
Source: ITS Systems Engineering Guidebook for Intelligent Transportation Systems, version 3.0
20
ITS project management: Vee
Public domain (US Federal Highway Administration and Caltrans).
Source: ITS Systems Engineering Guidebook for Intelligent Transportation Systems, version 3.0
21
Comparison of CMMI and ISO
• CMMI focus:
–
–
–
–
Software engineering practices
Program/project management practices
Increasing maturity levels
Appraisals when going to next maturity level
• ISO 9001:2008 focus:
–
–
–
–
Customer satisfaction
Process discipline across entire organization
Continual improvement
Principled management
• ITS: http://www.fhwa.dot.gov/cadiv/segb
– Transportation-specific, includes construction process
22
MIT OpenCourseWare
http://ocw.mit.edu
1.264J / ESD.264J Database, Internet, and Systems Integration Technologies
Fall
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.