Domain-Oriented Modeling, Estimation and
Improvement for Aerospace
Jairus Hihn
Jet Propulsion Laboratory, California Institute of Technology
November 2, 2011
26th International Forum on COCOMO
and
Systems/Software Cost Modeling
Background
JPL is a FFRDC for NASA operated by Caltech
JPL is a highly decentralized organization that allows great
autonomy to its project managers at all levels
JPL’s Engineering and Science Directorate (3000 people)
has twice been certified at CMMI ML 3 (2007, 2010)
Approximately 1000 people ($253M/year budget) in the software
community
This raises the question
How do we do it
How do we measure it?
We turn process sins into virtues
2
Data Sources
Data Sources
• Metrics Collection at Key Milestones
• Software Inventory (2006, 2007, 2009 and 2011)
• Process performance measures (Tailoring Record, Work Product
Checklist)
• Defect tracking systems (PRS, ISA, AAMS, local databases)
• SQI Surveys and Contact Log
• Product and Process Quality Assurance Activities
• Customer Feedback
Communicate results as widely as possible
• State of Software Report
• Noontime seminars
• Section manager meetings
3
Measuring Process Performance
Overall performance is used
to establish baselines
Identifying the least
performed activities
provides a short list from
which stakeholders can be
engaged to discuss process
improvement opportunities.
Metrics use is one of the
least performed activities
4
JPL Cost and Planning Models
Collect data at major milestones for mission critical and
mission support software
Labor intensive requiring numerous interviews
MSL flight software required 30-35 interviews
Enables us to
Calibrate cost models
Develop effort by role
use to do schedule models but they no longer work
Effort growth from PDR
Cost to repair defects
Size growth
Enables cost research
Data mining models developed with Tim Menzies (WVU)
5
Baselines and Trends
How are we doing?
Trends Suggest Improvements in process discipline
• Projects following a disciplined process exhibit below average budget
growth and higher productivity rates
•
Project Flight Software
6
NASA Software Cost improvement Task
Year 1: Establish Process and Product Baseline
Product: Report that documents current state of practice at
NASA with set of improvement recommendations and
recommended metrics.
Year 2:
Core team works to facilitate the implementation of high priority
recommendations across the Agency through a tailorable Agency-wide
process. Institute a cost metrics repository to gather information on
cost estimate accuracy and cost estimation processes.
Year 3:
Initiate a series of annual reports on NASA software cost estimation
accuracy and cost estimation practices.
• Continue to monitor and make recommendations
7
0
