Domain-Oriented Modeling, Estimation and
Improvement for Aerospace
Jairus Hihn
Jet Propulsion Laboratory, California Institute of Technology
November 2, 2011
26th International Forum on COCOMO
and
Systems/Software Cost Modeling
Background
 JPL is a FFRDC for NASA operated by Caltech
 JPL is a highly decentralized organization that allows great
autonomy to its project managers at all levels
 JPL’s Engineering and Science Directorate (3000 people)
has twice been certified at CMMI ML 3 (2007, 2010)

Approximately 1000 people ($253M/year budget) in the software
community
 This raises the question

How do we do it
 How do we measure it?
 We turn process sins into virtues
2
Data Sources
 Data Sources
• Metrics Collection at Key Milestones
• Software Inventory (2006, 2007, 2009 and 2011)
• Process performance measures (Tailoring Record, Work Product
Checklist)
• Defect tracking systems (PRS, ISA, AAMS, local databases)
• SQI Surveys and Contact Log
• Product and Process Quality Assurance Activities
• Customer Feedback
 Communicate results as widely as possible
• State of Software Report
• Noontime seminars
• Section manager meetings
3
Measuring Process Performance
 Overall performance is used
to establish baselines
 Identifying the least
performed activities
provides a short list from
which stakeholders can be
engaged to discuss process
improvement opportunities.
 Metrics use is one of the
least performed activities
4
JPL Cost and Planning Models
 Collect data at major milestones for mission critical and
mission support software
 Labor intensive requiring numerous interviews
 MSL flight software required 30-35 interviews
 Enables us to
 Calibrate cost models
 Develop effort by role
 use to do schedule models but they no longer work
 Effort growth from PDR
 Cost to repair defects
 Size growth
 Enables cost research
 Data mining models developed with Tim Menzies (WVU)
5
Baselines and Trends
 How are we doing?
Trends Suggest Improvements in process discipline
• Projects following a disciplined process exhibit below average budget
growth and higher productivity rates
•
Project Flight Software
6
NASA Software Cost improvement Task
Year 1: Establish Process and Product Baseline
 Product: Report that documents current state of practice at
NASA with set of improvement recommendations and
recommended metrics.
Year 2:
 Core team works to facilitate the implementation of high priority
recommendations across the Agency through a tailorable Agency-wide
process. Institute a cost metrics repository to gather information on
cost estimate accuracy and cost estimation processes.
Year 3:
 Initiate a series of annual reports on NASA software cost estimation
accuracy and cost estimation practices.
• Continue to monitor and make recommendations
7