Harmonizing Systems and Software Estimation
23rd International Forum on COCOMO and Systems/Software Cost Modeling
and ICM Workshop
USC Campus, Los Angeles, CA
Oct. 27, 2008
Gan Wang, BAE Systems, Reston, VA gan.wang@baesystems.com
Garry J. Roedler, Lockheed Martin, Philadelphia, PA garry.j.roedler@lmco.com
Ricardo Valerdi, MIT, Cambridge, MA rvalerdi@mit.edu
Aaron Ankrum, BAE Systems, Reston, VA aaron.ankrum@baesystems.com
John E. Gaffney, Jr., Lockheed Martin, Rockville, MD j.gaffney@lmco.com
Jared Fortune, USC, Los Angeles, CA fortune@usc.edu
Don Reifer, Reifer Consultants, Neptune, CA dreifer@earthlink.net
Acknowledgement
• This presentation uses material from the following:
– PSM Workshop on Harmonizing COSYSMO and COCOMO Results
Briefing, July 2008
– USC COSYSMO Workshop, March 2008
– BAE Briefing for COSYSMO Workshop, October 2008
– Lockheed Martin Project Briefings
2
Problem & Motivation
• Estimating tools available today (e.g., COSYSMO, COCOMO, PRICE,
SEER) are functionally oriented
– We assemble a total engineering bid from functional estimates
• Understand the issues related to integrating systems and software
estimation for software-intensive projects
– What are the exact estimate scopes of COSYSMO and COCOMO II?
– How do we deal with potential gaps and overlaps?
3
Summary of Work So Far
• COSYSMO Workshop in conjunction with 2008 USC Annual Research
Review
– Scoped the problem
– Prioritized with other COSYSMO projects – assessed as high priority
– Identified that operational guidance may be as important as model
constructs
• COSYSMO Workshop at 2008 PSM Users Group Conference (Mystic,
CT)
– Assigned WBS elements to functions
– Assessed the coverage by COSYSMO and COCOMO
– Identified potential gaps and overlaps
• Workshops within engineering communities at BAE Systems
• Project in Lockheed Martin to address Integrated Cost Estimation for
Systems and Software Engineering
• Joint paper to INCOSE Symposium
4
Areas for Consideration in Analysis of Harmonization
• Overlap/Gaps of tasks
– Per WBS, work products, and combined activities
•
•
•
•
•
Analysis of Cost Drivers
Commonality of Terminology, Constructs, Life Cycle Phases, Units, …
Compatibility issues from any findings or recommendations
Consideration of common size drivers
Base assumptions of the models
5
Approach
• Determine estimate coverage in a common project framework –
Engineering Work Breakdown Structure (WBS) vs. Organizational
Breakdown Structure (OBS)
– Defined based on MIL-HDBK-881A and EIA/ANSI 632
– Generic contract structure
• Two-step exercise conducted in roundtables/workshops:
– Identify functional ownership of the tasks (leaf elements)
– Determine estimate coverage by current COCOMO II and COSYSMO
• Additionally, address the other areas for consideration through analyses
in the workshops
Note:
The use of COCOMO is incidental. It addresses similar issues
with other models.
6
Contract Engineering WBS Based On Standards
1.0 – System/Project
Product-oriented
construct, by
1.2 – Systems Engineering
tailoring MILHDBK 881A and
1.3 – Prime Mission Product (PMP)
1.3.1 – Subsystem / Configuration Item (CI) 1…n (Specify Names) EIA/ANSI 632
1.3.2 – PMP Application Software
1.3.3 – PMP System Software
1.3.4 – PMP Integration, Assembly, Test & Checkout (IATC)
1.3.5 – Operations/Production Support
1.1 – Integrated Project Management (IPM)
1.4 – Platform Integration
1.5 – System Test & Evaluation (ST&E)
1.6 – Training
1.7 – Data Management
1.8 – Peculiar Support Equipment
1.9 – Common Support Equipment
1.10 – Operational / Site Activation
1.11 – Industrial Facilities
Six Functions:
1. Systems Engineering
2. Software Engineering
3. Electrical Engineering
4. Mechanical Engineering
5. Support Engineering
6. Project Engineering Management
7
The Exercise
Y: COSYSMO; S: COCOMO; X: Ownership but no model coverage; U: Uncertainty
8
How Does It Work?
•
•
•
•
•
•
We compare the systems and software columns. If any WBS element a mark
(Y, S, or X), then it is “owned” by a function.
Gaps: If an element has no mark or only “X” in either column, then we have a
potential gap. That means neither COSYSMO nor COCOMO, as they stand
today, covers that task.
Overlaps: If an element has both a “Y” *and* an “S”, then we have a potential
overlap. That means both models estimate that scope and we have to
reconcile/deconflict.
Uncertainty: “U” for an elements means it is not consistently estimated by the
model
A task may be “owned” by other functions, e.g., HW, in which case it should be
covered by other functional models
Important:
– Task ownership not (necessarily) execution

You, the IPT lead, can use anyone you like to do the job
– With this analysis, we attempt to cover nominal projects or majority behavior. There
can be exceptions and it could be different from the last program you worked on,
which is irrelevant!
9
Summary of Analysis
• WBS vs. OBS Cross-reference Matrix identifies areas of gaps and overlaps
• Analysis identified more potential gaps than overlaps
• Potential gap areas:
– Project engineering management
 Variations in costs for types of life cycle models
 Quality Management
 Technical Process Strategy/Definition/Mgt (at SW level)
 Accounting for subcontract or supplier mgt (at SW level)
– Prime Mission Product (PMP) systems software
 SRS Development
 Development for Reusability (at system level)
– ST&E test equipment, facility, and support
 Accounting for SW support as needed
 Test Database Size (at system level)
– Ownership issues for the following major areas (often responsibility of Supportability)




Training
Data management
Site construction/conversion
Industry facilities
10
Summary of Analysis (Cont’d)
• Additional gap, not related to WBS
– COSYSMO does not address duration or schedule
• Potential overlap areas:
– PMP system design
 Algorithm Development
– Development test and evaluation (DT&E)
• Areas of Uncertainty (due to lack of explicit COSYSMO guidance)
– Discrepancy Report (DR) work-off within PMP or support equipment (CI level)
– ST&E Mock-ups / Prototypes / Simulations & Test Equipment
– Contractor Technical Support – onsite during/after system activation/turnover
11
Results of Other Analysis
• Analysis of Cost Drivers
– Most of the drivers have mappings between the models, albeit different
in granularity or handling
– Potential concerns covered in Gaps or Recommendations
• Commonality of Terminology, Constructs, Life Cycle Phases, Units,
…
– Additional commonality could improve concurrent usage, but is not
essential
• Compatibility issues from any findings or recommendations
– No apparent compatibility issues (backward compatibility or with other
models in COCOMO Suite) from recommendations
• Consideration of common size drivers
– No essential to harmonization, but may add utility to COCOMO
• Base assumptions of the models
– Need to document assumption for COSYSMO
12
Additional Recommendations
• Standard phase alignment for both models
– Per definitions used in ISO/IEC 15288 and 12207
• Establish means to adequately account for recursion (at level of
hands-off to SW)
– Needed to resolve gaps
• Establish operational guidance to minimize variation in usage
• Add Guidance to COSYSMO Drivers TO:
– Account for constraints (e.g. Time & storage) as requirements in the
size.
– Describe volatility covered in Requirements/Architecture understanding
• Look into ability to use COSYSMO size drivers in COCOMO for early
estimates
• Add documented list of assumptions to COSYSMO
13
Next Steps
• Provide results and recommendations to USC team for these models
• Conduct 2-day workshop at USC in conjunction with COCOMO
Forum
– 29-30 OCT 2008
– Workshop to be led by Ricardo Valerdi, Garry Roedler, and Jared
Fortune
– All PSM Workshop participants are invited
14
Backup
15
Gaps – Details
16
Gaps – Details (cont.)
17
Overlaps - Details
18
Uncertainties/Inconsistencies
19
Notes & Discussions
20