• Mauricio Pena (USC/Boeing)
• Anca Juliana-Stoica
(Uppsala University)
• Andy Nolan (Rolls-Royce)
• Ed Colbert (USC)
• Jorge Aguilar (UPAEP)
• Mary Boghosian (Aerospace
Corp.)
• Sue Koolmanojwong (USC)
• Monvorath Phonpaibul
(IBM/Thammasat
University)
• Hunter Deane (ICONIX)
• Doug Rosenberg (ICONIX)
• Nupul Kukreja (USC)
• Warren Reid (WSR
Consulting)
• Ray Madachy (Naval
Postgraduate School)
• Ricardo Valerdi (University of Arizona)
2

• Goal: Engage stakeholders from the cost community interested in education and training to learn and share best practices related to curriculum development and delivery.
• Our discussion guided by
– Links between cost estimation curriculum and ongoing efforts to update the systems engineering body of knowledge (BKCASE) and software engineering reference curriculum (GSwERC)
– Sharing of cost estimation course syllabi from the Naval
Postgraduate School, University of Arizona, and other universities
– Review of Systems Engineering Cost Estimation Workbook being developed by Madachy and Valerdi
• Obtained feedback to move the cost estimation profession forward.
3

• Thammasat University in Thailand
– cost estimation is part of the Project Management class
(undergraduate and graduate students in software engineering)
• Uppsala University in Sweden
– software and systems engineering classes (Masters and PhD students in IT Project Management), deals with both HW and SW
• Rolls-Royce
– three courses: 1) how to estimate, 2) how to develop tools, 3) how to be a good recipient of an estimate. Management of uncertainty is a significant part.
• University of Southern California
– graduate level classes for Software Engineering Management &
Economics, and Software Engineering contain cost estimation topics
4

•
– E.g. business concerns, stakeholder management
•
•
5

• Include example systems, learning curve problems, life cycle cost problems
• Be problem-based vs. model-specific
• Add “problem domain” dimension to coverage matrix
• Include problems with size expansion ratios, DODAF, estimation risk/confidence
• Add bottom-up and top-down cost estimates
• Change reuse questions for students to figure out how much of the system is new/modified/adopted vs. handing the values
• Tradeoff exercises between cost and performance by negotiating scope; scope creep exercises to test cost model sensitivity
6

• Introduce workbook with everyday estimation problems
• Cover art and science of estimation
• Introduce case studies to include lessons learned
• Sync cost estimation course with another project course developing software so that students can estimate that development; then compare estimates vs. actuals
• Have examples include multi-criteria decision making
(cost, schedule, risk, performance, value, etc.)
• Add annotated bibliography and glossary
7

8

• A reference curriculum for a Masters Degree in
Software Engineering based on Software
Engineering Body of Knowledge SWEBoK
– Joint industry, academic, and government initiative
• Modified SWEBOK in several places, including adding Engineering Economics and Risk
Management to SWE Management Knowledge
Area
9

5. Understand the relationship between software engineering and systems engineering and be able to apply systems engineering principles and practices in the engineering of software.
– I.3. Risk Management
7. Be able to reconcile conflicting project objectives, finding acceptable compromises within limitations of cost, time, knowledge, risk, existing systems, and organizations.
– I.1. Software Project Planning
– I.2. Risk Management
– I.7. Engineering Economics
10

• The Body of Knowledge and Curriculum to Advance
Systems Engineering (BKCASE) project is defining a Systems Engineering Body of Knowledge (SEBoK) and developing an advanced Graduate Reference
Curriculum for Systems Engineering (GRCSE) based on the (SEBoK)
– Joint industry, academic, and government initiative
– See v.0.5 at bkcase.org
• SEBoK Knowledge Areas involving cost estimation
– Project Planning, Measurement, Risk Management,
Systems Analysis, others
11

•
– Offered multiple times per year in Monterey, CA
(and virtually)
•
– Offered every Spring in Tucson, AZ (and virtually)
12

•
•
•
13

• Understand how cost is used in choosing between alternatives
• Understand basics of cost behavior and cost allocation
• Understand how costs change over the life-cycle of a system
• Choose between alternatives using economic reasoning
• Understand cost issues related to program management
• Develop life-cycle cost models
• Choose between alternatives using net present value analysis
• Apply cost estimating techniques to develop cost estimates
• Incorporate uncertainty into cost estimates
• Quantify and evaluate cost risk
• Develop and analyze system cost
14

• Demonstrate the ability to identify, formulate, and solve operational, technical, and engineering problems in systems engineering and related disciplines using the techniques, skills, and tools of modern practice, including modeling and simulation. These problems may include issues of research, design, development, procurement, operation, maintenance or disposal of systems and processes for military applications.
• Demonstrate proficiency in the systems engineering process, including defining requirements, conducting functional analysis, designing and architecting a system, analyzing it against requirements, allocation of requirements to sub-systems, conducting trade-off studies,
determining the cost of the system, integrating human factors into the system, designing logistical supportability, and planning for its testing and evaluation.
• Demonstrate proficiency in core skills of systems analysis, to include deterministic and stochastic modeling of systems, optimization, decision analysis, risk analysis, economic models, and lifecycle supportability analysis. This includes familiarity with combat simulations and combat modeling.
• Demonstrate the ability to work as a team member or leader in a large systems engineering project, and to provide leadership in the systems engineering management process. The graduate must be able to interact with personnel from other services, industry, laboratories and academic institutions.
• Demonstrate competence in the planning and management of large systems engineering projects.
• Demonstrate proficiency in written and oral presentation of technical material.
15

• Provide a flexible, one-stop resource for companies and Universities who provide education/training in cost modeling
• Example University courses being taught by
– Young Kwak, George Washington University
– Peggy Brouse, George Mason University
– Dan Nussbaum, Naval Postgraduate School
– John Farr, West Point
– Sue Koolmanojwong, USC
– Etc.
16

• Systems Engineering
• Software Engineering
• Hardware Engineering
• Human Systems Integration
• Systems of Systems
• Product Lines
• Cost model extensions (quality, multiple modules, increments, etc.)
• Exercises in all areas
• Spreadsheets and web-based estimation tools
• Instructors version
17

• IN
– Equations for parametric models
– Cost driver rating scales and effort multipliers
– Tables to decompose estimates by phases and activities
– Equations for other decision metrics such as ROI, PV, etc.
– Worked out examples and student exercises for all models
– Calibration equations for models (linear and logarithmic)
– Instructor version includes all answers plus master tool to compose and solve modified problems
• OUT
– Background on how the models were developed
– Details of the phenomena behind the cost driver values
– See COSYSMO book, COCOMO II book, SOSE dissertation, COSYSMO reuse dissertation, etc. for above
18

19

•
•
•
•
20

• Coverage: Systems engineering reuse; equivalent size; tradeoff analysis
Three system configurations are under consideration that employ systems engineering reuse in varying degrees. The reuse categories are from the COSYSMO reuse model.
Option 1: All New Development
Option 2: Adapt Legacy System
50% Deleted from the legacy system
The system will then be comprised of:
50% New
50% Modified
Option 3: Product Line Approach
25% Designed for Reuse (across product line)
25% Modified
50% Managed
21

• Coverage: Software cost; software reuse; tradeoff analysis; military application
A top Army project is for an application that enables soldiers to find the location of resources for Forward
Operating Bases (FOBs). To achieve the Geo-Mapping capability, a geographic software vendor is offering a tailored version of Google Maps at an up-front license price of $500K.
However, an alternate proposal is to use a dedicated team with expertise in geographic applications and
Google Maps to build a better and cheaper map system. Prepare a COCOMO estimate of the custom development cost using these inputs:
Adapt the Google Map software of 80 KSLOC with the reuse parameters:
% Design Modified = 35
% Code Modified = 20
% Integration Required = 60
Assessment and Assimilation = 0
Software Understanding = 10
Unfamiliarity = 0.2
22

• Coverage: Software cost; software reuse; tradeoff analysis; military application
Cost drivers are all Nominal except for the following:
• Programmer Capability and Analyst Capability: 75th percentile
• Personnel Continuity: stable staff with 3% annual turnover
• Multi-site Development: entire team is co-located together
• Use of Software Tools: strong toolset, moderately integrated
The cost per person-month is $8K.
Additionally, the manager believes that they would have no case if they delivered the software system too late.
Thus they must have the software completed 6 weeks ahead of the deadline, which would suppress the schedule to 85% of the nominal schedule.
Determine the estimated cost of adapting the Geo-Mapping capability for the application. Is it a better option from a cost standpoint than the vendor’s tailored product?
23

• Coverage: Software and hardware lifecycle costs; software reuse; time value of money
Develop an integrated software and hardware lifecycle cost over time using the equipment procurement from
#1, the mini-drone first unit cost from problem #2, and the following for software labor.
Software Portion
The newly developed software size is 250,000 SLOC and modified software size is 90,000 SLOC.
- the schedule compression ratio is 85%
- the effect of a software failure is a large equipment loss
- average language and tool experience is 3 years
- there is expected to be a major change in the operating system once per year
- use of a comprehensive toolset fully integrated across all activities.
For the modified software:
- 40 out of 200 design modules were modified
- 30 KSLOC out of the total 90 KSLOC were modified
- the integration of the modified code is estimated to be 2/3 of the effort compared to if it was all new
- only basic module search and documentation was required for its assessment and assimilation
- the team did not develop the software being modified and are unfamiliar with it.
- to understand the software, it is described as: structure - high cohesion, low coupling application clarity - moderate correlation between program and application self-descriptiveness - moderate level of comments, headers, documentation.
24

• Coverage: Software and hardware lifecycle costs; software reuse; time value of money
Create a monthly budget profile of costs for the full project including labor by software engineering phase, the first drone development costs and non-labor items for the testing equipment. The drone development will dovetail with the software completion. Assume its monthly costs are constant for the duration of software Inception through Construction (i.e. spread evenly over this time). It finishes when software
Transition starts.
Assume a nominal annual interest rate of 5% and labor rate of $10K/Person-Month.
Your final output should provide the costs for every month of the project with separate items for the individual software engineering phases, hardware development, and other procurement costs. Provide the lifecycle totals for software, hardware, procurement and grand total cost.
25

• Coverage: Systems engineering process improvement; ROI
In a systems engineering organization, automated tools for architectural development and verification have been developed along with other process improvements. Their re-calibrated COSYSMO equation shows a reduced size exponent from 1.06 to 1.03:
Effort (Person-Months) = .254* System Requirements 1.03
On average, the organization develops 500 Equivalent System Requirements per year at $10,000/Person-
Month. What is their estimated cost savings per year compared to before the improvements? What is their ROI over time for 5 years if the improvements cost 2 person-years?
26

• Coverage: Systems and software lifecycle costs; equivalent size; time value of money; tradeoff analysis; military application
The Navy desires to develop information dominance capabilities to satisfy national security strategy objectives for Maritime Domain Awareness (MDA). These MDA capabilities will be developed through the use of new and emerging commercial software technologies.
By combining these technologies, different Courses of Action (COAs) for the capability of “Develop Awareness” were developed for the OPNAV Capability Based Assessment. COA 1 assumes that the technologies will be inserted within 13 existing systems. COA 2 creates a new architecture that combines the systems into one overall system, so the software to be developed is dramatically reduced. These ground rules and assumptions for these options are listed below.
Ground Rules and Assumptions for the Materiel Solutions
1. The wage inflation rate is 3% per year for all years beyond FY12.
2. Annual maintenance costs are 25% of the total software cost and do not start until the software development is complete.
3. Use $10,000 cost per person month for systems and software engineers (constant year dollars).
4. COAs are costed independently.
5. Assume both COAs start at beginning of FY12.
27

• Coverage: Systems and software lifecycle costs; equivalent size; time value of money; tradeoff analysis; military application
28

• Coverage: Systems and software lifecycle costs; equivalent size; time value of money; tradeoff analysis; military application
Use the above information to estimate an annual cost of COAs 1 and 2 for the years of FY12 to FY17. List the costs per year in constant (FY 12) and future (then year) dollars.
Assume that systems and software engineering are sequential activities, with software engineering not starting until the systems engineering is complete. Only the software Elaboration and Construction phases are to be estimated, as requirements will already be defined and transition starts immediately. Software maintenance costs start incurring when software engineering is complete (after Construction). The annual wage inflation rate is 3% (as FY12 dollars are used, start inflation in FY13). It can be assumed that all systems engineering costs will occur in Year 1 (FY12) for both COAs.
29