Systems Engineering and Project
Management: Similarities and
Differences for Cost Estimation
Dr. Ricardo Valerdi – Massachusetts Institute of Technology
Leone Young – Stevens Institute of Technology
COSYSMO Workshop
The University of Southern California (USC)
November 4th, 2010
The 25th Int’l Forum on COCOMO &
Systems/Software Cost Modeling
1
Workshop Agenda
• Introductions & Objectives
• Research Background
• Project Management Cost
Models and Feedback
• Survey
• Inputs & Discussion
1:30 – 1:35 pm
1:35 – 1:50 pm
1:50 – 2:05 pm
2:05 – 2:35 pm
2:35 – 2:50 pm
2
Objectives
• Discuss the similarities and differences between systems
engineering and project management
• Obtain feedback on a proposed project management
cost estimating model
• Conduct survey to determine the expected level of
project management efficiency multipliers
• Obtain inputs on the appropriateness and adequacy of
the proposed project management cost model, its cost
factors and drivers
• Provide an opportunity for participants to contribute their
expertise and perspective regarding project
management services and influence the direction of
future research
3
Research Background:
Systems Engineering (SE) & Project
Management (PM)
•Cost Management – Project Air Force by Rand
– Development Management = Systems Engineering (SE) & Project
Management (PM)
– Defense Industry – US Air Force Programs (Stem et al., 2006)
» Development Management (100/%) = SE (50%) + PM (50%)
»SE/PM costs doubled since 1960s
Aircraft SE/PM Costs as a Percentage of Total
Development Cost for All Development
Programs, 1960s–1990s (Stem, et al., 2006)
SE/PM as a function of Integrated Logistics
Support (ILS) for a typical Air Force program
(Stem, et al., 2006)
4
Research Background:
Systems Engineering (SE) & Project
Management (PM)
•SE Costs – significant amount of research has been
conducted
–The International Council on Systems Engineering (INCOSE)
surveyed (Honour, 2004):
•52% of systems projects spent 5% or less of total systems development cost on
SE tasks
–The Constructive Systems Engineering Cost Model (COSYSMO)
•As a SE cost estimating tool used by systems engineering, systems cost
estimators, etc (Valerdi, 2006)
•PM Cost Estimating?
–Methodology?
–Tools?
5
Research Background:
Systems Engineering (SE) & Project
Management (PM)
• Literature – limited information on PM related expenditures
or costs
• Organizations often do not identify or measure PM costs,
and a survey led by UC Berkeley (Ibbs and Kwak, 2000a, 2000b)
shows:
– 80% of companies surveyed spend less than 10% of total project
cost (TPC) for PM services
– Average = 6% of TPC, Range = 0.3% ~ 15% of TPC
– Another survey indicated the average = 10% of TPC (Ibbs and
Reginato, 2002)
• Evidently, PM costs varies among organizations
– Influential PM Cost Factors: project type, size, # of projects, PM
maturity level (Archibald, 2003)
6
Research Background:
Systems Engineering (SE) & Project
Management (PM)
• SE is “a methodical, disciplined approach for the design,
realization, technical management, operations, and
retirement of a system. SE is the art and science of
developing an operable system capable of meeting
requirements within often opposed constraints. Systems
engineering is a holistic, integrative discipline, wherein
the contributions of structural engineers, electrical
engineers, mechanism designers, power engineers,
human factors engineers, and many more disciplines are
evaluated and balanced, one against another, to
produce a coherent whole that is not dominated by the
perspective of a single discipline.” (NASA, 2007)
7
Research Background:
Systems Engineering (SE) & Project
Management (PM)
• Project Management Institute (PMI) - Project
Management Body of Knowledge (PMBOK) guidebook
defines PM as “the application of knowledge, skills, tools
and techniques to project activities in order to meet or
exceed stakeholder needs and expectations from a
project” (PMI, 2004)
• NASA defines PM as “the function of planning,
overseeing, and directing the numerous activities
required to achieve the requirements, goals, and
objectives of the customer and other stakeholders within
specified cost, quality, and schedule constraints” (NASA,
2007, 2010)
8
Research Background:
Systems Engineering (SE) & Project
Management (PM)
Technical Skills
Managerial Skills
The Overlapping Areas of SE & PM in a Project
(Kossiakoff and Sweet, 2003)
9
Research Background:
Systems Engineering (SE) & Project
Management (PM)
The Roles of Program/Project Manager and Systems Engineer in the
Defense Systems Project Life Cycle Processes
(DOD, 2010)
10
Research Background:
Systems Engineering (SE) & Project
Management (PM)
The Responsibility of Program/Project Manager and Systems Engineer
in the Defense Systems Project Life Cycle Processes
(DOD, 2010)
11
Research Model: PM Cost
Estimating Model
What is the dependency
between SE & PM?
Case 1
SE
PM
• how much SE effort do we
need to know before we
can determine PM effort
independently?
• If possible, how early in LC
can we assess PM effort?
Or it must be done
Interdependently?
SE
Scenario 1:
PM effort = f(SE effort)
Scenario 2:
PM effort ≠ f(SE effort)
November 3rd, 2010
Many suggested both SE/PM
are related, but their costs
and efforts were rarely
mentioned…
The 25th Int’l Forum on COCOMO &
Systems/Software Cost Modeling
12
Research Model: PM Cost
Estimating Model
• Is SE part of PM?
• Eisner’s view (2008) on
SE/PM
Case 2
• Is it common in industries?
PM
SE
SE
Scenario 1:
PM effort = f(SE effort)
Scenario 2:
PM effort ≠ f(SE effort)
November 3rd, 2010
The 25th Int’l Forum on COCOMO &
Systems/Software Cost Modeling
13
Research Model: PM Cost
Estimating Model
Case 3
If PM is part of SE…
• how much SE effort do
we need to know before
we can determine PM
effort?
SE
PM
Scenario 1:
PM effort = f(SE effort)
This view is usually SE
shared in
SE oriented communities &
gov’t entities…
Scenario 2:
PM effort ≠ f(SE effort)
November 3rd, 2010
The 25th Int’l Forum on COCOMO &
Systems/Software Cost Modeling
14
Research Model: PM Cost
Estimating Model
Model #1 ~ Synthesized via COSYSMO (Valerdi, 2005)
E
PM NS
14


 A    ( we ,k  e ,k  wn ,k  n ,k  wd ,k  d ,k )    EM j
j 1
 k

Where,
PMNS = effort in Person Months (Nominal Schedule)
A = calibration constant derived from historical project data
k = {REQ, IF, ALG, SCN}
wk = weight for “easy”, “nominal”, or “difficult” size driver
Case A: PM = f(SE)
Φk = quantity of “k” size driver
E = represents diseconomies of scale
EM = effort multiplier for the jth cost driver. The geometric product results in an overall
effort adjustment factor to the nominal effort.
SE PM
15
Research Model: PM Cost
Estimating Model
Potential model parameters were predetermined through various
knowledge sources (e.g. books, scholar publications, research
whitepapers, dissertations, professional and government publications,
etc.)
•Aerospace Engineering
•Civil Engineering
•Computer Science
•Construction Engineering and Management
•Defense/Military
•Engineering Management
•Government
•Information Technology
•Management Information Systems
•Professional Societies
•Project Management
•Risk Management
•Software Engineering
•Systems Engineering
16
Research Model: PM Cost
Estimating Model
The initial 18 PM effort multipliers are listed as following:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
Scope Understanding
Scope Volatility
Scope Growth
Requirements Volatility
Requirements Growth
Budget Constraints
Schedule Span
Project Complexities
Systems Complexities
Documentation Level
Level of Service Requirements
Stakeholder Cohesion
Project Management Maturity
Project Management Experience/Continuity
Process Capability
Technology Maturity and Risk
Tool Support
Multisite Coordination
These initial PM cost
indicators were determined
to be possibly correlated to
factors that have effects on
SE/PM cost adjustment
factors (Akintoye, 2000;
Anderson and Brown, 2004;
Crawford et al., 2005; de
Wit, 1988; Hamaker and
Componation, 2005;
Hartman and Ashrafi, 2002;
Honour, 2010; NASA, 2010;
Valerdi, 2005)
17
Research Model: PM Cost
Estimating Model
Is PM effort proportional to SE effort?
Project
Complexity
PM
SE
System
Complexity
What if PM ≠ f(SE) ?
November 3rd, 2010
The 25th Int’l Forum on COCOMO &
Systems/Software Cost Modeling
18
Research Model: PM Cost
Estimating Model
Model #2
E
PM NS
5


 A    ( we ,k  e ,k  wn ,k  n ,k  wd ,k  d ,k )    EM j
j 1
 k

Where,
PMNS = effort in Person Months (Nominal Schedule)
A = calibration constant derived from historical project data
k = {REQ, PCR, CST, SCM, DCL}
wk = weight for “easy”, “nominal”, “difficult”, or “low”, “medium”, “high” size driver
Φk = quantity of “k” size driver
E = represents diseconomies of scale
EM = project management efficiency multiplier for the jth cost driver. The geometric
product results in an overall effort adjustment factor to the nominal effort.
19
Research Model: PM Cost
Estimating Model
Consolidated 5 Cost Factor Categories
Requirements
& Scope (REQ)
X
REQ
Efficiency: PPT
X
PCR Efficiency:
PPT
X
CST Efficiency:
PPT
Stakeholder
Cohesion &
Multisite
Coordination
(SCM)
X
SCM
Efficiency: PPT
Document &
Comm Level
(DCL)
X
DCL Efficiency:
PPT
Project
Complexity &
Risk (PCR)
Constraints
(CST)
S
E
= PMNS
PPT = Project Management Capability and Maturity on People, Process & Tools
20
Research Model: PM Cost
Estimating Model
Cost Factor #1
Requirements & Scope (REQ)
Related
& Depends on SE
– How well understood is the project?
•Scope of requirements
•Number of requirements
•How well they are defined
•Statement of Work (SOW), Work Breakdown Structure
(WBS), etc
•Volatility/Rate at which they are changing
•Expect any new/additional requirements ~ Requirement
Creep?
21
Research Model: PM Cost
Estimating Model
Cost Factor #2
Project Complexity & Risk (PCR)
Related
& Depends on SE
– How much risk is there?
•What is the level of risk for the project?
•What is stakeholders’ risk attitude – risk adverse?
•How difficult is it to assess the risk?
•How complex is the project?
•Project Complexities
•organizational, technological/product
•e.g. PM related integration, coordination, etc
•Number of known project complexity and risks
22
Research Model: PM Cost
Estimating Model
Cost Factor #3
Constraints (CST)
Partially related
to SE
– How tight are the constraints?
•Schedule Span
•Time constraints
•Deliverable date
•Amount of slack time allowed
•Budget Constraints
•Money/Cost constraints
•Resource Constraints
•Human Resources
•Function/Feature
•Minimum acceptable features
•Quality
•Minimum acceptance by customers
23
Research Model: PM Cost
Estimating Model
Cost Factor #4
Stakeholder Cohesion & Multisite Coordination
(SCM)
– Amount of external PM work to be done
•Number of stakeholders
•Diversity of stakeholders
•e.g., have opposing goals/objectives, have different world
views
•Communication challenges
•external clients, internal clients, contractors,
languages, time zone difference, etc
24
Research Model: PM Cost
Estimating Model
Cost Factor #5
Documentation & Communication Level (DCL)
– Amount of internal PM work to be done
•Amount and complexity of required documentation
•e.g., project plan, resource management plan, status
reports, etc
•Amount and complexity of required communications
•Cubical/office noise
•Solving project issues
•e. g., number, length and frequency of meetings, etc
25
Research Model: PM Cost
Estimating Model
5 PM Efficiency Multipliers (PPT)
Requirements
& Scope (REQ)
X
REQ
Efficiency: PPT
X
PCR Efficiency:
PPT
X
CST Efficiency:
PPT
Stakeholder
Cohesion &
Multisite
Coordination
(SCM)
X
SCM
Efficiency: PPT
Document &
Comm Level
(DCL)
X
DCL Efficiency:
PPT
Project
Complexity &
Risk (PCR)
Constraints
(CST)
S
E
= PMNS
PPT = Project Management Capability and Maturity on People, Process & Tools
26
Research Model: PM Cost
Estimating Model
Project Management Efficiency Multipliers
Capability & Maturity of People, Process & Tools (PPT)
People Capability – PM Attributes*
•Communication skills
•PM experience
•Information sharing willingness
•Delegates appropriately
•Well-organized
•Supports and motivates project team
•Open-minded and flexible
•Provide constructive criticism
•Positive attitude
•Technical competency
•Team builder & player
•Ability to evaluate and select project resources
•Goal oriented
•Courage and conflict solving skills
•Problem solver
•Take initiative
•Creativity
•Integrator (team, PM activities, etc)
•Decision making skills
27
Research Model: PM Cost
Estimating Model
Project Management Efficiency Multipliers (cont’d)
Capability & Maturity of People, Process & Tools (PPT)
Process Maturity**
•PM process maturity (CMMI, The Berkeley PM Process Maturity Model)
•Organization PM maturity (PMI-OPM3)
•Initial
•Repeatable
•Defined
•Managed
•Optimized
Tool Support**
•Level of tool automation
•Very few primitive tools
•Basic/Micro tools
•Extensive/Few Integrative tools
•Moderately integrated environment
•Fully integrated environment
*Adapted from Software Development Cost Estimating Guidebook (USAF Air Logistics Center, July, 2009) & Essentials of Project and systems
Engineering Management (Eisner, 2008)
28
**Adapted from Software Development Cost Estimating Guidebook (USAF Air Logistics Center, July, 2009)
Survey
• Please take some time to answer each question
• Feel free to ask questions
29
Questions for Discussion
• In industry, if the same person who performs both
SE/PM functions, how do you estimate such effort?
• Which case and effort function scenario do you think is
more realistic? Why?
• How do industry corporations and government estimate
PM cost & effort?
• What type of tool & method do they use?
• How can the propose model become more practical and
applicable for industry use?
• What is missing in this research?
30
Additional Questions?
Suggestions?
Comments?
Thank You Very Much for Your Time!!!
31