University of Southern California
Center for Systems and Software Engineering
Expert COSYSMO
Ray Madachy, Ricardo Valerdi
USC Center for Systems and Software Engineering
MIT Lean Aerospace Initiative
madachy@usc.edu, rvalerdi@mit.edu
22nd International Forum on COCOMO and
Systems/Software Cost Modeling
November 1, 2007
©USC-CSSE
1
University of Southern California
Center for Systems and Software Engineering
Introduction
• An expert system tool for systems engineering risk
assessment based on the Constructive Systems
Engineering Cost Model (COSYSMO) [Valerdi 2005]
– Automatically identifies project risks in conjunction with cost
estimation similar to Expert COCOMO [Madachy 1997]
– Supports project planning by identifying, categorizing,
quantifying, and prioritizing system-level risks
– Includes 98 risk conditions
• Risk situations are characterized by combinations
of cost driver values indicating increased effort
with a potential for more problems
• Simultaneously calculates cost to enable tradeoffs
with risk
©USC-CSSE
2
University of Southern California
Center for Systems and Software Engineering
Method
• Analyzes patterns of cost driver ratings
submitted for a COSYSMO cost estimate
against pre-determined risk rules
– Identifies individual risks that an experienced
systems engineering manager might recognize
but often fails to take into account
– Helps users determine and rank sources of
project risk. With these risks, mitigation plans
can be created based on the relative risk
severities and provided advice
©USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
Method (cont.)
• COSYSMO cost factor combinations used as
abstractions for formulating risk heuristics
– E.g. if Architecture Understanding = Very Low and Level
of Service Requirements = Very High, then there is a
high risk
• Since systems with high service requirements are
more difficult to implement especially when the
architecture is not well understood
• Elicitation of knowledge from systems
engineering domain experts in CSSE-sponsored
workshops
– Survey used to identify and quantify risks
• Devised knowledge representation scheme and
risk quantification algorithm
©USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
ARCH
LSVC
MIGR
TRSK
DOCU
INST
RECU
TEAM
PCAP
PEXP
PROC
SITE
TOOL
SIZE (REQ + INTF + ALG + OPSC)
Requirements Understanding
Architecture Understanding
Level of Service Requirements (the ilities)
Migration Complexity (legacy system considerations)
Technology Risk (maturity of technology)
Documentation match to life cycle needs
Number and Diversity of Installations or Platforms
Number of Recursive Levels in the Design
Stakeholder Team Cohesion
Personnel/team capability
Personnel Experience and Continuity
Process Capability
Multisite Coordination
Tool Support
RQMT
SIZE
Risk Conditions
21
21
17
9
9
9
12
7
10
5
5
8
12
7
8
4
3
3
4
1
2
7
5
7
5
10
8
2
10
9
11
3
1
6
3
4
8
5
6
6
4
4
4
3
4
9
10
11
4
7
9
4
5
8
7
11
8
11
4
7
5
2
6
7
9
12
7
5
5
2
3
3
6
4
7
3
9
10
6
4
6
3
5
3
2
8
2
8
8
8
5
7
1
4
2
4
5
3
5
5
3
5
3
8
8
high risk
small x = 0.5; big X = 1
medium risk
n = 19
low risk
©USC-CSSE
5
University of Southern California
Center for Systems and Software Engineering
Risk Taxonomy and Weighting
Project Risk
Product risk
Process risk
Personnel risk
Platform risk
#categories #category risks
Project Risk=
risk level i , j * effort mu ltiplier p roduct i , j
j= 1
i= 1

where risk level =
1
2
4

moderate
high
very high
effort multiplier product=
(driver #1 effort multiplier) * (driver #2 effort multiplier) ... * (driver #n effort multiplier).
©USC-CSSE
6
University of Southern California
Center for Systems and Software Engineering
Next: Finer Assignment of Risk Levels
ATTRIBUTE 1
very low
extra high
very low
very high
high
moderate
increasing risk
ATTRIBUTE 2
very high
discretized into
ATTRIBUTE 1
VERY LOW
VERY LOW
LOW
ATTRIBUTE 2 NOMINAL
HIGH
VERY HIGH
LOW
NOMINAL
HIGH
VERY HIGH EXTRA HIGH
MODERATE HIGH
VERY HIGH
MODERATE HIGH
MODERATE
©USC-CSSE
7
University of Southern California
Center for Systems and Software Engineering
Expert COSYSMO Inputs
©USC-CSSE
8
University of Southern California
Center for Systems and Software Engineering
Expert COSYSMO Outputs
©USC-CSSE
9
University of Southern California
Center for Systems and Software Engineering
Current and Future Work
•
•
•
•
•
Currently scaling the risk summary outputs for each category and
defining ranges for low, medium and high risks
Create more granular risk quantification rules
Generate expert risk mitigation advice for each risk condition, and
provide that automated guidance to users to help develop their own
mitigation actions
Add rules to detect COSYSMO input anomalies
Systems engineering risk data from industrial projects will be
analyzed to enhance and refine the technique
– Perform statistical testing
•
Domain experts from industry and government will continue to
provide feedback and clarification
– Supporting surveys and workshops will be continued
•
Exploration of alternate risk and uncertainty approaches to
integrate multiple risk management viewpoints into a more
complete risk management framework
©USC-CSSE
10
University of Southern California
Center for Systems and Software Engineering
References
• R. Madachy, Heuristic Risk Assessment Using
Cost Factors, IEEE Software, May 1997
• Valerdi R., The Constructive Systems
Engineering Cost Model (COSYSMO), PhD
Dissertation, University of Southern California,
Los Angeles, CA, May 2005
• http://csse.usc.edu/tools/expert_cosysmo.php
©USC-CSSE
11