University of Southern California
Center for Systems and Software Engineering
Results of Reuse Survey
Jared Fortune, USC
Ricardo Valerdi, MIT
COSYSMO Workshop @
COCOMO Forum 2008
Los Angeles, CA
Gan Wang, BAE
1
University of Southern California
Center for Systems and Software Engineering
Outline
• Research Background
• State of the Practice Survey
• Results
• Implications for COSYSMO 2.0
2
University of Southern California
Center for Systems and Software Engineering
COSYSMO Reuse Development Timeline
COSYSMO-R
Developed At
Lockheed
Martin [5]
Extensions for
COSYSMO to
Represent Reuse
Published [3]
COSYSMO 1.0
Published [1]
2005
2006
Reuse Identified
as a Critical
Improvement [2]
BAE Pilot
Test Shows
Promising
Results [8]
2007
Survey Results on
State of the
Practice [11]
2008
2009
Reuse
Continuum
Identified;
“Bottoms-Up”
Approach
Proposed [7]
Other Reuse
Considerations
Identified [10]
Preliminary
Categories
of Reuse
Identified [4]
Reuse
Definitions
Identified [6]
[1] Valerdi. COSYSMO. Ph.D. Dissertation,2005.
[2] Valerdi. COSYSMO Workshop. USC ARR, 2006.
[3] Valerdi, Gaffney, Roedler, Rieff. COSYSMO Extensions. COCOMO
Forum, 2006.
[4] Valerdi. COSYSMO Working Group. PSM Workshop 2006.
[5] Gaffney. COSYSMO-R, 2007.
COSYSMO 2.0
“Bottoms-Up”
Results from
BAE
Presented [9]
[6] Valerdi. COSYSMO Working Group. PSM Workshop, 2007.
[7] Valerdi, Wang, Roedler, Rieff, Fortune. COSYSMO Reuse
Extension. COCOMO Forum, 2007.
[8] Wang. COSYSMO Reuse. COCOMO Forum, 2007.
[9] Wang, Valerdi, Fortune. COSYSMO Reuse Extension. IEEE, 2008.
[10] Fortune, Valerdi. Reuse Considerations. AIAA Space, 2008.
[11] Fortune, Valerdi, Wang. State of the Practice. COCOMO Forum,
2008
3
University of Southern California
Center for Systems and Software Engineering
COSYSMO 2.0 Development
Reuse
Considerations
Reuse
Observations
Revised Drivers
Literature Review
Industry Survey
COSYSMO 2.0
• Literature review helped formulate survey questions
• Survey results guided proposed COSYSMO 2.0
revisions
4
University of Southern California
Center for Systems and Software Engineering
State of the Practice Survey
How does industry handle reuse?
Reuse Survey Responders
BAE Systems
General Dynamics
Lockheed Martin
Orbital Sciences
Raytheon
Reynolds, Smith, and Hills
Eight responses,
representing eleven
subject matter experts
COSYSMO 2.0
Reuse Survey
5
University of Southern California
Center for Systems and Software Engineering
Survey Results (1)
How does your organization define reuse?
…it varies
Selected responses
• “No formal definitions”
• “Use of all or part of systems engineering work products”
• “Use of design, pattern, template, handbook, or other engineering
effort that shifts the way engineering is done”
• “Use of assets developed or acquired in response to requirements
for one application, in whole or in part to satisfy requirements for
another application”
Use of existing systems engineering products in a new application
6
University of Southern California
Center for Systems and Software Engineering
Survey Results (2)
What are the systems engineering artifacts your
organization reuses and how frequently are they reused?
Frequency of Reuse of Systems Engineering Artifacts
General
Documentation/Templates
COTS Products
Test Data/Procedures
Architecture/Design Models
Other
Specific
Requirements
Never
Occasionally
Always
7
University of Southern California
Center for Systems and Software Engineering
Survey Results (3)
Which of the artifacts listed above is the most effective at
providing a net benefit when reused?
Artifacts Cited as Most Effective at Providing a
Benefit When Reused
Requirements
45%
Test Data
11%
Requirements are
the home run of
reuse
Other
22%
Documentation
22%
8
University of Southern California
Center for Systems and Software Engineering
Survey Results (4)
To what extent does the reuse of systems engineering
artifacts occur?
Extent of Reuse of Systems Engineering Artifacts
Conceptualize
Similar to
distribution of
systems
engineering effort
Develop
Test and Evaluation
Transition to Operations
Unaware
Ad Hoc
Planned
9
University of Southern California
Center for Systems and Software Engineering
Survey Results (5)
What are the reasons for reuse successes?
–
–
–

Artifact reused with minimum or no change
Product lines with significant similarities
Requirements management
Utilization of personnel with experience on the project that
developed artifact
What are the reasons for reuse failures?
–
–
–

Underestimated modification required for reuse
New requirements placed on a modified product
Customer didn’t modify expectation of risk
Lack of knowledge/understanding
10
University of Southern California
Center for Systems and Software Engineering
Survey Results (6)
What are the most frequently promoted benefits as
justification for systems engineering reuse?
Promoted Benefits for Systems
Engineering Reuse
Quality
15%
Cost
29%
Cost benefits
implied in
others?
Schedule
21%
Risk
19%
Performance
16%
11
University of Southern California
Center for Systems and Software Engineering
Survey Results (7)
How frequently is systems engineering reuse mentioned in
an RFP for a new system?
Frequency of Systems Engineering Reuse
Mentioned in an RFP
Always
13%
Never
13%
Few instances
of systems
developed
without reuse
Occasionally
25%
Seldom
50%
12
University of Southern California
Center for Systems and Software Engineering
Survey Results (8)
How do the expected savings from reusing systems
engineering artifacts scale?
Scaling of Expected Reuse Savings
Other
13%
Linear
25%
Other is a combination
of linear and non-linear
Consensus was nonlinearly decreasing as
the number of
interfaces grows
Non-linear
62%
13
University of Southern California
Center for Systems and Software Engineering
Survey Results (9)
Evaluate the expected effort from utilizing a systems
engineering artifact classified in the first category, compared
to the second.
Expected Effort from Reusing a Systems Engineering Artifact
New vs. Modified
New vs. Adopted
New vs. Deleted
New vs. Managed
Modified vs. Adopted
Modified vs. Deleted
Modified vs. Managed
Adopted vs. Deleted
Adopted vs. Managed
Deleted vs. Managed
Less Than
About the Same
More Than
14
University of Southern California
Center for Systems and Software Engineering
Implications to COSYSMO 2.0
• Effects of systems engineering reuse are more than
what is captured in the size drivers
– Survey results identify personnel, processes, and platform
factors
• Reuse needs to be accounted for in both the size and
cost drivers
– Size drivers: previously proposed reuse extensions (Valerdi,
Gaffney, Wang)
– Cost drivers: newly proposed additional cost drivers (Fortune)
15
University of Southern California
Center for Systems and Software Engineering
Proposed Size Driver Extensions
New: Artifacts that are
completely new
New 1.0
Modified: Artifacts that are
inherited, but are tailored
Deleted: Artifacts that are
removed from a system
Managed: Artifacts that are
incorporated unmodified and
untested
Based on survey results
Reuse weight
Adopted: Artifacts that are
incorporated unmodified, also
known as “black box” reuse
Modified
Managed
Adopted
Deleted
0
Reuse category
16
University of Southern California
Center for Systems and Software Engineering
Proposed Additional Cost Drivers
Reuse Understanding
Artifact Unfamiliarity
Reuse Understanding
Organization
Processes to capture or implement the reuse of
artifacts; repeatable
Domain
Applicability
Overlap between the original domain of the artifact
and the domain the artifact is being reused within
Availability of documentation or other non-personnel
Technology
related knowledge assets that provide for or improve
Comprehension the understanding of the technology being addressed
in the reused artifact
Artifact Unfamiliarity
Completely
familiar
Systems engineer directly assisted in the
development of the artifact for the original
system; continual experience with the artifact;
first-hand knowledge of the heritage system is
available
Systems engineer participated in the
Mostly familiar development of the artifact for the original
system; infrequent experience with the artifact
Somewhat
familiar
Systems engineer has some familiarity with the
artifact and the original system which it was
derived from; no first-hand knowledge of the
heritage system
Mostly
unfamiliar
Systems engineer has experience with similar
artifacts but not the current one being reused;
limited knowledge of the heritage system
Completely
unfamiliar
Systems engineer has no previous experience
with the artifact or the system which the artifact
was derived from; completely unknown
17
University of Southern California
Center for Systems and Software Engineering
Conclusion
• Thanks to all who participated in the reuse survey
• Detailed discussion on the COSYSMO 2.0 model to
follow in the “Reuse Framework” presentation
18