University of Southern California
Center for Systems and Software Engineering
Enablers and Inhibitors for
Expediting Systems and
Software Development
Sue Koolmanojwong and Jo Ann Lane
COCOMO Forum
October 16, 2012
University of Southern California
Center for Systems and Software Engineering
Overview
• Characterize “expediting”
• Overview of current research
• Enablers and Inhibitors for “expediting”
– Single system
– Systems that participate in one or more systems of
systems (SoS)
– SoS capabilities
• Observations
2
University of Southern California
Center for Systems and Software Engineering
What Does it Mean to “Expedite
Systems Engineering”?
• Expedite “systems engineering” or “system
development”?
– Most are interested in “system development”:
Capability development schedule from concept to
delivery
– Some will include enhancement, maintenance,
retirement
Early decisions can affect ability to expedite later….
3
University of Southern California
Center for Systems and Software Engineering
General Ways to “Expedite”
•
•
•
•
Minimal engineering/quick solutions
Minimal features
Commercial-off-the-Shelf (COTS) solutions
Lean approach
– Eliminate non-value adding activities
– Reduce wait times
• Pacing
– Go slow to establish good
• Foundation
• Architecture
• Interfaces
• Relatively low complexity
– Then go fast
4
University of Southern California
Center for Systems and Software Engineering
Balancing “Expedited Engineering”
• Trades to consider when “expediting”
–
–
–
–
–
–
Long term affordability
Flexibility/adaptability for meeting future needs
Desired level of performance/speed/throughput
Maintainability
Securability
… and others
• Trades may
– Reduce future flexibility
– Result in
• Degradation of existing capabilities
• System limitations
• Later rework
Depending on the situation/need, it may be OK to
incur technical debt….
5
University of Southern California
Center for Systems and Software Engineering
Tradespace Example:
System Flexibility
• Goal of “flexibility” is to go beyond quick solution
to build in flexibility that will allow system to
– Easily evolve in the future to meet future (often
unknown) needs
– Interoperate with future systems (e.g., in one or more
SoS environments)
• Must balance “flexibility” with “complexity”
– Performance issues may result if system tries to be
“everything for everyone”
• Ways to evaluate flexibility
– Total ownership costs
– Option analyses using Monte Carlo techniques
6
University of Southern California
Center for Systems and Software Engineering
Expediting Development and
Increasing Value through Flexibility*
•
Flexibility in design
– Routinely improves expected value by 25% or more
– Enables system to
• Avoid future downside risks
• Take advantage of new opportunities
– Often reduces initial capital expenditures
• Greater expected value at less cost
• Enables manager to better control the risks
• Substantial increases on the return on investment
•
“Sweet-spot” found through Monte Carlo analysis of business
options
– Identifies how much engineering/system performance/system capacity
is enough
– Allows future decisions/investments to be made when more is known
about the future
•
Types of flexibility to explore depend on the context
* Richard de Neufville and S. Scholtes, Flexibility in Engineering Design, The MIT Press, Cambridge MA, 2010.
7
University of Southern California
Center for Systems and Software Engineering
Factors in Expedited Systems
Engineering
• 2012 PSM Workshop
– Representatives from commercial, military/ defense, and
academic sectors
– List the enablers and inhibitors
• A Single System
• An Existing Single System
• A System of Systems
– Rate High, Medium, Low for the impact level
8
University of Southern California
Center for Systems and Software Engineering
Enablers and Inhibitors
A Single System
An Existing Single System
A System of Systems
•
•
•
•
•
•
A clean sheet of paper
Greenfield
Considerable freedom in
development
• NDI
• Architecture
• Process choices
•
•
Enhancement,
Maintenance, Perfection
Brownfield or continuation
of previously Greenfield
Major of minor
modification after first
delivery
•
Multidisciplinary
Constituent-systems are
operationally independent
Diseconomy of scale
9
University of Southern California
Center for Systems and Software Engineering
Top 10 Inhibitors – Similarities
A New Single System
An Existing Single System
A System of Systems
Requirements Volatility
Requirements Volatility
Lack of Interoperability
Unprecedentedness
High numbers of external interfaces
Lack of / incompatible standard &
protocol
Delayed authority to proceed/start with fixed
Unprecedentedness
milestone
Requirements Volatility
Infeasible schedule/staffing profile
Vague Requirements
Unprecedentedness
Lack of Domain Experience
Embedded poor quality software
High numbers of external interfaces
Technology Volatility
Conflicting Stakeholders
Infeasible schedule/staffing profile
High numbers of external interfaces
Delayed authority to proceed/start with
fixed milestone
Inability to test across systems
Vague Requirements
Infeasible schedule/staffing profile
Delayed authority to proceed/start with
fixed milestone
Under average people / Personnel
Capability
Technical debt
Lack of Domain Experience
Technology Immaturity
Interoperability / compatibility
Technology Volatility
10
University of Southern California
Center for Systems and Software Engineering
Top 10 Inhibitors – Differences
A New Single System
An Existing Single System
A System of Systems
Requirements Volatility
Requirements Volatility
Lack of Interoperability
Unprecedentedness
High numbers of external interfaces
Lack of / incompatible standard &
protocol
Delayed authority to proceed/start with fixed
Unprecedentedness
milestone
Requirements Volatility
Infeasible schedule/staffing profile
Vague Requirements
Unprecedentedness
Lack of Domain Experience
Embedded poor quality software
High numbers of external interfaces
Technology Volatility
Conflicting Stakeholders
Infeasible schedule/staffing profile
High numbers of external interfaces
Delayed authority to proceed/start with
fixed milestone
Inability to test across systems
Vague Requirements
Infeasible schedule/staffing profile
Delayed authority to proceed/start with
fixed milestone
Under average people / Personnel
Capability
Technical debt
Lack of Domain Experience
Technology Immaturity
Interoperability / compatibility
Technology Volatility
11
University of Southern California
Center for Systems and Software Engineering
Top 10 Enablers - Similarities
A New Single System
An Existing Single System
A System of Systems
Rapid Prototyping
Target hardware lab / test like you fly &
simulation
Customer /tech requirements flexibility
Target hardware lab / test like you fly &
simulation
Incremental test and feedback
Rapid Prototyping
Customer /tech requirements flexibility
Incremental Delivery & feedback
Target hardware lab / test like you fly &
simulation
Incremental test and feedback
Flexible / Tailorable rules
Incremental test and feedback
Incremental Delivery & feedback
Agile/lean approach
Common standard and protocol
Decision making authority
Rapid Prototyping
Reusing assets
Best people / personnel capability
Common standard, interface
Tools and automation
Agile/lean approach
Customer /tech requirements flexibility
Common standard, interface
Less context switching when doing
multiple projects
Domain knowledge
Best people / Personnel Capability
Tools and automation
Understanding of the existing system
and interfaces
Team cohesion
12
University of Southern California
Center for Systems and Software Engineering
Top 10 Enablers - Differences
A New Single System
An Existing Single System
A System of Systems
Rapid Prototyping
Target hardware lab / test like you fly &
simulation
Customer /tech requirements flexibility
Target hardware lab / test like you fly &
simulation
Incremental test and feedback
Rapid Prototyping
Customer /tech requirements flexibility
Incremental Delivery & feedback
Target hardware lab / test like you fly &
simulation
Incremental test and feedback
Flexible / Tailorable rules
Incremental test and feedback
Incremental Delivery & feedback
Agile/lean approach
Common standard and protocol
Decision making authority
Rapid Prototyping
Reusing assets
Best people / personnel capability
Common standard, interface
Tools and automation
Agile/lean approach
Customer /tech requirements flexibility
Common standard, interface
Less context switching when doing
multiple projects
Domain knowledge
Best people / Personnel Capability
Tools and automation
Understanding of the existing system
and interfaces
Team cohesion
13
University of Southern California
Center for Systems and Software Engineering
Observations
Flip side of a coin
Enablers
Inhibitors
Best people
Under average people
Decision making authority
Lack of decision making authority
Team Cohesion
Conflicting stakeholders
Incremental Test and Feedback
Inability to test across systems
14
University of Southern California
Center for Systems and Software Engineering
Observations
Grey Area
• Common standard
• Flexible rules
• Requirements Volatility
• Requirements Flexibility
• Agile/lean approach
• Constituent Documentation
15
University of Southern California
Center for Systems and Software Engineering
Observations
Overlap & Similar but not the same
• Lack of Domain Experience
• Unprecedentedness
• Technology Immaturity
• Technology Volatility
• Lack of decision making at lower levels
• Lack of decision making authority
16
University of Southern California
Center for Systems and Software Engineering
Next Steps
• Consolidate list of enablers and inhibitors
– Considerable overlap
• Continue
– Identification of enablers and inhibitors
– Solicit additional evaluations of enablers and inhibitors
• Evaluate related technical debt issues
• Map enablers and inhibitors to cost model
parameters
• Refine cost models using
– Enablers/inhibitors
– Technical debt information
University of Southern California
Center for Systems and Software Engineering
Backup Charts
University of Southern California
Center for Systems and Software Engineering
Conclusions, Recommendations, and Results
Single System
New
Number
Identified
Estimated Impact Levels (5 assessments)
High
Medium
Low
Enablers
26
37
55
35
Inhibitors
29
29
70
42
Single System
Existing
Number
Identified
Estimated Impact Levels (5 assessments)
High
Medium
Low
Enablers
28
30
54
48
Inhibitors
35
32
89
46
SoS
Number
Identified
Estimated Impact Levels (5 assessments)
High
Medium
Low
Enablers
30
38
61
43
Inhibitors
35
49
84
37
Scope of enablers and inhibitors: People, process, tools, product
University of Southern California
Center for Systems and Software Engineering
A New Single System
Rank
Enablers
1
Rapid Prototyping
2
Target hardware lab (experimentation / test lab) /
test like you fly & simulation
3
Customer /tech requirements flexibility
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Incremental test and feedback
Incremental Delivery & feedback
Decision making authority
Best people / personnel capability
Agile/lean approach
Less context switching when doing multiple
projects
Tools and automation
Common standard, interface
Flexible / tailorable rules
Model-based engineering
Building the common architecture/foundation
Reusing assets
Risk Management
Team cohesion
Business process reengineering / process
streamlining
COTS
Overnight build
Inhibitors
Requirements Volatility
Unprecedentedness
Delayed authority to proceed/start with fixed
milestone
Infeasible schedule/staffing profile
Lack of Domain Experience
Technology Volatility
High numbers of external interfaces
Vague Requirements
Under average people / Personnel Capability
Technology Immaturity
Conflicting Stakeholders
Lack of decision making authority
large number of subcontractors / stakeholders
Lack of development infrastructure
Rules and Regulations
Sequential Development
Classification / sensitivity
Fear to protest the contract award resulting in
poor requirements
Personnel Turnover
Bad RFP
20
University of Southern California
Center for Systems and Software Engineering
An Existing Single System
Rank
1
2
3
4
5
6
7
8
9
10
Enablers
Target hardware lab (experimentation / test lab)
Incremental test and feedback
Incremental Delivery & feedback
Flexible / tailorable rules
Agile/lean approach
Rapid Prototyping
Common standard, interface
13
Customer /tech requirements flexibility
Domain knowledge
Understanding of the existing system and
interfaces
Best people / personnel capability
Less context switching when doing multiple
projects
Tools and automation
14
15
16
17
18
19
Reusing assets
Team cohesion
Feasibility Evidence, Milestone review
Model-based engineering
Colocation of hw & sw engineers
Development process tailoring/adjustment
20
Risk Management
11
12
Inhibitors
Requirements Volatility
High numbers of external interfaces
Unprecedentedness
Vague Requirements
Embedded poor quality software
Conflicting Stakeholders
Delayed authority to proceed/start with fixed
milestone
Infeasible schedule/staffing profile
Technical debt
Interoperability / compatibility
large number of subcontractors / stakeholders
Backpropagation
Lack of understanding of the existing system and
interfaces
Under average people / Personnel Capability
Lack of Domain Experience
Technology Volatility
Technology Immaturity
Classification / sensitivity
Multiple operational sites with different configuration
/ platform / OS
21
Outdated / stovepipe technology
University of Southern California
Center for Systems and Software Engineering
A System of Systems
Rank
1
2
3
4
5
6
7
8
Enablers
Customer /tech requirements flexibility
Rapid Prototyping
Target hardware lab (experimentation / test lab) /
test like you fly & simulation
Incremental test and feedback
Common standard and protocol
Reusing assets
Tools and automation
Common standard, interface
9
10
11
12
13
14
15
16
17
18
19
20
Best people / Personnel Capability
Team cohesion
Synchronization and Stabilization
flexible / tailorable rules
Model-based engineering
Building the common architecture/foundation
Agile/lean approach
Incremental Delivery & feedback
COTS
Constituent Documentation
Decision making authority
Development process tailoring/adjustment
Inhibitors
Lack of Interoperability
Lack of / incompatible standard & protocol
Requirements Volatility
Unprecedentedness
High numbers of external interfaces
Infeasible schedule/staffing profile
Inability to test across systems
Delayed authority to proceed/start with fixed
milestone
Lack of Domain Experience
Technology Volatility
Technology Immaturity
Vague Requirements
Large number of subcontractors / stakeholders
Lack of development infrastructure
Lack of communication between teams
Classification / sensitivity
Poor / unknown heritage/pedigree
Bad RFP
Personnel Turnover
Under average people / Personnel Capability
22
University of Southern California
Center for Systems and Software Engineering
Recent Research Overview
Capability Options
New system or system of system(s)
New procedures for using existing systems
Changes to existing system or SoS
- Some robust, well integrated
- Others very fragile, close to end of life
- Which to invest in/which to retire
Existing vs. new technologies
How much, how fast, how accurate, etc. is
enough
Key Approaches for Incorporating Flexibility
Employ open architectures
Design for reuse
Develop/use product lines
Standard interfaces, protocols, services, data
Options-based design
Incremental commitment
Key Approaches for Expedited Engineering
Commercial-Off-the-Shelf (COTS) Products
Investment in product-line architectures
Reuse of existing systems/components
Repurposing existing systems/components Using the
right
Value-stream focus (lean)
people
Going fast in general (crisis response)
Single purpose architecture
Common Causes of Technical Debt
Pressures to compress schedule
Lack of requirements understanding
Lack of system understanding
Inflexible architectures/software
Overly complex design/implementation
Delayed defect resolution
Inadequate testing
Lack of current documentation
Parallel development in isolation
Delayed refactoring
Can be extended to incorporate other “-ilities”…
23
University of Southern California
Center for Systems and Software Engineering
Single System Development
Perspective
Choices driven by potential
–
–
–
–
Market share
Future opportunities
Technical debt
Cost of failure to provide needed
capability
24