University of Southern California
Center for Systems and Software Engineering
Modeling System and Software Engineering
Processes with System Dynamics
Ray Madachy
USC Center for Systems and Software Engineering
madachy@usc.edu
Annual Research Review
March 17, 2008
3/16/08
©USC-CSSE
1
University of Southern California
Center for Systems and Software Engineering
Outline
• Background and Introduction
• System Dynamics Examples
– Optimizing quality assurance, defect dynamics
– Brooks’s Law model
– Process model for very large SISOS
• References
• Detailed Backups
–
–
–
–
3/16/08
Inspection model
Value-based business and software process/product model
Spiral hybrid process model for SISOS
Process concurrence
©USC-CSSE
2
University of Southern California
Center for Systems and Software Engineering
Background
• The evaluation of process strategies for the
architecting and engineering of complex systems
involves many interrelated factors.
• Effective systems and software engineering
requires a balanced view of technology, business
or mission goals, and people.
• System dynamics is a rich and integrative
simulation framework used to quantify the
complex interactions and the strategy tradeoffs
between cost, schedule, quality and risk.
3/16/08
©USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
System Dynamics Principles
• Major concepts
–
–
–
–
–
–
–
Defining problems dynamically, in terms of graphs over time
Striving for an endogenous, behavioral view of the significant dynamics of a
system
Thinking of all real systems concepts as continuous quantities interconnected in
information feedback loops and circular causality
Identifying independent levels in the system and their inflow and outflow rates
Formulating a model capable of reproducing the dynamic problem of concern by
itself
Deriving understandings and applicable policy insights from the resulting model
Implementing changes resulting from model-based understandings and insights.
• The continuous view
– Individual events are not tracked
– Entities are treated as aggregate quantities that flow through a system
3/16/08
©USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
System Dynamics Notation
• System represented by x’(t)= f(x,p).
• x: vector of levels (state variables), p: set of parameters
• Legend:
level
source/sink rate
information link
auxiliary variable
defects
• Example system:
defect generation
rate
undetected defects
defect escape
rate
defect detection rate
defect detection
efficiency
detected defects
3/16/08
©USC-CSSE
5
University of Southern California
Center for Systems and Software Engineering
Model Elements
Element
level
rate
level
information link
Description
Process Instances
An accumulation over time, also
called a stock or state variable. A
storage device for material, energy,
information.
Flows; the “actions” in a system that
often represent policies. They exist
with and effect the changes in levels.
source/
rate
sink
iary variable
information link
level
source/
auxiliary variable
rate
sink
Represent infinite supplies or
repositories, indicating real-world
accumulations outside boundary of
the system being modeled
information link
3/16/08
©USC-CSSE

work artifacts (requirements, tasks,
lines of code, documentation pages)
 defect levels
 personnel levels
 effort expenditure
 revenue
 software productivity rate
 defect generation rate
 personnel hiring and de-allocation
 learning rate
 burn rate
 source of requirements
 software delivered to customers
 employee hiring sources and attrition
sinks
6
University of Southern California
Center for Systems and Software Engineering
level
Model Elements (continued)
source/
rate
sink
Element
Description
information link
level
auxiliary variable
/
Process Instances
Converters of input to output that
elaborate detail of stock and flow
structure. They often represent
“score-keeping” variables.
Information linkages.
rate
 quantitative goals or planned values
 constants like average delay times
 percent of job completion
 defect density
 progress and status information for
decision making
 linking process parameters to other
information link
vaariables
uxiliary variable
3/16/08
©USC-CSSE
7
University of Southern California
Center for Systems and Software Engineering
Modeling Criteria Overview
Process V&V
Does not catch specific defects, but can model the aggregate effects of processes on defect
rates/levels. Modeling the effects on n different types of defects involves scaling up a model
multiplicatively by n model elements (flow chains, rates, levels and parameters).
• E.g. Dynamic ODC COQUALMO has defect chains replicated for 10 types (see slides later)
Ambiguity Tolerance
No tolerance for ambiguity in flow chain representation, but parameters can be stochastic.
Scalability
The size of process is unlimited. As the size/complexity grows, the technique can cut the level
of entity aggregation higher. No overhead for increasing # of entities at a chosen aggregation.
• E.g. Extremely large SISOS processes we have modeled at the capability level (see slides later)
Extensibility
Requires little to moderate training in simulation to extend existing models.
Coverage
Well suited to estimate high-level budgets, schedules, and quality levels.
Uniquely suited to monitor progress with respect to plans interactively and/or continuously.
• E.g. Earned Value model and others in Software Process Dynamics can generate time-phased
plans, track them against actuals, and recalibrate new plans based on actuals
No independent composition of documents or process elements.
Dynamism
Perfectly suited to model instant or time-phased changes to processes.
• E.g. Brooks’s Law model of adding people to a late project (see slides later)
• E.g. Dynamic ODC COQUALMO model of quality process changes on defect introduction and
removal rates (see slides later)
Distribution
3/16/08
Widely supported mature commercial modeling tools, but no vendors of software process
©USC-CSSE
8
models.
University of Southern California
Center for Systems and Software Engineering
Additional Strengths / Weaknesses
•
Strengths
– Provides a global system perspective and the ability to analyze combined strategies
– Analysis of integrated process factors and feedback loops (e.g. holistic enterprise
models, positive and negative feedback, process delays, process concurrence)
– Finding the right process balance (e. g., quality assurance levels, staffing levels)
– Adapting to change; strategic long-term policy analysis
– Can model inherent tradeoffs between schedule, cost and quality
– Attractive for schedule analysis accounting for critical path flows, task
interdependencies and bottlenecks not available with static models or PERT/CPM
methods
•
Weaknesses
– Assumption of homogeneity of entities; cannot attach attributes to individual
entities
– Sequential activities difficult to represent
– Queuing and other discrete statistics not captured
3/16/08
©USC-CSSE
9
University of Southern California
Center for Systems and Software Engineering
Outline
• Background and Introduction
• System Dynamics Examples
– Optimizing quality assurance, defect dynamics
– Brooks’s Law model
– Process model for very large SISOS
• References
• Detailed Backups
–
–
–
–
3/16/08
Inspection model
Value-based business and software process/product model
Spiral hybrid process model for SISOS
Process concurrence
©USC-CSSE
10
University of Southern California
Center for Systems and Software Engineering
QA Policy Tradeoff Analysis
2500
re w o rk a nd te s ting c o s t
Cost (person-days)
Cost (person-days)
3000
2000
1500
1000
QA cost
500
0
0
10
20
30
40
50
5500
o v e ra ll pro je c t c o s t
5000
4500
4000
3500
3000
0
Q A e ffo rt % o f to ta l
10
20
30
40
50
Q A e ffo rt % o f to ta l
From [Abdel-Hamid/Madnick 91]
3/16/08
©USC-CSSE
11
University of Southern California
Center for Systems and Software Engineering
Dynamic ODC COQUALMO Portion
•
Portion for Requirements Completeness defects only:
3/16/08
©USC-CSSE
12
University of Southern California
Center for Systems and Software Engineering
Dynamic ODC COQUALMO
Sample Outputs
•
Example of applying increased V&V for Execution Testing and Tools at 18 months:
3/16/08
©USC-CSSE
13
University of Southern California
Center for Systems and Software Engineering
Brooks’s Law Model
•
•
“Adding manpower to a late software
project makes it later” [Brooks 75].
A simple model based on the
following assumptions:
– New personnel require training by
experienced personnel to come up
to speed
– More people on a project entail
more communication overhead
– Experienced personnel are more
productive then new personnel, on
average.
3/16/08
©USC-CSSE
14
University of Southern California
Center for Systems and Software Engineering
Brooks’s Law Model Output
Function points/day
Days
Sensitivity of Software Development Rate to Varying Personnel Allocation Pulses
(1: no extra hiring, 2: add 5 people on 100th day, 3: add 10 people on 100th day)
3/16/08
©USC-CSSE
15
University of Southern California
Center for Systems and Software Engineering
•
Large SISOS Model Overview
Built around a cyclic flow chain for
capabilities
–
•
•
•
Each team is represented with a level and
corresponding staff allocation rate
Changes arrive a-periodically via the
volatility trends time function and flow
into the level for capability changes
Changes are processed by the agile team
and allocated to increments per the
deferral policies
–
•
Constant or variable staffing for the agile team
For each increment the required
capabilities are developed into developed
capabilities and then V&V’ed into V &
V’ed capabilities
–
–
•
Arrayed for multiple increments
Productivities and team sizes for development
and V&V calculated with a Dynamic
COCOMO variant and continuously updated
for scope changes
Flow rates between capability changes and V
& V’ed capabilities are bi-directional for
capability “kickbacks” sent back up the chain
User-driven changes from the field are
identified as field issues that flow back
into the capability changes
3/16/08
©USC-CSSE
16
University of Southern California
Center for Systems and Software Engineering
Sample Response to Volatility
•
•
•
•
An unanticipated change occurs at month 8 shown as a volatility trend [1] pulse
It flows into capability changes [1] which declines to zero as the agile team processes the change
The change is non-deferrable for increment 1 so total capabilities [1] increases
Development team staff size dynamically responds to the increased scope
1: volatility trend s[1]
1:
2:
3:
4:
2: ca pabi lity chan ges[1]
3: devel opme nt team
4
1
40
20
4: total capa bili ties [1]
3
3
3
3
1:
2:
3:
4:
2
1
20
15
1:
2:
3:
4:
0
0
0
10
4
4
4
4
1
0.00
2
1
6.00
2
1
12 .00
2
1
18 .00
2
24 .00
Months
In crem ent 1
* [1] refers to increment #1
3/16/08
©USC-CSSE
17
University of Southern California
Center for Systems and Software Engineering
Outline
• Background and Introduction
• System Dynamics Examples
– Optimizing quality assurance, defect dynamics
– Brooks’s Law model
– Process model for very large SISOS
• References
• Detailed Backups
–
–
–
–
3/16/08
Inspection model
Value-based business and software process/product model
Spiral hybrid process model for SISOS
Process concurrence
©USC-CSSE
18
University of Southern California
Center for Systems and Software Engineering
References
•
•
•
•
•
•
Abdel-Hamid T, Madnick S, Software Project Dynamics, Englewood Cliffs, NJ,
Prentice-Hall, 1991
Kellner M, Madachy R, Raffo D, Software Process Simulation Modeling: Why? What?
How?, Journal of Systems and Software, Spring 1999
Madachy R, System Dynamics Modeling of an Inspection-Based Process, Proceedings of
the Eighteenth International Conference on Software Engineering, IEEE Computer
Society Press, Berlin, Germany, March 1996
Madachy R, Boehm B, Lane J, Spiral Lifecycle Increment Modeling for New Hybrid
Processes, Journal of Systems and Software, 2007
Madachy R., Software Process Dynamics, Wiley-IEEE Computer Society, 2008
Madachy R., Boehm B., Assessing Quality Processes with ODC COQUALMO,
Proceedings of the 2008 International Conference on Software Process, Liepzig,
Germany, 2008
USC Web Sites
• http://csse.usc.edu/softwareprocessdynamics
• http://rcf.usc.edu/~madachy/spd
3/16/08
©USC-CSSE
19
University of Southern California
Center for Systems and Software Engineering
Outline
• Background and Introduction
• System Dynamics Examples
– Optimizing quality assurance, defect dynamics
– Brooks’s Law model
– Process model for very large SISOS
• References
• Detailed Backups
–
–
–
–
3/16/08
Inspection model
Value-based business and software process/product model
Spiral hybrid process model for SISOS
Process concurrence
©USC-CSSE
20
University of Southern California
Center for Systems and Software Engineering
Inspection Model Example
• Research problem addressed
– What are the dynamic effects to the process of performing inspections?
• Model used to evaluate process quantitatively
– Demonstrates effects of inspection practices on cost, schedule and quality
throughout lifecycle
– Can experiment with changed processes before committing project
resources
– Benchmark process improvement
– Support project planning and management
• Model parameters calibrated to Litton and JPL data
– Error generation rates, inspection effort, efficiency, productivity, others
• Model validated against industrial data
3/16/08
©USC-CSSE
21
University of Southern California
Center for Systems and Software Engineering
System Diagram
3/16/08
©USC-CSSE
22
University of Southern California
Center for Systems and Software Engineering
System Diagram (continued)
3/16/08
©USC-CSSE
23
University of Southern California
Center for Systems and Software Engineering
Inspection Policy Tradeoff Analysis
• Varying error generation rates shows diminishing returns from
inspections [Madachy 94]:
Total Effort (Person-days)
5500
5000
w ith inspections
w ithout inspections
4500
4000
3500
3000
10
20
30
40
50
60
70
80
Defects/KSLOC
3/16/08
©USC-CSSE
24
University of Southern California
Center for Systems and Software Engineering
Value-Based Model Background
• Purpose: Support software business decisionmaking by experimenting with product strategies
and development practices to assess real earned
value
• Description: System dynamics model relates the
interactions between product specifications and
investments, software processes including quality
practices, market share, license retention, pricing
and revenue generation for a commercial software
enterprise
3/16/08
©USC-CSSE
25
University of Southern California
Center for Systems and Software Engineering
Model Features
• A Value-Based Software Engineering (VBSE) model covering the
following VBSE elements:
–
–
–
Stakeholders’ value proposition elicitation and reconciliation
Business case analysis
Value-based monitoring and control
• Integrated modeling of business value, software products and
processes to help make difficult tradeoffs between perspectives
–
Value-based production functions used to relate different attributes
• Addresses the planning and control aspect of VBSE to manage the
value delivered to stakeholders
–
–
Experiment with different strategies and track financial measures over time
Allows easy investigation of different strategy combinations
• Can be used dynamically before or during a project
–
–
3/16/08
User inputs and model factors can vary over the project duration as opposed to a static model
Suitable for actual project usage or “flight simulation” training where simulations are
interrupted to make midstream decisions
©USC-CSSE
26
University of Southern California
Center for Systems and Software Engineering
Model Sectors and Major Interfaces
• Software process and
product sector computes
the staffing and quality
over time
• Market and sales sector
accounts for market
dynamics including effect
of quality reputation
• Finance sector computes
financial measures from
investments and revenues
3/16/08
Product Specifications
Software
Process and
Product
©USC-CSSE
Product Quality
Market and
Sales
Sales Revenue
Staffing Rate
Finances
27
University of Southern California
Center for Systems and Software Engineering
Software Process and Product
cumulative effort
effort and schedule calculation
with dynamic COCOMO variant
staffing rate
learning function
start staff
estimated total effort
manpower buildup parameter
~
Function Points
effort multiplier
~
Reliability Setting
defect density
defects
product defect flows
~
defect generation rate
3/16/08
actual quality
©USC-CSSE
defect removal rate
28
University of Southern California
Center for Systems and Software Engineering
Finances, Market and Sales
investment and revenue flows
software license sales
market share dynamics
including quality reputation
3/16/08
©USC-CSSE
29
University of Southern California
Center for Systems and Software Engineering
Quality Assumptions
• COCOMO cost driver Required Software Reliability is a proxy for all quality
practices
• Resulting quality will modulate the actual sales relative to the highest potential
• Perception of quality in the market matters
– Quality reputation quickly lost and takes much longer to regain (bad news travels fast)
– Modeled as asymmetrical information smoothing via negative feedback loop
3/16/08
©USC-CSSE
30
University of Southern California
Center for Systems and Software Engineering
Market Share Production Function
and Feature Sets
Features with
Diminishing Returns
Added Market Share Percent
25%
20%
Reference Case
(700 Function Points)
Cases from Example 1
15%
Case 1 and Case 2
(550 Function Points)
High Payoff
Features
10%
5%
Core
Features
0%
0
2
4
6
8
Cost ($M)
3/16/08
©USC-CSSE
31
University of Southern California
Center for Systems and Software Engineering
Sales Production Function and Reliability
100%
Very High
High
Percent of Potential Sales
90%
Required Reliability
Settings
80%
70%
Nominal
60%
Reference Case
and Case 1
50%
Cases from Example 1
40%
Case 2
Low
30%
0.9
3/16/08
1
1.1
1.2
Relative Effort to Achieve Reliability
©USC-CSSE
1.3
32
University of Southern California
Center for Systems and Software Engineering
Example 1: Dynamically Changing
Scope and Reliability
• Shows how model can assess the effects of combined
strategies by varying the scope and required reliability
independently or simultaneously
• Simulates midstream descoping, a frequent strategy to
meet time constraints by shedding features
• Three cases are demonstrated:
– Unperturbed reference case
– Midstream descoping of the reference case after ½ year
– Simultaneous midstream descoping and lowered required
reliability at ½ year
3/16/08
©USC-CSSE
33
University of Southern California
Center for Systems and Software Engineering
Control Panel and Simulation Results
Descope
1: staffing rate
1:
2:
3:
15
35
3
2: market share
3: ROI
Case 1
1
1
2
2
3
2
1:
2:
3:
8
23
1
3
2
2
1:
2:
3:
3
3
3
0
10
-1
0.00
1
2.00
1.00
1
3.00
Page 1
Unperturbed Reference Case
5.00
Years
Descope +
Lower Reliability
1: sta ffin g rate
1:
2:
3:
1
4.00
2: market share
3: ROI
Case 2
15
35
3
1
1
1:
2:
3:
8
23
1
3
3
2
2
3
2
3
3
1:
2:
3:
0.00
Pag e 1
3/16/08
©USC-CSSE
2
0
10
-1
1.00
1
2.00
1
3.00
2
1
4.00
Ye ars
34
5.00
University of Southern California
Center for Systems and Software Engineering
Case Summaries
Case
Delivered
Size
(Function
Points)
Delivered
Reliability
Setting
Cost
($M)
Delivery
Time
(Years)
Final
Market
Share
ROI
Reference Case:
Unperturbed
700
1.0
4.78
2.1
28%
1.3
Case 1:
Descope
at Time = ½ years
550
1.0
3.70
1.7
28%
2.2
Case 2:
Descope and
Lower Reliability
at Time = ½ years
550
.92
3.30
1.5
12%
1.0
3/16/08
©USC-CSSE
35
University of Southern California
Center for Systems and Software Engineering
Example 2: Determining the Reliability
Sweet Spot
• Analysis process
– Vary reliability across runs
– Use risk exposure framework to find process optimum
– Assess risk consequences of opposing trends: market delays and bad
quality losses
– Sum market losses and development costs
– Calculate resulting net revenue
• Simulation parameters
– A new 80 KSLOC product release can potentially increase market share
by 15%-30% (varied in model runs)
– 75% schedule acceleration
– Initial total market size = $64M annual revenue
• Vendor has 15% of market
• Overall market doubles in 5 years
3/16/08
©USC-CSSE
36
University of Southern California
Center for Systems and Software Engineering
Cost Components
$35
development cost
market delay loss
bad quality loss
total cost
Cost (Millions)
$30
$25
3-year time horizon
$20
$15
$10
$5
$0
Low
Nominal
High
Very High
Software Reliability
3/16/08
©USC-CSSE
37
University of Southern California
Center for Systems and Software Engineering
Value-Based Model Conclusions
• To achieve real earned value, business value attainment must be a
key consideration when designing software products and processes
• Software enterprise decision-making can improve with information
from simulation models that integrate business and technical
perspectives
• Optimal policies operate within a multi-attribute decision space
including various stakeholder value functions, opposing market
factors and business constraints
• Risk exposure is a convenient framework for software decision
analysis
• Commercial process sweet spots with respect to reliability are a
balance between market delay losses and quality losses
• Model demonstrates a stakeholder value chain whereby the value
of software to end-users ultimately translates into value for the
software development organization
3/16/08
©USC-CSSE
38
University of Southern California
Center for Systems and Software Engineering
Value-Based Model Future Work
• Enhance product defect model with dynamic version of COQUALMO
to enable more constructive insight into quality practices
• Add maintenance and operational support activities in the workflows
• Elaborate market and sales for other considerations including pricing
scheme impacts, varying market assumptions and periodic upgrades of
varying quality
• Account for feedback loops to generate product specifications (closedloop control)
– External feedback from users to incorporate new features
– Internal feedback on product initiatives from organizational planning and control
entity to the software process
• More empirical data on attribute relationships in the model will help
identify areas of improvement
• Assessment of overall dynamics includes more collection and analysis
of field data on business value and quality measures from actual
software product rollouts
3/16/08
©USC-CSSE
39
University of Southern California
Center for Systems and Software Engineering
Outline
• Research introduction
• Value-based business and software
process/product model
• Spiral hybrid process model for SoftwareIntensive System of Systems (SISOS)
• References
• Backup slides
3/16/08
©USC-CSSE
40
University of Southern California
Center for Systems and Software Engineering
Spiral Hybrid Process Introduction
•
•
•
The spiral lifecycle is being extended to address new challenges for SoftwareIntensive Systems of Systems (SISOS), such as coping with rapid change
while simultaneously assuring high dependability
A hybrid plan-driven and agile process has been outlined to address these
conflicting challenges with the need to rapidly field incremental capabilities
A system-of-systems (SOS) integrates multiple independently-developed
systems and is very large, dynamically evolving, unprecedented, with
emergent requirements and behaviors
– However, traditional static approaches cannot capture dynamic feedback loops and
interacting phenomena that cause real-world complexity (e.g. hybrid processes,
project volatility, increment overlap and resource contention, schedule pressure,
slippages, communication overhead, slack, etc.)
•
•
A system dynamics model is being developed to assess the incremental hybrid
process and support project decision-making
Both the hybrid process and simulation model are being evolved on a very
large scale incremental project for a SISOS
3/16/08
©USC-CSSE
41
University of Southern California
Center for Systems and Software Engineering
Scalable Spiral Model Increment Activities
•
•
•
•
Organize development
into plan-driven
increments with stable
specs
Agile team watches for
and assesses changes,
then negotiates changes
so next increment hits
the ground running
Try to prevent usage
feedback from
destabilizing current
increment
Three team cycle plays
out from one increment
to the next
3/16/08
Unforseeable Change
(Adapt)
Rapid
Change
Short
Development
Increments
Agile
Future Increment Baselines
Rebaselining for
Future Increments
Deferrals
Foreseeable
Change (Plan)
Increment N Baseline
Stable Development
Increments
Current V&V
High
Assurance
Resources
Short, Stabilized
Development
of Increment N
Artifacts
Increment N Transition/O&M
Concerns
V&V
of Increment N
Future V&V
Resources
Continuous V&V
©USC-CSSE
42
University of Southern California
Center for Systems and Software Engineering
Spiral Hybrid Model Features
• Estimates cost and schedule for multiple
increments of a hybrid process that uses three
specialized teams (agile re-baseliners, developers,
V&V’ers) per the scalable spiral model
• Considers changes due to external volatility and
feedback from user-driven change requests
• Deferral policies and team sizes can be
experimented with
• Includes tradeoffs between cost and the timing of
changes within and across increments, length of
deferral delays, and others
3/16/08
©USC-CSSE
43
University of Southern California
Center for Systems and Software Engineering
Model Input Control Panel
3/16/08
©USC-CSSE
44
University of Southern California
Center for Systems and Software Engineering
Model Overview
•
Built around a cyclic flow chain for
capabilities
–
•
•
•
Each team is represented with a level and
corresponding staff allocation rate
Changes arrive a-periodically via the
volatility trends time function and flow
into the level for capability changes
Changes are processed by the agile team
and allocated to increments per the
deferral policies
–
•
Constant or variable staffing for the agile team
For each increment the required
capabilities are developed into developed
capabilities and then V&V’ed into V &
V’ed capabilities
–
–
•
Arrayed for multiple increments
Productivities and team sizes for development
and V&V calculated with a Dynamic
COCOMO variant and continuously updated
for scope changes
Flow rates between capability changes and V
& V’ed capabilities are bi-directional for
capability “kickbacks” sent back up the chain
User-driven changes from the field are
identified as field issues that flow back
into the capability changes
3/16/08
©USC-CSSE
45
University of Southern California
Center for Systems and Software Engineering
Volatility Cost Functions
Volatility Effort Multiplier
•
•
Lifecycle Timing Effort
Multiplier
The volatility effort multiplier for construction effort and schedule is an aggregate
multiplier for volatility from different sources (e.g. COTS, mission, etc.) relative to the
original baseline for increment
The lifecycle timing effort multiplier models increased development cost the later a
change comes in midstream during an increment
3/16/08
©USC-CSSE
46
University of Southern California
Center for Systems and Software Engineering
Sample Response to Volatility
•
•
•
•
An unanticipated change occurs at month 8 shown as a volatility trend [1] pulse
It flows into capability changes [1] which declines to zero as the agile team processes the change
The change is non-deferrable for increment 1 so total capabilities [1] increases
Development team staff size dynamically responds to the increased scope
1: volatility trend s[1]
1:
2:
3:
4:
2: ca pabi lity chan ges[1]
3: devel opme nt team
4
1
40
20
4: total capa bili ties [1]
3
3
3
3
1:
2:
3:
4:
2
1
20
15
1:
2:
3:
4:
0
0
0
10
4
4
4
4
1
0.00
2
1
6.00
2
1
12 .00
2
1
18 .00
2
24 .00
Months
In crem ent 1
* [1] refers to increment #1
3/16/08
©USC-CSSE
47
University of Southern California
Center for Systems and Software Engineering
Sample Test Results
•
•
•
•
Test case for two increments of 15 baseline capabilities each
A non-deferrable change comes at month 8 (per previous slide)
The agile team size is varied from 2 to 10 people
Increment 1 business value also lost for agile team sizes of 2 and 4
Total Effort (PM)
3500
4000
Agile Effort (PM)
3500
Effort (PM)
2000
1500
2500
500
500
0
0
8
10
65
60
2
4
6
8
10
# Agile People
# Agile People
3/16/08
70
1500
1000
6
75
2000
1000
4
85
3000
2500
2
Total Schedule (Mths.)
80
Dev+V&V Effort (1) (PM)
3000
Effort (PM)
Dev+V&V (2) Effort (PM)
©USC-CSSE
48
Schedule (Mths
4000
University of Southern California
Center for Systems and Software Engineering
Spiral Hybrid Model
Conclusions and Future Work
•
•
•
System dynamics is a convenient modeling framework to deal with the complexities of a
SISOS
A hybrid process appears attractive to handle SISOS dynamic evolution, emergent
requirements and behaviors
Initial results indicate that having an agile team can help decrease overall cost and schedule
–
•
•
Will obtain more empirical data to calibrate and parameterize model including volatility and
change trends, change analysis effort, effort multipliers and field issue rates
Model improvements
–
–
–
–
•
3/16/08
Additional staffing options
• Rayleigh curve staffing profiles
• Constraints on development and V&V staffing levels
More flexible change deferral options across increments
Increment volatility balancing policies
Provisions to account for (timed) business/mission value of capabilities
Additional model experimentation
–
–
–
•
Model can help find the optimum balance
Include capabilities flowing back from developers and V&V’ers
Vary deferral policies and volatility patterns across increments
Compare different agile team staffing policies
Continue applying the model on a current SISOS and seek other potential pilots
©USC-CSSE
49
University of Southern California
Center for Systems and Software Engineering
Process Concurrence Basics
• Process concurrence describes interdependency
constraints between tasks
– can be an internal constraint within a development stage or an
external constraint between stages
• Describes how much work becomes available for
completion based on previous work accomplished
• Accounts for realistic bottlenecks on work availability
– vs. a model driven solely by resources and productivity that can
finish in almost zero time with infinite resources
• Concurrence relations can be sequential, parallel, partially
concurrent, or other dependent relationships
3/16/08
©USC-CSSE
50
University of Southern California
Center for Systems and Software Engineering
Process Concurrence Suitability for
Integration of Processes
• Process concurrence models the constraints on making
work available between process activities.
• Provides a framework to characterize different approaches
in terms of their ability to parallelize or accelerate
activities
– Can be used to derive optimal staffing profiles for different project
situations
• Hence, it can model the integration and sequencing (in
aggregate) of systems engineering and software
engineering activities
– E.g. The following external concurrence model can optimize
staffing levels for given system architectural profiles
3/16/08
©USC-CSSE
51
University of Southern California
Center for Systems and Software Engineering
External Concurrence Model
the time profile of tasks
ready to elaborate ~
“ideal” staffing curve shape
3/16/08
©USC-CSSE
52
University of Southern California
Center for Systems and Software Engineering
Simulation Results and Sample Lessons
flat
Rayleigh
COTS pulse
at front
N/A
Critical customer decision delays
slow progress
- e.g. can’t design until timing
performance specs are known
Early stakeholder concurrence
enables RAD
- e.g. decision on architectural
framework or COTS package
3/16/08
©USC-CSSE
53