Revisiting cost of change
(and other things on my mind in 2009)
David J. Anderson
dja@djandersonassociates.com
USC CSSE Annual Research Review 2009
Lifecycle throughput & capacity
Example lifecycle with throughput capacity (approx) Function Points per unit of time (e.g. month)
Idea
Analysis
~90
~40
Error
Working
Code
Design
Acceptance
Test
~40
Code
~100
Error
System
Test
~70
~100
Error
Unit
Test
~100
In this example, the developers have 2.5x greater capacity than
the analysts. Analysis and customer acceptance (same resources)
is the bottleneck. System testing has 1.5x greater capacity than
the system throughput.
Development is seldom the bottleneck
• Teaching Agile Management class over 3 years
<10% of participants report development as
their lifecycle/workflow bottleneck
• Alistair Cockburn has pointed out that rework
in non-bottleneck stations is free and can be
used as a process strategy
Cockburn, Alistair, “Spending efficiency to go faster”, Humans & Technology Report,
March 2008, http://alistair.cockburn.us/%22Spending%22+Efficiency+to+Go+Faster
Cockburn, Alistair, “Two case studies motivating efficiency as a ‘spendable’ resource”,
ICAM 2005 International Conference on Agility, Helsinki July 27–28, 2005
http://alistair.cockburn.us/Two+Case+Studies+Motivating+Efficiency+as+a+%22Spenda
ble%22+Quantity
Possible Conclusions
• The cost of change is unique to each project
and value-stream and its resource allocation
• Research is required to discover the cost of
change / rework framework for calculating
cost of change in a specific workflow
• Question: How does throughput capacity
change over project lifetime? Does the
bottleneck move?
Development cycle time has a non-linear relationship to quality
CYCLE TIME RELATIONSHIP TO
QUALITY
WIP is directly related to Lead Time and Quality
Lead Time
Time
Inventory
Started
Designed
Coded
Complete
30
-M
ar
23
-M
ar
16
-M
ar
9M
ar
2M
ar
eb
24
-F
eb
WIP
17
-F
eb
240
220
200
180
160
140
120
100
80
60
40
20
0
10
-F
Features
Device Management Ike II Cumulative Flow
3 month lead time led to a >30x reduction in quality
compared to 1 week lead time
Project B Cumulative Flow
175
125
100
Lead Time
75
50
25
Time
Inventory
Started
Designed
Coded
Complete
11-Mar
26-Feb
12-Feb
29-Jan
15-Jan
1-Jan
18-Dec
4-Dec
20-Nov
6-Nov
23-Oct
0
9-Oct
Features
150
Defect insertion rate
What I believe?...
linear
~5-10 days
time
Research Opportunities
• Is there existing data that can be analyzed to
compare with this empirical result?
• Does the data correlate?
• Can guidance be derived from the shape of
the resultant curve?...
– Iteration lengths should not exceed 2 weeks
– WIP and multi-tasking should be limited to
manage cycle time below 10 (working) days
Real Options is Dead! Long live real options!
REAL OPTION THEORY
Real Options are not very useful
• Huchzermeier & Loch 1998 [INSEAD, Project
Management under Risk]
• 5 types of uncertainty
– Market payoff
– Quality
– Budget
– Requirements
– Schedule
Observations derived from
Huchzermeier & Loch 1998
• Market payoff & budget uncertainty increase
option valuations
• Quality uncertainty decreases option valuations.
=> Options are more valuable in high quality
conditions
• Requirements uncertainty reduces options value.
=> In uncertain requirements environments lots
of cheap/free options are desirable. With higher
certainty, fewer higher value options are optimal
• Option value under schedule uncertainty
depends on market payoff function
Conclusion
• Real Options valuation is only useful and
possible in low uncertainty projects
undertaken by high maturity (CMMI ML4/5
equiv.) organizations
• => Real Options are of no practical use in
software engineering applications
Real Options is Dead! Long live real options!
SALVAGING VALUE IN THE
WRECKAGE OF REAL OPTIONS
Abandon analogous mapping financial
options but leverage some concepts
•
•
•
•
•
•
Expiry Date
Liquidity
Optimal Exercise point?
Underlying asset?
Broker/market maker?
Valuation – determined as not useful
There is value in having options and making
optimal decisions under uncertainty
• Understand goals
• Create options
• Define “last responsible moment” (decision
point at option expiry date)
• Push back decision point as late as possible
• Maximize information gathering before
decision
Ref: Matts, Chris, “Last Responsible Moment”, Agile Journal, Jan 2009,
http://www.agilejournal.com/component/content/article/884-last-responsiblemoment-
Ref: Matts, Chris, “Last Responsible Moment”, Agile Journal, Jan 2009,
http://www.agilejournal.com/component/content/article/884-last-responsible-moment-
Liquidity
• Liquidity is ability to transfer assets from one
form to another – ability to exercise an option
• Liquidity is an aggregate of
– Temporal elements
– Work-in-progress (inventory)
– Personnel (resources)
• Constraint personnel liquidity is ability to learn
– (investment in) training will increase option
liquidity
Liquidity guidance
• Generalists provide options liquidity (in comparison to
specialists)
– Rule: assign a specialist to a suitable task, hold generalists
in reserve
• Resource availability affects liquidity
– Rule: assign a non-instant availability resource to nonurgent tasks, hold instant availability resources in reserve
• Cross-training, resource rotation generates option
liquidity
– E.g. Pair programming good, promiscuous pairing better
Ref: Belshee, Arlo, “Promiscuous Pairing and the Beginner’s Mind”, Proceedings of Agile
Development Conference 2005, IEEE Computer Society
Liquidity & Task Allocation
SkillAA
Task requiring skill
Skill B
Skill C
Skill D
Resource 1
(Generalist)
Resource 2
Resource 3
Resource 4
(Specialist)
Who should we allocate the task
in order to maintain option
liquidity?
If we assign a generalist…
Skill A
Skill B
Skill C
Skill D
Resource 1
(Generalist)
Resource 2
Resource 3
Liquidity 6
Resource 4
(Specialist)
3 Options
But if we assign a specialist…
Skill A
Skill B
Skill C
Skill D
Resource 1
(Generalist)
Resource 2
Resource 3
Resource 4
(Specialist)
Liquidity 9
4 Options
Some General Observations
•
•
•
•
•
•
Options liquidity is a key tool in project risk management
IID more liquidity than Waterfall (iterations)
Agile generally better than IID (generalist workforce)
2 week Sprint Scrum than 4
1 week iteration XP better than 2 weeks
Kanban/Pull system better than XP/Scrum (lower WIP, risk
based pull prioritization)
• Marginal utility supplied by specialists can be traded
against liquidity from generalists
• Efficiency (waste versus value-add) can be traded for
liquidity
• Slack provides liquidity => line should not be balanced (i.e.
Lean manufacturing “balance the line” reduces liquidity)
Ref: http://finance.groups.yahoo.com/group/real_options_discussion/message/369
Research questions?
• Can we measure/quantify liquidity?
• Can we match required liquidity level to risk
profile (and/or uncertainty/variability in
domain)?
• Can we provide (objective) guidance to
managers on how much liquidity they need
and how to maintain it?
Other research questions?
• Can we calculate an optimal exercise point
(decision point) without calculating a value (and
cost of exercising) for an option?
• Can this be useful if options are not free (or near
free) to acquire and easy/low risk to exercise?
• Can these techniques be applied to architecture
and design options and decisions
[Alternative Architecture Assessment, Decision
Analysis and Resolution (DAR)]?
Contact Details
David J. Anderson
dja@djandersonassociates.com
http://www.agilemanagement.net/
Tel: 206.201.2717
Skype: agilemanager
http://www.linkedin.com/in/agilemanagement