Cycle Time Mathematics
Presented at KLRAT 2013
Troy.magennis@focusedobjective.com
Conclusions
• Forecasting using cycle time is proving useful
• Cycle time follows a Weibull/Lognormal shape
– We can estimate the actual distribution with just a
minimum and a maximum guess (initially)
• Controlling the shape of the distribution
(narrowing) improves predictability
• Proposing a way of identifying what risks and
delays will have the greatest impact…
2
Commercial in confidence
Prediction Intervals
Q. What is the chance of the 4th sample
being between the range seen after the
first three samples?
Actual Maximum
(no duplicates, uniform distribution, picked at random)
2
3
1
Actual Minimum
4
Q. What is the chance of the 4th sample
being between the range seen after the
first three samples?
Actual Maximum
(no duplicates, uniform distribution, picked at random)
Highest sample
2
?
?
4
3
?
1
Lowest sample
Actual Minimum
?
Q. What is the chance of the 4th sample
being between the range seen after the
first three samples?
Actual Maximum
(no duplicates, uniform distribution, picked at random)
Highest sample
25% chance higher
than highest seen
2
25% lower than highest
and higher than second highest
4
3
25% higher than lowest and
lower than second lowest
1
Lowest sample
Actual Minimum
25% lower than
lowest seen
A. 50%
% = (1 - (1 / n – 1)) * 100
Q. What is the chance of the 12th sample
being between the range seen after the
first three samples?
Actual Maximum
(no duplicates, uniform distribution, picked at random)
Highest sample
2
9
5
5% chance higher than
highest seen
?
3
12
4
10
6
11
?
7
1
8
Lowest sample
Actual Minimum
5% lower than
lowest seen
A. 90%
% = (1 - (1 / n – 1)) * 100
# Prior Samples
3
4
5
6
7
8
9
10
11
12
13
15
17
20
Prediction Next Sample Within Prior Sample Range
50%
67%
75%
80%
83%
86%
88%
89%
90%
91%
92%
93%
94%
95%
Halved the
risk with 3
samples
Monte Carlo
If a measurement changes
over time or is different each
time you measure it, it is a
DISTRIBUTION
Can’t apply normal mathematical operators….
10
Commercial in confidence
1. Easy to
Capture Metric
3. Follows Known
Distribution Pattern
Cycle Time Forecasting
2. Forecasting using
Historical Data
(even a little)
11
Commercial in confidence
4. Forecast at
Project or Feature
or Story Level
Sum Random Numbers
Historical Story Lead Time Trend
Sum:
25
11
29
43
34
26
31
45
22
27
31
43
65
45
8
7
34
73
54
48
295
410
…..
Basic Cycle Time Forecast Monte Carlo Process
1. Gather historical story lead-times
2. Build a set of random numbers based on pattern
3. Sum a random number for each remaining story
to build a single outcome
4. Repeat many times to find the likelihood (odds)
to build a pattern of likelihood outcomes
𝑇𝑜𝑡𝑎𝑙 𝐷𝑎𝑦𝑠
𝑆𝑢𝑚 ( 𝑆𝑡𝑜𝑟𝑦𝑛 × 𝑅𝑎𝑛𝑑𝑜𝑚𝑛 )
=
𝐸𝑓𝑓𝑜𝑟𝑡
Days To Complete
19
12
24
27
21
3
9
20
23
29
187
How I Quantify Lead Time Reduction
Lead Time
# Stories / Year
Throughput Benefit
Current
6
0%
10% Decrease
7
17% More
20% Decrease
8
33% More
When no ROI is easily discerned (maintenance teams?)
Even a small decrease in Cycle Time has a increased
impact on throughput over a year..
How I Quantify Cycle Time Reduction
Cycle
Time
Forecast Date
Forecast
Cost
Cash flow to EOY14
(cost saving + revenue)
Benefit
Current
15-Jul-2014
$1,000,000
$0 + $60,000 = $60,000
0%
10%
27-May-2014
Decrease
$912,500
$87,500 + $90,000 =
$177,500
296%
Better
20%
04-Apr-2014
Decrease
$820,000
$120,000 + $145,000 =
$265,000
442%
Better
Revenue Estimates for product:
Month
Revenue
April
$30,000
May
$25,000
June
$20,000
July
$20,000
Aug-Dec
$10,000
Even a small decrease in Cycle
Time can have huge a benefit on
cash flow over time.
This example shows the cost of missing a seasonal uptick in summer
sales (revenue estimates shown to the left). Cost per work day is
calculated at $2,500 day.
Shape and Impact of
Cycle-Time & Scope
Probability Density Function
Note: Histogram from actual data
0.32
0.28
0.24
0.2
0.16
0.12
0.08
0.04
0
-10
0
10
20
30
40
50
60
70
80
x
Histogram
16
Gamma (3P)
Lognormal
Rayleigh
Commercial in confidence
Weibull
90
100
110
120
1
Probability Density Function
0.32
0.28
0.24
0.2
0.16
0.12
0.08
0.04
0
-10
0
10
20
30
40
50
60
70
x
Histogram
17
Weibull
Commercial in confidence
80
90
100
110
120
1
Likelihood…
18
Commercial in confidence
Story / Feature Inception
5 Days
Pre
Work
30
days
Total
Story
Lead
Time
Waiting in Backlog
25 days
“Active Development”
30 days
30
days
Post
Work
10
days
Waiting for Release Window
5 Days
System Regression Testing & Staging
5 Days
Binary Permutations
Risk Risk Risk Risk Risk Risk Risk Risk Risk Risk
1
2
3
4
5
6
7
8
9
10
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
Yes
No
No
No
No
No
No
No
No
Yes
No
No
No
No
No
No
No
No
No
Yes
Yes
No
No
No
No
No
No
No
Yes
No
No
…
…
…
…
…
…
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
20
Yes
Commercial in confidence
Why Weibull
• Now for some Math – I know, I’m excited too!
•
•
•
•
•
Simple Model
All units of work between 1 and 3 days
A unit of work can be a task, story, feature, project
Base Scope of 50 units of work – Always Normal
5 Delays / Risks, each with
– 25% Likelihood of occurring
– 10 units of work (same as 20% scope increase each)
Normal, or it will
be after a few
thousand more
simulations
Base + 1 Delay
Base + 2 Delays
Base + 3 Delays
Base + 4 Delays
Base + 5 Delays
Shape – How Fat the
distribution. 1.5 is a
good starting point.
Probability Density Function
0.28
0.24
f(x)
0.2
Scale – How Wide in
Range. Related to the
Upper Bound. *Rough*
Guess: (High – Low) / 4
Location – The
Lower Bound
0.16
0.12
0.08
0.04
0
0
10
20
30
40
50
60
70
80
90
100
110
x
Histogram
Weibull
KEY POINT: WITH JUST MIN AND MAX THE CURVE CAN BE INFERRED
28
Commercial in confidence
120
Changing Cycle Time
Shape
Probability Density Function
0.026
0.024
0.022
When forecasting, the wider
the curve, the MORE higher
value numbers will occur
0.02
0.018
0.016
f(x)
0.014
0.012
0.01
0.008
0.006
0.004
0.002
0
0
8
16
24
32
40
48
56
64
72
80
88
x
Weibull (1.5; 30)
Probability Density Function
0.028
Teams following this curve
WILL be able to predict more
predictability because
forecast range will be tighter
0.024
f(x)
0.02
0.016
0.012
0.008
0.004
0
0
8
16
24
32
40
48
56
64
x
31
Commercial in confidence
Weibull (1.2; 25)
72
80
88
Binary Permutations
Risk Risk Risk Risk Risk Risk Risk Risk Risk Risk
1
2
3
4
5
6
7
8
9
10
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
Yes
No
No
No
No
No
No
No
No
Yes
No
No
No
No
No
No
No
No
No
Yes
Yes
No
No
No
No
No
No
No
Yes
No
No
…
…
…
…
…
…
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
EVERY RISK OR DELAY YOU CAN REMOVE
REDUCE COMBINATIONS BY 2n
32
Commercial in confidence
Order of Priority for Improvement
• Prioritized list of blockers and delay states
• Balanced to include most frequent & biggest delay
• Order Risks and Delays by weighted impact
Impact = Frequency Risk or Delay “Type” x Duration
• This will remove the most combinations of delay
and shrink the area of our distribution giving
biggest benefit of predictability
33
Commercial in confidence
Mining
Cycle Time Data
Mining / Testing Cycle Time Data
35
Fit Shape
Capture
Data
Histogram
Scatter
Plot
Commercial in confidence
Cycle Time Capture Practices
• Clearly understand from where and to where
• Capture begin and end date; compute the
number of days in-between
• Does cycle time include defect fixing?
• Are there multiple types of work in the same
cycle time data
– Stories
– Defects
– Classes of Service
36
Commercial in confidence
Things that go wrong…
•
•
•
•
•
Zero values
Repetitive (erroneous) values
Batching of updates
Include/exclude weekends
Project team A staff raided impacting their
cycle time (Team A up, team B’s down)
• Work complexity changes
• Team skill changes
37
Commercial in confidence
Probability Density Function
0.36
0.32
Many low values. Often zero or values
below what makes sense. Check the
most frequent low values.
0.28
f(x)
0.24
0.2
0.16
0.12
0.08
0.04
0
0
20
40
60
80
100
120
x
Histogram
Weibull
Probability Density Function
0.22
0.2
0.18
Multiple modes. In this case two
overlapping Weibulls. Often due to
multiple classes of service, or most
often - defects versus stories.
0.16
0.14
f(x)
0.12
0.1
0.08
0.06
0.04
0.02
0
0
20
40
60
80
100
x
38
Histogram
Weibull
120
140
160
Commercial in confidence
Shape – How Fat the
distribution.
Probability Density Function
0.28
0.24
f(x)
0.2
Scale – How Wide in
Range. Related to the
Upper Bound
Location – The
Lower Bound
0.16
0.12
0.08
0.04
0
0
10
20
30
40
50
60
x
Histogram
39
Weibull
Commercial in confidence
70
80
90
100
110
120
Estimating
Distributions using
Historical Data
What Distribution To Use...
• No Data at All, or Less than < 11 Samples (why 11?)
– Uniform Range with Boundaries Guessed (safest)
– Weibull Range with Boundaries Guessed (likely)
• 11 to 50 Samples
– Uniform Range with Boundaries at 5th and 95th CI
– Weibull Range with Boundaries at 5th and 95th CI
– Bootstapping (Random Sampling with Replacement)
• More than 100 Samples
– Use historical data at random without replacement
– Curve Fitting
41
Commercial in confidence
Sampling at Random Strategies
• If you pick what samples to use, you bias the
prediction…
• Strategies for proper random sampling –
– Use something you know is random (dice, darts)
– Pick two groups using your chosen technique and
compute your prediction separately and compare
– Don’t pre-filter to remove “outliers”
– Don’t sort the data, in fact randomize more if
possible
42
Commercial in confidence
Estimating Concurrent Effort from
Cumulative Flow Chart
Concurrent WIP Sample : Find the smallest and the biggest or
take at least 11 samples to be 90% sure of range
43
Commercial in confidence
20%
40%
60%
80%
Scope Creep Over Time
Look at the rate new scope is added over time
45
Commercial in confidence