Software Estimation:
What, Why & How
Nupul Kukreja
17th September 2013
1
Based On
Software Estimation:
Demystifying The Black Art
Steve McConnell
Microsoft Press.
3
Agenda
•
•
•
•
•
•
Purpose of Estimation
Estimation and Plans
What are “Good Estimates”?
Estimation Quiz
Under-estimation or Over-estimation?
Estimation Tools and Techniques
4
Why Do We Estimate?
• To determine if project’s targets are realistic
enough to control progress to meet them
• And NOT to predict a project’s outcome
• Estimate vs. Target Gap:
 ≤ 20%  Easy to control feature set, schedule,
team size towards realization
 Else  Not possible to control project towards
successful realization  Targets need to be better
aligned with reality
5
Estimates vs. Plans
• Estimates form foundations for plans
• Plans don’t have to be same as estimates
– If (Targets – Estimates) >> 1
• Plans must account for high risk
– Else
• Plans can assume less risk
• Accurate Estimates needed for:
– Creating a detailed schedule
– Prioritizing functionality for delivery
– Breaking project into iterations etc.
6
Estimates As Probability Statements
• Single-point estimates “assume” 100% odds of
success; NOT realistic
• Usually a ‘Target’ in disguise
• Must factor in uncertainty  i.e. project
success follows a probability distribution
100%
Nominal Outcome
Nominal Outcome
Probability
Schedule (or Cost of Effort)
Common Assumption
Schedule (or Cost of Effort)
More Realistic
7
What Is A Good Estimate?
“An estimate that provides a clear enough view
of the project reality to allow the project
leadership to make good decision about how
to control the project to hit its targets”
As quoted from Steve McConnell’s book.
8
How Good an Estimator Are YOU?
• Estimate the questions (in the following slide)
to the best of your ability (WAGs)
• Fill lower/upper bound so that there is a 90%
chance of including the correct value
• Please estimate without any electronic,
human or supernatural help 
9
Estimation Quiz
Question
Lower Bound
Estimate
Upper Bound
Estimate
1. How far is the Moon from Earth?
2. What is the surface temperature of the Sun?
3. In which year was the “Arpanet” established as a
military communications system?
4. What year was William Shakespeare born?
5. What is latitude of Los Angeles?
6. How deep is the Mariana Trench?
7. How many time zones does Russia have?
8. When was UCLA founded?
9. What is the length of an average business card?
10. What is Nupul’s weight? 
(Answer with 90% Confidence i.e. 90% chance that answer is within bounds)
10
Love Vegas?
• You win $1000 if the actual answer is within your range
OR
• You win $1000 by spinning the wheel below:
$0
10%
90%
Win $1000
11
10%
Estimation Quiz
Question
Lower Bound
Estimate
90%
Upper Bound
Estimate
1. How far is the Moon from Earth?
2. What is the surface temperature of the Sun?
3. In which year was the “Arpanet” established as a
military communications system?
4. What year was William Shakespeare born?
5. What is latitude of Los Angeles?
6. How deep is the Mariana Trench?
7. How many time zones does Russia have?
8. When was UCLA founded?
9. What is the length of an average business card?
10. What is Nupul’s weight? 
(Answer with 90% Confidence i.e. 90% chance that answer is within bounds)
12
Implications of Choice
Spin the Wheel
Your Estimate/Range
‘Wheel’ has higher chance of payoff
Estimate has higher chance of payoff
Estimate range too narrow and thus
needs to be widened i.e. it was NOT 90%
Confident
Estimate range too wide and needs to be
narrowed i.e. it was more than 90%
Confident
Over-confident estimates
Under-confident estimate
Desirable:
Set range just right so as to be indifferent between gamble and your
estimate i.e. 90% chance, not more and not less, that answer is within range
90% Confident  90% of the time 90% of the answers within range
(i.e. you get 9 answers correct on 9 of 10 such quizzes)
13
Accuracy of Estimates
Effort, Cost, Schedule
Nonlinear penalty due
to planning errors,
upstream defects,
high-risk practices
Linear penalty due to
Parkinson’s Law or
Student’s Syndrome
←Underestimation
< 100%
Overestimation→
100%
> 100%
Target as a Percentage of Nominal Estimate
Penalties of underestimation more severe than those for overestimation. If you can’t
estimate with complete accuracy, it’s better to err on the side of overestimation
– Steve McConnell
15
Estimation Techniques
22
Count, Compute, Judge
• Count First – If you can count something
directly please do so
• If you can’t count the answer directly, count
something else (i.e. correlated to the item you
wish to estimate) and compute the answer
(preferably by using calibration data)
• Use judgment as a last resort
23
Fermi-lize Your Estimation Skills
• Enrico Fermi – Won a Nobel Prize in Physics in
1938
• Well known for his creative and intuitive, even
casual sounding estimates
• A "Fermi question" is a question which seeks a
fast, rough estimate of quantity which is
either difficult or impossible to measure
directly. Ex:
– Number or chauffeurs currently in LA
– Number of piano repairmen in Chicago
24
1. Fermi Decomposition
• Figure out something that is known about the
quantity in question
• Estimate other things that may have a bearing
on that quantity – it’s okay to have rough
approximations
• Sometimes you can just Google some
numbers to extrapolate from there
• It’s ALWAYS possible to estimate and get
ballpark idea about the quantity in question
25
Team In-Class
• How many golf balls can fit in this class room
(OHE 123)?*
*DEN Students: You may think of the room you are viewing the lecture in 
26
2. Individual Expert Judgment
• Most common estimation approach
• Experts  those are doing the task
• Task level estimation:
– Decompose estimates into tasks requiring no
more than 2 days of effort (rule of thumb to avoid
estimation error)
• Example: Estimate duration for:
– As a user I can log in to the system, via valid a
username and password, so that I can access my
account data
27
Individual Expert Judgment
Feature
Feature 1
Feature 2
Feature 3
Feature 4
Feature 5
Feature 6
Feature 7
Feature 8
Feature 9
Estimated Days to Complete
1.5
1.5
2
0.5
0.5
0.25
2
1
0.75
Feature 10
1.25
Total
11.25
Example of developer single-point
estimates (not preferable)
Feature
Feature 1
Feature 2
Feature 3
Feature 4
Feature 5
Feature 6
Feature 7
Feature 8
Feature 9
Feature 10
Total
Estimated Days to Complete
Best Case
Worst Case
1.25
2
1.5
2.5
2
3
0.75
2
0.5
1.25
0.25
0.5
1.5
2.5
1
1.5
0.5
1
1.25
2
10.5
18.25
Example of individual estimation using best
case and worst case.
Provides better estimates
Forces thinking of worst case estimates –
leading to better overall range
28
Individual Expert Judgment
Feature
Estimated Days to Complete
Best Case
Most Likely Case
Worst Case
Expected Case
Feature 1
1.25
1.5
2
1.54
Feature 2
1.5
1.75
2.5
1.83
Feature 3
2
2.25
3
2.33
Feature 4
0.75
1
2
1.13
Feature 5
0.5
0.75
1.25
0.79
Feature 6
0.25
0.5
0.5
0.46
Feature 7
1.5
2
2.5
2.00
Feature 8
1
1.25
1.5
1.25
Feature 9
0.5
0.75
1
0.75
Feature 10
1.25
1.5
2
1.54
Total
10.5
13.25
18.25
13.63
Even Better: Compute a 3-point estimate including the most-likely case
Compute expected case using the PERT formula:
Expected Case = [BestCase + (4* MostLikelyCase) + WorstCase]/6
29
Individual Expert Judgment
Estimated Days to Complete
Feature
MRE (%)
Best Case
Most Likely Case
Worst Case
Expected Case
Actual
Outcome
Feature 1
1.25
1.5
2
1.54
2
23%
YES
Feature 2
1.5
1.75
2.5
1.83
2.5
27%
YES
Feature 3
2
2.25
3
2.33
1.25
87%
NO
Feature 4
0.75
1
2
1.13
1.5
25%
YES
Feature 5
0.5
0.75
1.25
0.79
1
21%
YES
Feature 6
0.25
0.5
0.5
0.46
0.5
8%
YES
Feature 7
1.5
2
2.5
2.00
3
33%
NO
Feature 8
1
1.25
1.5
1.25
1.5
17%
YES
Feature 9
0.5
0.75
1
0.75
1
25%
YES
Feature 10
1.25
1.5
2
1.54
2
23%
YES
Total
10.5
13.25
18.25
13.63
16.25
Average:
In Range?
80% Yes
29%
Compare estimates to actuals to improve estimation accuracy over time
Magnitude of Relative Error: |(Actual – Estimate)/Actual|
30
3. Decomposition & Recomposition
• Idea:
– Separate an estimate into multiple pieces
– Estimate each piece individually
– Recombine individual estimates into an overall
aggregate estimate
• AKA “bottom up” estimation or “Work
Breakdown Structure (WBS)
• Very important and highly used technique
• Leads to quite accurate estimates
31
Work Breakdown Structure (WBS)
• Example: Cost of owing and operating a car
– Buy the car
•
•
•
•
Pay down payment
Pay taxes, licensing and registration fees
Insure the car
Pay monthly loan installments
Estimate each piece
individually and
aggregate the estimates
all the way to the top
– Operate and maintain the car
• Pay semi-annual insurance payments
• Fill car with gas when needed
• Change oil every 3000 miles
–
–
–
–
Take car to oil change shop
Let them do work
Pay fees and taxes
Drive back
• Other routine maintenance
Law of Large Numbers:
Overestimating some
pieces will help cancel
out some of the
underestimates of the
rest. Leading to better
estimates
– Sell the car
32
Buy Car
Operate and
maintain the car
Down payment
Pay semi-annual
insurance payments
taxes, licensing and
registration
Fill car with gas when
needed
Sell the car
Change oil every 3000
miles
Insurance
Monthly loan installments
•Take car to oil change shop
•Let them do work
•Pay fees and taxes
•Drive back
Other routine
maintenance
33
4. Estimation by Analogy
• Create estimates of new project by comparing it
to a similar past project
• Can help create accurate estimates (by following
the process below, instead of relying on memory)
– Get detailed size, effort and cost results for similar
previous project (WBS is preferable if possible)
– Compare size of new, piece-by-piece to previous
– Build up estimate for new project’s size as a
percentage of old project’s size
– Create an effort estimate based on size of new project
compared to that of previous one
– Check for consistent assumptions across the two
34
5. Proxy-Based Estimates
• Very difficult to estimate SLOC count looking at a
feature
• Or expected #defects, #test-cases, #classes etc.,
• Proxy-based estimation:
– Identify a proxy that is correlated with quantity to be
estimated
– Proxy is usually easier to estimate/count or available
sooner in project
– Compute estimate based on proxy and past historical data
• Useful for creating whole-project or whole-iteration
estimates but NOT for detailed task-by-task or featureby-feature
• Example of two proxy-based estimates…
35
5.1 Story Points
• Unit-less measure of ‘complexity’ or ‘size’ of
feature
• Scales:
– Powers of 2: 1, 2, 4, 8, 16 …
– Fibonacci: 1, 2, 3, 5, 8, 13 …
• #Story-Points per iteration = Velocity i.e.
looking at past velocity estimate completion
time of project (use historical data if available
or forecast or run one iteration)
36
Story Point Practice
As a user I can log in to the system, via valid a
username and password, so that I can access my
account data
1 3 5 8
13 20 40 100
37
6. Expert Judgments in Groups
• Similar to planning poker
1. Have each team member estimate pieces of
project individually
2. Meet to compare the estimates
3. Reach mutual consensus as group
– Without averaging the estimates. You may
average the estimates but you still need to
discuss individual results
• Extremely effective technique to help
improve estimation accuracy
40
7. Estimation by Tools
• Helps perform tasks that can’t be done manually
– Simulating project outcomes (i.e. sensitivity analysis)
– Probability Analysis i.e. viewing the (cumulative)
probability distribution of the estimates
– What-if analyses
– Serves as referee for unrealistic project expectations
– Estimation of less common software issues
• Works best if you have historical data for
calibration
41
List of Available Tools
• COCOMO II
• Construx Estimate
• Costar (commercial implementation of
COCOMO II)
• Price-S
• SEER
• SLIM-Estimate and Estimate Express
• Home grown?
42
Using Multiple Approaches
• No single technique is perfect
• Best to augment estimation with multiple
approaches
• Each approach works best in specific context
• Convergence  good estimate
• Divergence  something overlooked or needs
to be understood better
43
Conclusion
• Estimation is NOT an art (somewhat but not
entirely)
• Can effectively be executed as a science
• Need not rely purely on intuition or memory
• Improves over time especially if historical data is
captured
• Requires basic arithmetic to understand
• Complex models can be created with the help of
statistics – premise of most ‘tools’
• VERYYYYYYYYYYYYYYYY Critical skill-set to have in
the 21st Century 
47
References
• Software Estimation: Demystifying the black
art – Steve McConnell
• How to Measure Anything – Doug Hubbard
• Software Estimation Economics – Barry
Boehm (a.k.a., The Blue Book)
• Software Cost Estimation with COCOMO –
Boehm et. al.
48