Software Estimation:
What, Why & How
Nupul Kukreja
19th October 2012
1
Based On
Software Estimation:
Demystifying The Black Art
Steve McConnell
Microsoft Press.
2
Agenda
• What is an “Estimate”?
• Purpose of Estimation
• In-class quiz to gauge & develop your
estimation skills
• What are “Good Estimates”?
• Estimation and Cone of Uncertainty
• Estimation Techniques
3
Agenda: VBSE 4+1 View
5a, 7b. Options, solution
development & analysis
Dependency
Theory
2a. Results chains
2. Identify SCS
3b, 5a, 7b. Cost/schedule/
performance tradeoffs
3b, 7a. Solution Analysis
6, 7c. Refine, execute,
monitor & control plans
Control
Theory
6a, 7c. State measurement,
prediction correction;
Milestone synchronization
3. SCS Value
Propositions
(Win Conditions)
Theory-W:
SCS Win-Win
1. Protagonist goals
3a. Solution Exploration
7. Risk, opportunity,
change management
Utility
Theory
4. SCS expectations
management
5a, 7b. Prototyping
5. SCS WinWin
Negotiation
Decision
Theory
5a. Investment
analysis, Risk analysis
4
What Is An ‘Estimate’?
• Estimate: Prediction of duration or cost of project
• Target: Statement of desirable business objective
Ex.:
– We need to have product ready by Christmas
– We must limit cost of next release to $3 million owing
to budget constraints.
• Commitment: Promise to deliver defined
functionality at specific level of quality by defined
date
• Commitment != Estimate (doesn’t have to be )
5
Why Do We Estimate?
• To determine if project’s targets are realistic
enough to control progress to meet them
• And NOT to predict a project’s outcome
• Estimate vs. Target Gap:
 ≤ 20%  Easy to control feature set, schedule,
team size towards realization
 Else  Not possible to control project towards
successful realization  Targets need to be better
aligned with reality 
6
Estimates vs. Plans
• Estimates form foundations for plans
• Plans don’t have to be same as estimates
– If (Targets – Estimates) >> 1
• Plans must account for high risk
– Else
• Plans can assume less risk
• Planning considerations that partially depend on
accurate estimates:
– Creating a detailed schedule
– Prioritizing functionality for delivery
– Breaking project into iterations etc.
7
Estimates As Probability Statements
• Single-point estimates “assume” 100% odds of
success; NOT realistic
• Usually a ‘Target’ in disguise 
• Must factor in uncertainty  i.e. project
success follows a probability distribution
100%
Nominal Outcome
Nominal Outcome
Probability
Schedule (or Cost of Effort)
Common Assumption
Schedule (or Cost of Effort)
More Realistic
8
What Is A Good Estimate?
“An estimate that provides a clear enough view
of the project reality to allow the project
leadership to make good decision about how
to control the project to hit its targets”
As quoted from Steve McConnell’s book.
9
How Good an Estimator Are YOU?
• Estimate the questions (in the following slide)
to the best of your ability
• Take a few WAGs if you will 
• Fill lower/upper bound so that there is a 90%
chance of including the correct value
• This is a quiz on estimation and NOT on
Googling skills. Estimate without any
electronic, human or supernatural help 
10
Estimation Quiz
Question
Lower Bound
Estimate
Upper Bound
Estimate
1. How far is the Moon from Earth?
2. What is the surface temperature of the Sun?
3. In which year was the “Arpanet” established as a
military communications system?
4. What year was William Shakespeare born?
5. What is latitude of Los Angeles?
6. How deep is the Mariana Trench?
7. How many time zones does Russia have?
8. When was UCLA founded? 
9. What is the length of an average business card?
10. What percentage of a square could be covered by
a circle of the same width?
(Answer with 90% Confidence i.e. 90% chance that answer is within bounds)
11
Love to Gamble?
Let’s say we gave you the following proposition:
• You win $1000 if the actual answer is within your range
• You win $1000 by spinning the wheel below:
$0
10%
90%
Win $1000
• What would you choose?
Your estimate/range or spin-the-wheel?
12
10%
Estimation Quiz
Question
Lower Bound
Estimate
90%
Upper Bound
Estimate
1. How far is the Moon from Earth?
2. What is the surface temperature of the Sun?
3. In which year was the “Arpanet” established as a
military communications system?
4. What year was William Shakespeare born?
5. What is latitude of Los Angeles?
6. How deep is the Mariana Trench?
7. How many time zones does Russia have?
8. When was UCLA founded? 
9. What is the length of an average business card?
10. What percentage of a square could be covered by
a circle of the same width?
(Answer with 90% Confidence i.e. 90% chance that answer is within bounds)
13
Implications of Choice
Spin the Wheel
Your Estimate/Range
‘Wheel’ has higher chance of payoff
Estimate has higher chance of payoff
Estimate range too narrow and thus
needs to be widened i.e. it was NOT 90%
Confident
Estimate range too wide and needs to be
narrowed i.e. it was more than 90%
Confident
Over-confident estimates
Under-confident estimate
Desirable:
Set range just right so as to be indifferent between gamble and your
estimate i.e. 90% chance, not more and not less, that answer is within range
90% Confident  90% of the time 90% of the answers within range
(i.e. you get 9 answers correct on 9 of 10 such quizzes)
14
Accuracy of Estimates
• Is it better to overestimate or underestimate?
Arguments against Overestimation
Arguments against Underestimation
Parkinson’s law may kick in:
Work expands to fill available volume
Reduced effectiveness of project
plans: understaffing, reduced
coordination across groups etc.
Goldratt’s Student Syndrome:
Procrastinate work until late in
project and then hurry to finish it
Statistically reduced chance of ontime completion: developers typically
estimate 20-30% lower than actual
effort
Motivation for underestimation:
To instill a sense of urgency
Poor technical foundations: May
spend less time on upstream activities
(e.g. requirements) leading to
suboptimal designs and rework
Destructive late project dynamics:
once late will make project worse
than nominal owing to ‘correction’
meetings/activities
16
Accuracy of Estimates
Effort, Cost, Schedule
Nonlinear penalty due
to planning errors,
upstream defects,
high-risk practices
Linear penalty due to
Parkinson’s Law
←Underestimation
< 100%
Overestimation→
100%
> 100%
Target as a Percentage of Nominal Estimate
Penalties of underestimation more severe than those for overestimation. If you can’t
estimate with complete accuracy, it’s better to err on the side of overestimation
– Steve McConnell
17
Value of Accurate Estimates
• Improved status visibility – compare planned progress with actual
progress
• Higher quality – Schedule-stress-related problems avoided i.e. not
patchy code etc.
• Better coordination with non-software functions – i.e. business
function such as testing, document writing, training, financial
projections etc.
• Better budgeting – enable better forecasting
• Increased credibility for development team - by providing
realistic/reliable estimates
• Early risk information – correct initial mismatch between project
goals and estimates
• Increased predictability – ability to know cost, schedule and
functionality in advance. Need to make commitments to customers,
investors, suppliers, the marketplace and other stakeholders!
18
Estimation Errors @#$%^&*(
• Four generic sources:
– Inaccurate information about project being
estimated
– Inaccurate information about capabilities of
organization implementing the project
– Too much chaos in project (i.e. moving target)
– Inaccuracies arising from estimation process itself
• In simple terms: missing, inaccurate, incorrect
understanding of features, costs, schedule,
effort etc., owing to the above factors
19
Cone of Uncertainty
4x
16x Error range!
2x
1.5x
1.25x
1.0x
0.8x
0.67x
0.5x
0.25x
Initial
Concept
Approved
Product
Definition
Requirements
Complete
User Interface
Design
Complete
Detailed Design
Complete
Software
Complete
The more refined the software’s definition the more accurate the estimate
20
“Front-Loaded” Cone of Uncertainty
4x
The cone narrows itself only if
you make decisions that
eliminate variability. Else it’d just
blow up even more
2x
1.5x
1.25x
1.0x
Software
Complete
0.8x
Detailed Design
Complete
0.67x
0.5x
0.25x
Initial
Concept
Approved
Product
Definition
Requirements
Complete
User Interface
Design
Complete
Milestones usually ‘front-loaded’  Improved estimation accuracy for first 30% of
project i.e. from ±4x to ±1.25x
21
Using the Cone of Uncertainty
• If using a single point estimate:
– Come up with estimate
– Use multiplying factors from previous chart for
relevant milestone to get range
Phase
Lower Bound
Upper Bound
Range
Initial Concept
0.25x
4.0x
16x
Approved Product
Definition
0.50x
2.0x
4x
Requirements Complete
0.67x
1.5x
2.25x
UI Design Complete
0.80x
1.25x
1.6x
Detailed Design Complete
(for sequential projects)
0.90x
1.10x
1.2x
22
Estimation Techniques
23
Count, Compute, Judge
• Count First – If you can count something
directly please do so 
• If you can’t count the answer directly, count
something else (i.e. correlated to the item you
wish to estimate) and compute the answer
(preferably by using calibration data)
• Use judgment as a last resort
24
Fermi-lize Your Estimation Skills
• Enrico Fermi – Won a Nobel Prize in Physics in
1938. Well known for his creative and
intuitive, even casual sounding estimates
• A "Fermi question" is a question which seeks a
fast, rough estimate of quantity which is
either difficult or impossible to measure
directly
• In-class: Estimate the number of words in the
today’s newspaper (Daily Trojan @ USC)*
*DEN Students: You may use whatever newspaper you have access to at home or
work. If not, you could just imagine one 
25
Fermi Decomposition
• Figure out something that is known about the
quantity in question
• Estimate other things that may have a bearing on
that quantity – it’s okay to have rough
approximations
• Sometimes you can just Google some numbers to
extrapolate from there!
• It’s ALWAYS possible to estimate and get ballpark
idea about the quantity in question 
• In-class 2: How many racket-ball or golf balls can
fit in this class room
26
Individual Expert Judgment
• Most common estimation approach
• Experts = those are doing the task 
• These are ‘task-level’ estimates i.e. specific
tasks like feature development, testing etc.,
• Task level estimation: Decompose estimates
into tasks requiring no more than 2 days of
effort (rule of thumb to avoid estimation
error)
27
Individual Expert Judgment
Feature
Feature 1
Feature 2
Feature 3
Feature 4
Feature 5
Feature 6
Feature 7
Feature 8
Feature 9
Estimated Days to Complete
1.5
1.5
2
0.5
0.5
0.25
2
1
0.75
Feature 10
1.25
Total
11.25
Example of developer single-point
estimates (not preferable)
Feature
Feature 1
Feature 2
Feature 3
Feature 4
Feature 5
Feature 6
Feature 7
Feature 8
Feature 9
Feature 10
Total
Estimated Days to Complete
Best Case
Worst Case
1.25
2
1.5
2.5
2
3
0.75
2
0.5
1.25
0.25
0.5
1.5
2.5
1
1.5
0.5
1
1.25
2
10.5
18.25
Example of individual estimation using best
case and worst case. Provides better
estimates and forces thinking of worst case
estimates – leading to better overall range
28
Individual Expert Judgment
Feature
Estimated Days to Complete
Best Case
Most Likely Case
Worst Case
Expected Case
Feature 1
1.25
1.5
2
1.54
Feature 2
1.5
1.75
2.5
1.83
Feature 3
2
2.25
3
2.33
Feature 4
0.75
1
2
1.13
Feature 5
0.5
0.75
1.25
0.79
Feature 6
0.25
0.5
0.5
0.46
Feature 7
1.5
2
2.5
2.00
Feature 8
1
1.25
1.5
1.25
Feature 9
0.5
0.75
1
0.75
Feature 10
1.25
1.5
2
1.54
Total
10.5
13.25
18.25
13.63
Even Better: Compute a 3-point estimate including the most-likely case
Compute expected case using the PERT formula:
Expected Case = [BestCase + (4* MostLikelyCase) + WorstCase]/6
29
Individual Expert Judgment
Estimated Days to Complete
Feature
MRE (%)
Best Case
Most Likely Case
Worst Case
Expected Case
Actual
Outcome
Feature 1
1.25
1.5
2
1.54
2
23%
YES
Feature 2
1.5
1.75
2.5
1.83
2.5
27%
YES
Feature 3
2
2.25
3
2.33
1.25
87%
NO
Feature 4
0.75
1
2
1.13
1.5
25%
YES
Feature 5
0.5
0.75
1.25
0.79
1
21%
YES
Feature 6
0.25
0.5
0.5
0.46
0.5
8%
YES
Feature 7
1.5
2
2.5
2.00
3
33%
NO
Feature 8
1
1.25
1.5
1.25
1.5
17%
YES
Feature 9
0.5
0.75
1
0.75
1
25%
YES
Feature 10
1.25
1.5
2
1.54
2
23%
YES
Total
10.5
13.25
18.25
13.63
16.25
Average:
In Range?
80% Yes
29%
Compare estimates to actuals to improve estimation accuracy over time and
‘narrow’ the cone of uncertainty
Magnitude of Relative Error: |(Actual – Estimate)/Actual|
30
Decomposition & Recomposition
• Process:
– Separate and estimate into multiple pieces
– Estimate each piece individually
– Recombine individual estimates into an overall
aggregate estimate
• AKA “bottom up” estimation or “Work
Breakdown Structure (WBS)
• Very important and highly used technique
• Leads to quite accurate estimates
31
Work Breakdown Structure (WBS)
• Example: Cost of owing and operating a car
– Buy the car
•
•
•
•
Pay down payment
Pay taxes, licensing and registration fees
Insurance the car
Pay monthly loan installments
Estimate each piece
individually and
aggregate the estimates
all the way to the top
– Operate and maintain the car
• Pay semi-annual insurance payments
• Fill car with gas when needed
• Change oil every 3000 miles
–
–
–
–
Take car to oil change shop
Let them do work
Pay fees and taxes
Drive back
• Other routine maintenance
Law of Large Numbers:
Overestimating some
pieces will help cancel
out some of the
underestimates of the
rest. Leading to better
estimates
– Sell the car
32
Estimation by Analogy
• Create estimates of new project by comparing it
to a similar past project
• Can help create accurate estimates (by following
the process below, instead of relying on memory)
– Get detailed size, effort and cost results for similar
previous project (WBS is preferable if possible)
– Compare size of new, piece-by-piece to previous
– Build up estimate for new project’s size as a
percentage of old project’s size
– Create an effort estimate based on size of new project
compared to that of previous one
– Check for consistent assumptions across the two
33
Proxy-Based Estimates
• Very difficult to estimate SLOC count looking at a
feature
• Or expected #defects, #test-cases, #classes etc.,
• Proxy-based estimation:
– Identify a proxy that is correlated with quantity to be
estimated
– Proxy is usually easier to estimate/count or available
sooner in project
– Compute estimate based on proxy and past historical
data
• Useful for creating whole-project or wholeiteration estimates but NOT for detailed task-bytask or feature-by-feature
34
Story Points
• Unit-less measure of ‘complexity’ or ‘size’ of
feature
• Scales:
– Powers of 2: 1, 2, 4, 8, 16 …
– Fibonacci: 1, 2, 3, 5, 8, 13 …
• #Story-Points per iteration = Velocity i.e.
looking at past velocity estimate completion
time of project (use historical data if available
or forecast or run one iteration)
35
T-Shirt Sizing
• Break features into various (albeit fuzzy)
T-shirt sizes
– Small, Medium, Large, X-Large, XX-Large etc.
• Estimate Business Value and Ease of
Realization using T-shirt sizes for each feature
• Gauge overall feature value based on
responses
• Provides crude, sufficiently accurate, initial
high-level estimates in the wide part of the
cone
36
T-Shirt Sizing Score Chart
• Can create one that suits the organization or
use the one below as per Steve McConnell:
Development Cost (Ease of Realization)
Business
Value
X-Large
Large
Medium
Small
X-Large
0
4
6
7
Large
-4
0
2
3
Medium
-6
-2
0
1
Small
-7
-3
-1
0
• Use scores from above chart to compute
approximate net business value for each
feature (in sorted order)
37
Expert Judgments in Groups
• Similar to planning poker 
1. Have each team member estimate pieces of
project individually
2. Meet to compare the estimates
3. Reach mutual consensus as group
– Without averaging the estimates. You may
average the estimates but you still need to
discuss individual results
• Extremely effective technique to help
improve estimation accuracy
38
Estimation by Tools
• Helps perform tasks that can’t be done manually
– Simulating project outcomes (i.e. sensitivity analysis)
– Probability Analysis i.e. viewing the (cumulative)
probability distribution of the estimates
– What-if analyses
– Serves as referee for unrealistic project expectations
– Estimation of less common software issues
• Works best if you have historical data for
calibration
39
List of Available Tools
• COCOMO II
• Construx Estimate
• Costar (commercial implementation of
COCOMO II)
• Price-S
• SEER
• SLIM-Estimate and Estimate Express
• Your own home grown one?
40
Using Multiple Approaches
• No single technique is perfect
• Best to augment estimation with multiple
approaches
• Each approach works best in specific context
• Convergence amongst estimates suggests a
good estimate
• Divergence helps see if something was
overlooked or needs to be understood better
41
Special Issues in Estimating Size
• Difficult to estimate raw SLOC
• Employ various techniques for ‘estimating’
– Function Point
– GUI Elements (i.e. counting and computing)
– Dutch Method
– Use-case points
– Class points? (i.e. estimating #classes etc.)
– COTIPMO 
– Application Points
– Object Points (not OOAD objects)
42
Special Issues in Estimating Effort
• Quite difficult to get right
• Various techniques maybe employed:
– Informal comparison to past projects
– Estimation software tools
– Industry-average effort graphs
– ISBSG Method (International Software
Benchmarking Standards Group)
• Has developed various equations fitting various types
of projects that can be used for estimating effort (based
on function-points for size)
43
Special Issues in Estimating Schedule
• As evident, equally hard as the other two
• Various techniques:
– Schedule Equation (Courtesy Dr. Boehm ):
• ScheduleInMonths = 3.0 x StaffMonths(1/3)
– Informal comparison to past projects
– Estimation software tools
44
Conclusion
• Estimation is NOT an art (somewhat but not
entirely)
• Can effectively be executed as a science
• Need not rely purely on intuition or memory
• Improves over time especially if historical data is
captured
• Requires basic arithmetic to understand
• Complex models can be created with the help of
statistics – premise of most ‘tools’
• VERYYYYYYYYYYYYYYYY Critical skill-set to have in
the 21st Century 
45