Software Technology Support Center
The Problems in
Software
Estimating
Dr. Randall Jensen
Software Technology Support Center
Hill AFB, UT
BE AMERICA’S BEST
Outline
OGDEN AIR LOGISTICS CENTER
It’s all about models – “Existence is elsewhere”
 Evolution
of software estimating models
 Software
problems
 Management
is the issue
 Data
collection concerns
 New
models?
BE AMERICA’S BEST
Project Uncertainty Principle
OGDEN AIR LOGISTICS CENTER
BE AMERICA’S BEST
Software model genealogy
OGDEN AIR LOGISTICS CENTER
Nordon
IBM 1970
Putnam/SLIM
US Army/GE/
QSM 1976
Seer
HAC 1979
Doty
RADC 1977
JS1,2,3
CEI 1980
SEER-SEM
GAI 1989
Validation
COCOMO
TRW 1981
PRICE-S
Price 1977
Seer II / Sage
SEI 1995
COCOMO II
USC 1995
REVIC
USAF 1991
TRUE-S
Price 2003
Sage/SEER-SEM - Normal form
OGDEN AIR LOGISTICS CENTER
Se  Cte KTd
sloc
K
D 3
Td
where
K = life cycle cost (person years),
Td = development time (years)
BE AMERICA’S BEST
Common estimate dilemma
OGDEN AIR LOGISTICS CENTER
BE AMERICA’S BEST
Software problems are not new
OGDEN AIR LOGISTICS CENTER

KEY
Unreliable

Late delivery

Modification costs prohibitive

Impossible to maintain

Inadequate performance

Product exceeds budget costs
1968 CS Conference, Munich, Germany
BE AMERICA’S BEST
Historic note:
OGDEN AIR LOGISTICS CENTER
“More software projects have
gone awry for lack of calendar
time than for all other causes
combined…”
F. P. Brooks, Jr., Mythical Man Month
BE AMERICA’S BEST
Common technology claim
OGDEN AIR LOGISTICS CENTER
If you use (new technology),
Productivity will improve by an order of magnitude
And
Defects will reduce to zero
Note: None of the history-based estimating models (tools) support this
claim, not even close.
Neither does history.
BE AMERICA’S BEST
There is always HOPE
OGDEN AIR LOGISTICS CENTER
P
R
O
D
U
C
T
I
V
I
T
Y
lppm
100
90
Structured Analysis
Ada
Structured Design
80
OOD
70
PWB
60
50
Process Maturity
Structured Programming
3rd Generation Languages
1960
1970
BE AMERICA’S BEST
1980
1990
3 Dimensions of development
OGDEN AIR LOGISTICS CENTER
1938
1975
PROJECT
1985
1995
PROCESS
PEOPLE
BE AMERICA’S BEST
Impact of management on cost
OGDEN AIR LOGISTICS CENTER
“Poor management can increase software costs
more rapidly than any other factor…
Despite this variation, COCOMO does not include
a factor for management quality, but instead
provides estimates which assume that the project
will be well managed [italics mine, RWJ]”
Boehm, B. W., Software Engineering Economics, (Englewood
Cliffs, NJ: Prentice-Hall, Inc.), 1981, pg. 486
Note: On a scale of 0 to 10 well managed must rank approximately 3.4?
BE AMERICA’S BEST
Where would you focus effort?
OGDEN AIR LOGISTICS CENTER
3
11
17
64
TOOLS
PEOPLE
SYSTEMS
MANAGEMENT
Source: G. Weinberg, Quality Software Management, Vol. 3
BE AMERICA’S BEST
Biased research?
OGDEN AIR LOGISTICS CENTER
70
Percent
60
SEI Papers
50
40
30
20
10
0
Tools
People Systems Managers
Source: G. Weinberg, Quality Software Management, Vol. 3
BE AMERICA’S BEST
Cost driver relative impact
OGDEN AIR LOGISTICS CENTER
Relative Development Cost Impact
Staff Capability
Practices Experience
Application Experience
Modern Practices
Development Tools
Resource Location
Multiple Organizations
Develop Sys Experience
Multiple Sites
Target Experience
0
0.5
1
1.5
2
2.5
It takes people to implement processes and use technology.
April 21st, 2009
BIG productivity drivers
OGDEN AIR LOGISTICS CENTER
 Analyst
capability
Management style
 Motivation
 Use of team methods
 Working environment
 Problem solving skills

 Programmer
capability
 Application domain experience
 Use of modern practices
 Automated tool support
BE AMERICA’S BEST
Capability shift (environment)
OGDEN AIR LOGISTICS CENTER
1980
2005
8635
(1990)
5500
6200
6500
7500
Basic Technology Constant
BE AMERICA’S BEST
8000
8500
Historic data observations
OGDEN AIR LOGISTICS CENTER

Models are made from historic data







More data is better
Adequate historic data takes time to collect (Lots of time)
Repository vs Database
Repository = a place where things may be stored for safekeeping
 Data stored without concern for a formal definition of the data
placed in the repository
 SRDR is defined as a financial report that happens to contain
technical information
Database = a collection of data arranged for ease and speed of
retrieval
 Requires consistency of entries
 Each data field must satisfy common definition
DCARC is a repository as was SMC database
Data definitions?
BE AMERICA’S BEST
18
Problems?
OGDEN AIR LOGISTICS CENTER
BE AMERICA’S BEST
Popular estimating solution
OGDEN AIR LOGISTICS CENTER
Oh, really!
Are you sure?
 Let’s create a new software model

Models are out of date

Models are inaccurate

Too difficult to use
BE AMERICA’S BEST
New estimating models?
OGDEN AIR LOGISTICS CENTER




Lack of confidence in existing tools (current issues)
 Optimistic estimates
 Unacceptable estimates
Aging tools (?)
 Culture is constant
 New models require validation
New models -- No silver bullets
 Quality data (if, where, proprietary)
 New models require validation
New (or better) estimators?
 Experience
 Skill
 Integrity
BE AMERICA’S BEST
OGDEN AIR LOGISTICS CENTER
Estimating
Prediction is very hard, especially when it’s about the
future