“Raising Your Technical Debt Ceiling
Or NOT”
CQAA Webcast
December 6, 2011
Michael Mah
Managing Partner
QSM Associates, Inc.
75 South Church Street
Pittsfield, MA 01201
413-499-0988
Fax 413-447-7322
e-mail: michael.mah@qsma.com
Website: www.qsma.com
Blog: www.optimalfriction.com
Rayleigh Curve Defect Rate
(#2)
Rayleigh Curve Defect Rate
Defect Discovery Rate
Discovered Defects
per reporting
period at time t
Area under the curve is
equal to the
Total Discovered Defects
on the project
Elapsed Calendar Time
(#3)
t
Defects and Sources of Technical Debt
(#4)
Defects and Sources of Technical Debt
(#5)
Defects and Sources of Technical Debt
(#6)
Case Study #1
(#7)
Case Study #1
(#8)
Case Study #1
(#9)
Project Sketch – Core Metrics
(#10)
Trendline Assessment –
Defects/Quality
Defects During Test
10,000
1,000
Errors (SysInt-Del)
Rel 8.0
Rel 6.0
Rel 7.5
Rel 6.5
Rel 7.0
Rel 5.0
100
Far Fewer Defects: 50% - 66% Below Industry
10
1,000
100
Effective SLOC (thousands)
Business Sy stems
Av g. Line Sty le
(#11)
Av ionic Sy stems
1 Sigma Line Sty le
Command & Control
Microcode Sy stems
Process Control
QSM 2005 Business
Trendline Assessment – Build Phase
Schedule
Main Build Phase Duration vs Size
100
Rel 6.5
Rel 5.0
Rel 6.0
Time (Months)
10
Rel 8.0
Rel 7.0
Rel 7.5
Schedules are Half Industry
1
1,000
100
Effective SLOC (thousands)
Business Sy stems
Av g. Line Sty le
(#12)
Av ionic Sy stems
1 Sigma Line Sty le
Command & Control
Microcode Sy stems
Process Control
QSM 2005 Business
Follett vs. Industry Average
(#13)
Industry
Average
Current
Performance
Delta
Project Cost
$3.5 Million
$2.2 Million
-$1.3M
Schedule
12.6 months
7.8 months
-4.8 mos
QA Defects
242
121
-50%
Staffing
35
35
n/a
Case Study #2
(#14)
(#15)
(#16)
(#17)
(#18)
(#19)
Benchmark Assessment — Staffing
Average Staff
1,000
Smaller Teams
100
Headcount
10
1
10
1
1,000
100
New + Modified Size (thousands)
(#20)
All Systems
Special Project
QSM 2008 Business
Avg. Line Style
1 Sigma Line Style
Benchmark Assessment — Schedule
Time-to-Market
100
Faster Schedules
1
10
1
1,000
100
New + Modified Size (thousands)
(#21)
All Systems
Special Project
QSM 2008 Business
Avg. Line Style
1 Sigma Line Style
Months
10
Benchmark Assessment – Quality
Bugs During QA
10,000
Fewer Defects
1,000
10
1
10
1
1,000
100
New + Modified Size (thousands)
(#22)
All Systems
Special Project
QSM 2008 Business
Avg. Line Style
1 Sigma Line Style
Defects
100
Monstrous Technical Debt Project
(#23)
Monstrous Technical Debt Project
INCEP
ELAB
CONST
TRANS
Avg Staff (people)
<Overall September Baseline>
0
1
3
4
5
6
8
9
10
120
100
80
40
20
0
Apr
'03
(#24)
1
Jun
3
Aug
5
Oct
7
Dec
9
Feb
'04
11
Apr
13
Jun
15
Aug
17
Oct
19
Dec
21
Feb
'05
Avg Staff (people)
60
Monstrous Technical Debt Project
Gantt Chart By Sub-Phase
<July Baseline Plan>
0
2
3
5
6
7 10
11
Inception
Elaboration 1
Elaboration 2
Elaboration 3
Construction 1 - Pre-FIT
Sub-Phases
Construction 2 - FIT
Construction 3 - UAT 1
Construction 4 - UAT 2
Construction 5 - Prod DataLoad
Production Pilot
Post Production
(#25)
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16
Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep
'03
'04
Defects vs. Industry Average
Defects Found
0
1
3
4
5
5
6
6
78
9
7 8
9
10
10 350
300
250
200
Defects
150
100
50
0
05/17
'03
(#26)
6
06/28
12
08/09
Current Plan
18
09/20
24
11/01
30
12/13
Actuals
36
01/24
'04
42
03/06
48
04/17
54
05/29
Green Control Bound
60
07/10
66
08/21
72
10/02
78
11/13
Yellow Control Bound
84
12/25
90
02/05
'05
Project: CEP
Forecast to Complete (4 Month Delay)
Nov 19
(#27)
Comparison to Industry Average
(#28)
Industry
Average
Offshore
Average
Delta
Project Cost
$3.5 Million
$3.2 Million
-$0.3M
Schedule
12.6 months
9.6 months
-3.0 mos
QA Defects
242
677
+280%
Staffing
35
50
+15
* Normalized to same project size
Some
Practical
Advice
(#29)
Build a Little Less
(#30)
Take a Little More Time
(#31)
Get Smart People
(#32)
Use Small A-Teams
(#33)
Give Them the Best Tools
(#34)
(#35)
(#36)
(#37)
(#38)
(#39)
(#40)
(#41)
(#42)
(#43)
(#44)
www.qsma.com
(#45)
SLIMEstimate:
Size, Schedule,
Cost & Quality
Estimating
SLIMMasterPlan:
Incremental
Development &
Project
SLIM-Control:
Aggregation
Variance
Analysis
&
Adaptive
Forecasting
SLIM-Metrics:
Industry
Benchmarking
& Process
Improvement
SLIM-DataManager
Software Project Metrics Repository
(#48)
footer
For Additional Information
Michael Mah
email: michael.mah@qsma.com
website: www.qsma.com
blog: www.optimalfriction.com
twitter: @michaelcmah
Tel: 1 413-499-0988
(#49)