University of Southern California
Center for Systems and Software Engineering
Assessing the IDPD Factor:
Quality Management Platform Project
Thomas Tan
Qi Li
Mei He
University of Southern California
Center for Systems and Software Engineering
Table of Content
• Overview of IDPD
• Quality Management Platform (QMP)
Project
–
–
–
–
Project Information
Data Collection
Data Analysis Results
Discussion
• Q&A
University of Southern California
Center for Systems and Software Engineering
Incremental Development Productivity Decline (IDPD)
• Example: Site Defense BMD Software
– 5 builds, 7 years, $100M
– Build 1 productivity over 300 SLOC/person month
– Build 5 productivity under 150 SLOC/PM
• Including Build 1-4 breakage, integration, rework
• 318% change in requirements across all builds
• IDPD factor=20% productivity decrease per build
– Similar trends in later unprecedented systems
– Not unique to DoD: key source of Windows Vista
delays
University of Southern California
Center for Systems and Software Engineering
QMP Project Information
• Project Information:
– Web-based application
– System is to facilitate the process improvement initiatives in
many small and medium software organizations
– 6 builds, 6 years, different increment duration
– Size after 6th build: 548 KSLOC mostly in Java
– Average staff on project: ~20
University of Southern California
Center for Systems and Software Engineering
Data Collection
• Data Collection
– Most data come from release documentation, build reports,
and project member interviews/surveys
– Data include product size, effort by engineering phase, effort
by engineering activities, defects by phase, requirements
changes, project schedule, COCOMO II driver ratings (rated
by project developers and organization experts)
– Data collection challenges:
• Incomplete and inconsistency data
• Different data format, depends on who filled the data report
• No system architecture documents available
University of Southern California
Center for Systems and Software Engineering
Data Analysis – Staffing
•
•
•
•
Staffing is stable for most early builds, and enough talents stayed in the
project to overcome loss of developers
Some staff turnover occurred during build 5 and build 6
Experience gained – application and platform
Team cohesion improved
Staffing and Personnel Capabilities Ratings
Build
TEAM
PCON
ACAP
PCAP
APEX
LTEX
PLEX
1
NOM
HI
NOM
NOM
NOM
HI
VLO
2
HI
HI
NOM
NOM
HI
HI
LO
3
HI
HI
NOM
NOM
HI
HI
NOM
4
HI
HI
NOM
NOM
NOM
HI
HI
5
VHI
LO
NOM
NOM
NOM
HI
HI
6
VHI
LO
HI
NOM
NOM
HI
HI
University of Southern California
Center for Systems and Software Engineering
Data Analysis – Phase Effort Distribution
Phase Effort Percentage
Effort % by Phase
Build
Rqt
Design
Impl
Test
1
13.2%
16.5%
51.3%
18.9%
2
13.7%
16.3%
48.5%
21.6%
3
13.9%
16.7%
48.6%
20.8%
4
18.8%
21.5%
28.2%
31.6%
5
2.6%
6.9%
39.5%
50.9%
6
10.9%
•
•
•
•
6.9%
34.3%
48.0%
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Test
Impl
Design
Rqt
1
2
3
4
5
6
Builds
Experienced major integration difficulties in build 3 – major drop in
productivity (see next slide)
Forced re-architecting in build 4 – increase in requirement and design effort
Re-architecting paid off in build 5 and 6, which focused primarily on
implementation and testing
Testing and fixing as a major source of added integration effort – testing
phase effort increased from build to build
University of Southern California
Center for Systems and Software Engineering
Data Analysis – Productivity Trends
Productivity Trend
Build
Size
(KSLOC)
Effort
(PH)
Productivity
(SLOC/PH)
Productivity
Variance
1
69.12
7195
9.61
0.0%
2
64
9080
7.05
-26.6%
12.00
10.00
9.61
8.00
7.05
6.53
6.00
3
19
6647
2.86
-59.4%
4
39.2
8787.5
4.46
56.1%
5
207
31684.5
6.53
46.5%
4.46
4.00
4.01
2.86
2.00
0.00
0
1
2
3
4
5
6
Builds
6
65
•
•
•
16215.1
4.01
-38.6%
Productivity
Linear (Productivity)
The slope of the trend line is -0.76 SLOC/PH per build
Across the five builds, this corresponds to a 14% average decline in
productivity per build. This is smaller than the 20% Incremental
Development Productivity Decline (IDPD) factor for a large defense program
Most likely because the project is considerably smaller in system size and
complexity
7
University of Southern California
Center for Systems and Software Engineering
Discussion
• Staffing stability helps to improve team cohesion and developer
experience, thus provide positive contribution to productivity
outcome
• Design deficiency and code breakage causes productivity
declines
– In our case study, the development team encountered integration difficulties in
build 3, where the original design was insufficient to accommodate additional
modules, and a re-architecting effort was necessary to put this project back
on track – as what they have done in build 4.
– Inserting new code into the previous build adds effort to read, analyze, and
test both the new and old code in order to ensure nothing is broken, this extra
effort may be mitigated by experienced staff
• Extra testing and fixing effort, particularly regression and
integration tests, is inevitable, and the amount of this extra effort
will increase as the system becomes larger and larger
University of Southern California
Center for Systems and Software Engineering
Future Research
• Using COCOMO II Cost Drivers to normalize new size and effort:
– Product Effort Multipliers on Size
– Personnel Effort Multipliers on Effort
– Find Significant Effort Multipliers and analyze its impacts on
productivities
• Calibrate Equivalent New Size
– Calculate equivalent new size based on CodeCountTM “Diff” for each
increment and compare that with actual size
– Use the results to adjust parameters for calculating equivalent new
size with integration rework consideration
University of Southern California
Center for Systems and Software Engineering
Q&A
• Questions?
• Comments?
• Thank you very much