Analysis of Process Maturity and
Productivity with SRDR Data
USC CSSE Annual Research Review
April 29 – May 1, 2014
Anandi Hira, Jo Ann Lane
Outline
•
•
•
•
•
Motivation
Explanation of the SRDR Data Repository
Data Processing
Analysis Procedure
Results of Analyses per Taxonomy and Comparison
– COCOMO II Comparisons
– Application Domains
– Productivity Types
• Factors to Consider in Productivity Analysis -> Future Work
• Parameter Suggestions for Future Research and Analyses
• Questions/Suggestions
Motivation
• Improve Productivity
• Invest resources to improve processes
• Process Certification  Productivity ?
• SRDR Data
SRDR Data
• Software Resources
• Relevant Parameters
Data Reporting (SRDR)
provided:
– Total Effort (hours)
• Quantitative data and
– Equivalent Total SLOC
associated parametric
– SLOC Counting Method
project characteristics
– CMM/CMMI Levels
• DoD software-intensive
• Unused Parameters
system development
– Effort distribution per
projects
phase, Programming
• Data analysis and trends
language(s), Personnel
research
Experience
Data Processing
• Filtering Data
– Remove outliers
– Remove points
without relevant
parameters
– Projects < 10 EKSLOC
– Levels 2 and 4
• Normalizing Data
– Logical SLOC
– Counting adjustment
factors
– Non-comment: 0.66 *
SLOC
– Physical: 0.34 * SLOC
Analysis Procedure
1
• Combine CMM and CMMI Levels 3 and 5
2
• Compare productivity to parameter ratings of
COCOMO II™
3
• Categorize data by Application Domains
• ANOVA to test significance
4
• Categorize data by Productivity Types
• ANOVA to test significance
COCOMO Comparisons
Size
Range
Mean
Level 3
Mean
Level 5
%
Increase
/ Level
3.28%
COCOMO II
168.88
%
Increase
(Prod)
6.57%
10-50
EKSLOC
158.47
50 – 100
EKSLOC
> 100
EKSLOC
278.19
267.09
-3.99%
-1.995%
261.33
350.45
34.10%
17.05%
7% for 75
EKSLOC
9.5% for
300 EKSLOC
87
EKSLOC
215.33
242.50
12.62%
6.31%
5% for 30
EKSLOC
7% for 75
EKSLOC
Application Domains
Application Domains – ANOVA Test Results
Application Domain Fvalue
Command & Control 0.564
Communications
0.767
PResult
value
0.464 Rejected
0.387 Rejected
Productivity Types
Productivity Types – ANOVA Test Results
Productivity Type
Command & Control (C & C)
Mission Processing (MP)
Real Time Embedded (RTE)
Telecommunications (TEL)
Vehicle Payload (VP)
F-value
0.564
3.070
0.274
0.745
0.031
P-value
0.464
0.091
0.603
0.393
0.863
Result
Rejected
Rejected
Rejected
Rejected
Rejected
Findings and Conclusions
• COCOMO Comparisons
– Data broken up by sizes do not closely correspond to
parameter ratings
– Average of all data corresponds to parameter rating
• Application Domains and Productivity Types
– Inconsistence with regards to productivity
increase/decrease from Level 3 to Level 5
– Difference in productivity ranges of Level 3 and 5 are
statistically insignificant
Future Work – Factors to Consider
• Counting methods not
standard and may skew
analysis
• Code reuse gains
factored and normalized
in data
• Analysis of trends of
productivity over time
• IDPD
• Cost drivers and
parameters that effect
productivity not
provided and random
with respect to time
–
–
–
–
Staff experience
Tool support
Code reuse
Improved architecting
and risk resolution
Productivity Over Time
Future Work – Parameter Suggestions
• (Relative) Time of
• Equivalent Output
Project Implementation
Metric per
Phase/Activity
– Other data points
– Adopting process
• Rework SLOC and Effort
maturity levels
• Volatility
• Equivalent Metric for
• Complexity
Non-Development Effort
Questions and Suggestions