University of Southern California
Center for Systems and Software Engineering
Productivity Data Analysis and Issues
Brad Clark, Thomas Tan
USC CSSE Annual Research Review
March 8, 2010
University of Southern California
Center for Systems and Software Engineering
Table of Contents
•
•
•
•
Background
Productivity Data Analysis by Application Domain
Reducing the number of domains: Application Difficulty
Topics for further discussion
This work is sponsored by the Air Force Cost Analysis Agency
USC CSSE Annual Research Review - Mar 2010
2
University of Southern California
Center for Systems and Software Engineering
Background
• DoD has been collecting Software resource data for a number of
years
–
–
–
–
Product and development description
Product size
Resources and schedule
Product quality
• Analyzing ~140 records out of ~300
– Additional data is coming in
• Objective: Improved cost estimation of future DoD softwareintensive systems, as well as to the DoD cost community.
– Characterize different Application Domains within DoD
– Analyze collected data for simple cost estimating relationships within
each domain
– Develop rules-of-thumb for missing data
• Make collected data useful to oversight and management entities
USC CSSE Annual Research Review - Mar 2010
3
University of Southern California
Center for Systems and Software Engineering
Software Resources Data Report
USC CSSE Annual Research Review - Mar 2010
4
University of Southern California
Center for Systems and Software Engineering
SRDR Data
Application Domain
Business Systems
Command & Control
Communications
Controls & Displays
Executive
Information Assurance
Infrastructure or Middleware
Mission Management
Mission Planning
Process Control
Scientific Systems
Sensor Control and Processing
Simulation & Modeling
Spacecraft Payload
Test & Evaluation
Tool & Tool Systems
Training
Weapons Delivery and Control
Total
Avionics
Fixed
Ground
1
1
2
8
35
1
12
1
1
2
2
4
Operating Environment
Mobile
Unmanned
Missile
Ground
Shipboard
Space
4
5
2
1
1
3
3
1
3
1
4
3
10
3
2
9
1
1
3
1
4
21
68
7
10
16
25
2
Total
4
14
39
7
3
1
3
18
5
4
3
12
12
1
1
3
1
11
142
Missing Domains: Internet, Maintenance and Diagnostics, Spacecraft bus
Notes:
SRDR: Software Resources Data Report
USC CSSE Annual Research Review - Mar 2010
5
University of Southern California
Center for Systems and Software Engineering
Preliminary Results - Do Not Use!
Ground System Architecures Workshop 2010
6
University of Southern California
Center for Systems and Software Engineering
Preliminary Results - Do Not Use!
USC CSSE Annual Research Review - Mar 2010
7
University of Southern California
Center for Systems and Software Engineering
Preliminary Results - Do Not Use!
Ground System Architecures Workshop 2010
8
University of Southern California
Center for Systems and Software Engineering
Preliminary Results - Do Not Use!
Ground System Architecures Workshop 2010
9
University of Southern California
Center for Systems and Software Engineering
Preliminary Results - Do Not Use!
Ground System Architecures Workshop 2010
10
University of Southern California
Center for Systems and Software Engineering
Preliminary Results - Do Not Use!
Ground System Architecures Workshop 2010
11
University of Southern California
Center for Systems and Software Engineering
Preliminary Results - Do Not Use!
Ground System Architecures Workshop 2010
12
University of Southern California
Center for Systems and Software Engineering
Preliminary Results - Do Not Use!
Ground System Architecures Workshop 2010
13
University of Southern California
Center for Systems and Software Engineering
Simple Cost Estimating Relationships
PM = A * (EKSLOC)B
Preliminary Results - Do Not Use!
Notes:
PM: Person Months (152 labor hours / month)
EKSLOC: Equivalent Thousands of Source Lines of Code
USC CSSE Annual Research Review - Mar 2010
14
University of Southern California
Center for Systems and Software Engineering
Sizing Issues -1
• Multiple SLOC counting methods
– Physical: total number of lines in a file
– Non-commented Source: no blank or comment lines
– Logical
• No Deleted Code Counts
• SLOC Conversion Experiment
–
–
–
–
Use the results of USC’s Code Count Tool to find conversion ratios
Physical to Logical
NCSS to Logical
Results segregated by programming language
USC CSSE Annual Research Review - Mar 2010
15
University of Southern California
Center for Systems and Software Engineering
NCSS to Logical Conversion
Ada: 45%
C/C++: 61%
C#:
61%
Java: 72%
USC CSSE Annual Research Review - Mar 2010
16
University of Southern California
Center for Systems and Software Engineering
Sizing Issues -2
• No Modified Code parameters
–
–
–
–
–
Percent Design Modified (DM)
Percent Code Modified (CM)
Percent Integration and Test Modified (IM)
Software Understanding (SU)
Programmer Unfamiliarity (UNFM)
• Program interviews provided parameters for some records
USC CSSE Annual Research Review - Mar 2010
17
University of Southern California
Center for Systems and Software Engineering
Effort Issues
• Missing effort reporting for different lifecycle phases
–
–
–
–
–
–
Software requirements analysis (REQ)
Software architectural design (ARCH)
Software coding and testing (CODE)
Software integration (INT)
Software qualification testing (QT)
Software management, CM, QA, etc. (Other – very inconsistent)
USC CSSE Annual Research Review - Mar 2010
18
University of Southern California
Center for Systems and Software Engineering
Collapsing Application Domains
• Propose to reduce the number of application domains
– Currently have a “sparse” data table
• Use a model-independent approach
– 5-level scale to capture the “difficulty” (and therefore impact) of an
application domain on productivity
USC CSSE Annual Research Review - Mar 2010
19
University of Southern California
Center for Systems and Software Engineering
Software Application Difficulties
Difficulty would be described in terms of required software reliability, database
size, product complexity, integration complexity, information assurance, realtime requirements, different levels of developmental risks, etc.


USC CSSE Annual Research Review - Mar 2010
Attitude Control
Radar / Sonar / Telemetry
processing
Process Control
Seismic Processing
Factory Automation
Digital Switches & PBX

Firmware (ROM)

Very Challenging
Autonomous
operation
Extremely high
reliability
Complex algorithms
Micro-second
control loop
Micro-code
programming
Guidance / Navigation

Case Tools / Compilers
Website Automation
Business Systems
Very Easy
 Simple handling of
events / inputs
 Relies on O/S or
middleware for
control
 Can be restarted
with minor
inconvenience
20
Application Difficulty Issues
Application Domains
Very Easy
Nominal
Challenging
Large biz
system
Business Systems
Internet
Easy
Simple web
pages
Trillion $/day
transaction
Web application
(shopping)
Tools and Tool Systems
Mega-web
application
Verification tools
Safety critical
Scientific Systems
Offline data
reduction
Large dataset
Simulation and Modeling
Low fidelity
simulator
Physical
phenomenon
Test and Evaluation
Usual
Distributed
debugging
Training
Command and Control
Very Challenging
Simulation
network
Set of screens
Taxi-cab
dispatch
SOS (C4ISR)
Mission Management
Usual
Multi-level security
and safety
Weapon Delivery and
Control
Weapon space
Safety
Communications
Noise,
anomalies
handling
Radio
Safety/Security
Frequencyhopping
USC CSSE Annual Research Review - Mar 2010
21
University of Southern California
Center for Systems and Software Engineering
For more information, contact:
Thomas Tan
thomast@usc.edu
626-617-1128
Or
Brad Clark
bkclark@usc.edu
703-754-0115
Questions?
USC CSSE Annual Research Review - Mar 2010
22