Software Maintenance Influence Factors – Delphi
SOFTWARE OPERATIONS, MAINTENANCE AND SUSTAINMENT
INFLUENCE FACTORS
DELPHI ROUND 1
I. Participant Information:
Multiple attendees at COCOMO forum. Process was:
- Each attendee rated each factor
- Group jointly discussed which factors were important
- Group ranked the most important factors
Years of experience in cost modeling and estimation: 15 – 27 years (average 22)
Years of experience with software operations, maintenance, and sustainment: 5-39 years
(average 26)
What category would best describe your application domain? (Check all that apply)
Military - Airborne
Military - Ground
Military - Missile
Military - Sea
Military - Simulation
Military - Space
Military - Support
Military – Trainers
Commercial – General
Information Systems
Medical Systems
Process Control
Software Products
System Software
Telecommunications
Commercial - Aerospace
Web
Other (please specify): Military and IT system, NASA, Battle Command Sensors
Contact Information:
Donald J. Reifer
Reifer Consultants LLC
Email: don@reifer.com
Phone: (928) 237-9060
Cheryl Jones
U.S. Army/Picatinny Arsenal
Email: Cheryl.jones5@us.army.mil
Phone: (973) 724-2644
Fax: (973) 724-2382
Page 1 of 7
Software Maintenance Influence Factors – Delphi
Introduction
In an effort to better support software operations, maintenance, and sustainment estimation and
budgeting, the U.S. Army is endeavoring to develop new and more accurate models for estimating
costs. In this Delphi exercise, we need your help to determine what factors influence the costs of
software operations, maintenance, and sustainment so that we can control variation and develop
more accurate estimates.
Software operations, maintenance and sustainment for this survey is defined to include the four
major activities identified as follows:
 Software maintenance: those activities associated with modifying a software product/system
or component after delivery to correct faults, improve performance or attributes, or adapt it to a
changed environment.
 Software sustaining engineering: those activities associated with supporting a deployed
software product/system in its operational environment.
 Software support infrastructure and facilities: those infrastructure tasks associated with
managing the integrity and quality of software releases and the processes, suppliers and their
products, and the facilities used to develop them as they are generated, distributed and fielded.
 Program/project management: all project, organizational and enterprise activities associated
with planning, organizing, funding, and managing software support resources to achieve
mission and performance objectives.
Factors that have a direct impact on operations, maintenance, and sustainment costs estimated in
pursuit of these four activities at both the enterprise and program/project levels include influence
factors, size drivers and effort multipliers. Influence factors are parameters outside the
program/project’s control. Size and effort multipliers are factors that the manager has some power
over.
1. Instructions
The questionnaire consists of three parts: influence factors, size drivers and effort multipliers. We
would appreciate your rating the importance of each candidate for these factors, drivers and
multipliers using a scale of 1 to 10, with 10 being the most important.
2. Software Operations, Maintenance and Sustainment Influence Factors
Factors that influence estimates are many and stem from the business complexity, task alignment
and execution models imposed on the program/project organizationally from outside sources as
illustrated in the Figure below. Please rate these factors in the paragraphs that follow:
Page 2 of 7
Software Maintenance Influence Factors – Delphi
Business Factors (rate important using scale of 1 to 10 with 10 being most important)
Program and domain characteristics #1
Color of money #2 (not for commercial programs)
Source of funds #3
Other (write-in): Changing Mission, users #4
Low level technical and business decision autonomy #5 (drives cost is don’t have authority)
Resourcing business models #6 (having staff in place, going from contractor to govt)
Extent of policy coverage – governance
Diverse organizational task and activity portfolios
Product and data rights
Estimation/budgeting approaches
Information system capabilities (including accounting systems)
Other _______________________________
Complexity Factors (rate important using scale of 1 to 10 with 10 being most important)
System of system integration requirements #1
Legacy software architectures #2
Backfit security, safety, and other certification requirements #3 (2 were merged for this)
Legacy software technologies
Other (write-in): Level of hw/sw integration (little “m”)
Other _______________________________
Page 3 of 7
Software Maintenance Influence Factors – Delphi
Resource and Task Alignment Factors (rate important using scale of 1 to 10 with 10 being most
important)
Alignment of task and funding models #1
Top-level expectations versus realities #2 (must for model) (sked)
Policies, budgets, resources, tasks and outputs
Autonomous personnel and funding decisions
Plans versus execution
Management reserves (reduced allocations)
Overhead versus direct funded functions
Amount of “technical debt”
(this may be the wrong place for this –should be in little “m”)
Other _______________________________
Other _______________________________
Execution Factors (rate important using scale of 1 to 10 with 10 being most important)
Event-driven requirements and reprioritizations #1
No. of releases being done in parallel (using same resources) #2
Multiple customers – direct user involvement #3 (lack of, too much, joint program)
Short-term mission driven execution schedules
Multiple funding streams
(little “m” – skeds dictated – can’t affect it)
(goes with multiple customers)
Backlog at start of release
Uncertainty of planning parameters
Organizational capability – flexibility – stability
Other _______________________________
Other _______________________________
Page 4 of 7
Software Maintenance Influence Factors – Delphi
Software Maintenance
Term refers to those software activities associated with modifying a software product/system or
component after delivery to correct faults, improve performance or attributes, or adapt it to a
changed environment.
3. Software Maintenance Size Drivers
The size of the software maintenance tasks is directly influenced by factors like the how many
programs/ projects are competing for a limited number of resources (labs, people, etc.) at the
enterprise level and the number of changes and repairs that have to be incorporated into the release
at the program/project level. Please rate these factors in the paragraphs that follow:
Enterprise Size Factors (rate important using scale of 1 to 10 with 10 being most important)
Number of programs/projects competing for resources
Stability of releases measured by change rate
(size driver)
(cost driver)
Number of releases being supported being supported by program/project
Stability of core funding
(cost driver)
(unstable – drives to contractor – lose personnel – expensive)
Other _______________________________
Other _______________________________
Other _______________________________
Program/Project Size Factors (rate important using scale of 1 to 10 with 10 being most
important)
Number of change requests (in release) #1a (input queue – merge top 3 into “scope”
Number of repairs (in release) by priority and type #1b
Number of COTS packages updated #1C
Number of emergency repairs being replaced (patches) #2 (if good, have less of these)
Other _______________________________
Other _______________________________
Other _______________________________
Page 5 of 7
Software Maintenance Influence Factors – Delphi
4. Software Maintenance Effort Multipliers
The effort needed to perform the software maintenance tasks is influenced first by the work that
needs to be performed and those factors or drivers to which effort is sensitive (effort multipliers).
For software maintenance, please rate the factors which follow to provide us with assessments of
which have the greatest impact on effort. Effort multipliers have been taken from COCOMO II
and SEER and are defined by texts describing them.
People (rate important using scale of 1 to 10 with 10 being most important)
Personnel continuity #1 (if maintainer is different than developer, need to affect personnel)
Analyst capability #2
Applications experience #3
Platform experience #4
Programmer capability
Language & tool experience
Other _______________
Product (rate important using scale of 1 to 10 with 10 being most important)
Product complexity #1
Required reliability #2
Other (write-in): Required Security #3
Application type #4
Other (write-in): Hw/sw integration #5
Platform/domain type
Degree of reuse
Language type
Documentation match to needs
Requirements volatility (change)
Execution time constrain
Main storage constrain
Data base size
Platform volatility
Page 6 of 7
Software Maintenance Influence Factors – Delphi
Project (rate important using scale of 1 to 10 with 10 being most important)
Architecture/risk resolution #1
Team cohesion #2
Constrained schedule #3 (changed from “Required development schedule” – creates technical
debt – could also be “Degree of Technical Debt”)
Degree of precedentedness
Use of software tools
Development flexibility
Multi-site capability
Resource dedication
Other _______________
Other _______________
Process (rate important using scale of 1 to 10 with 10 being most important)
Process volatility (change) #1
Use of modern programming practices #2
Process maturity
measurement)
#3 (not CMMI – spirit of process maturity – quality & performance
Acquisition method
Development method
Development standard
Other _______________
Other _______________
Page 7 of 7