Software Maintenance Influence Factors – Delphi
SOFTWARE OPERATIONS, MAINTENANCE AND SUSTAINMENT
INFLUENCE FACTORS
DELPHI ROUND 1
I. Participant Information:
Your Name:
Organization name:
Division:
Location (City, State):
Email address:
Phone:
Date prepared:
Years of experience in cost modeling and estimation:
Years of experience with software operations, maintenance, and sustainment:
What category would best describe your application domain? (Check all that apply)
Military - Airborne
Military - Ground
Military - Missile
Military - Sea
Military - Simulation
Military - Space
Military - Support
Military – Trainers
Commercial – General
Information Systems
Medical Systems
Process Control
Software Products
System Software
Telecommunications
Commercial - Aerospace
Web
Other (please specify):
Contact Information:
Donald J. Reifer
Reifer Consultants LLC
Email: don@reifer.com
Phone: (928) 237-9060
Cheryl Jones
U.S. Army/Picatinny Arsenal
Email: Cheryl.jones5@us.army.mil
Phone: (973) 724-2644
Fax: (973) 724-2382
Page 1 of 6
Software Maintenance Influence Factors – Delphi
Introduction
In an effort to better support software operations, maintenance, and sustainment estimation and
budgeting, the U.S. Army is endeavoring to develop new and more accurate models for estimating
costs. In this Delphi exercise, we need your help to determine what factors influence the costs of
software operations, maintenance, and sustainment so that we can control variation and develop
more accurate estimates.
Software operations, maintenance and sustainment for this survey is defined to include the four
major activities identified as follows:
 Software maintenance: those activities associated with modifying a software product/system
or component after delivery to correct faults, improve performance or attributes, or adapt it to a
changed environment.
 Software sustaining engineering: those activities associated with supporting a deployed
software product/system in its operational environment.
 Software support infrastructure and facilities: those infrastructure tasks associated with
managing the integrity and quality of software releases and the processes, suppliers and their
products, and the facilities used to develop them as they are generated, distributed and fielded.
 Program/project management: all project, organizational and enterprise activities associated
with planning, organizing, funding, and managing software support resources to achieve
mission and performance objectives.
Factors that have a direct impact on operations, maintenance, and sustainment costs estimated in
pursuit of these four activities at both the enterprise and program/project levels include influence
factors, size drivers and effort multipliers. Influence factors are parameters outside the
program/project’s control. Size and effort multipliers are factors that the manager has some power
over.
1. Instructions
The questionnaire consists of three parts: influence factors, size drivers and effort multipliers. We
would appreciate your rating the importance of each candidate for these factors, drivers and
multipliers using a scale of 1 to 10, with 10 being the most important.
2. Software Operations, Maintenance and Sustainment Influence Factors
Factors that influence estimates are many and stem from the business complexity, task alignment
and execution models imposed on the program/project organizationally from outside sources as
illustrated in the Figure below. Please rate these factors in the paragraphs that follow:
Page 2 of 6
Software Maintenance Influence Factors – Delphi
Business Factors (rate important using scale of 1 to 10 with 10 being most important)
Extent of policy coverage – governance
Low level technical and business decision autonomy
Diverse organizational task and activity portfolios
Program and domain characteristics
Product and data rights
Source of funds
Color of money
Resourcing business models
Estimation/budgeting approaches
Information system capabilities (including accounting systems)
Other _______________________________
Other _______________________________
Complexity Factors (rate important using scale of 1 to 10 with 10 being most important)
Legacy software architectures
Legacy software technologies
Backfit security requirements
Backfit safety and other certification requirements
System of system integration requirements
Other _______________________________
Other _______________________________
Page 3 of 6
Software Maintenance Influence Factors – Delphi
Resource and Task Alignment Factors (rate important using scale of 1 to 10 with 10 being most
important)
Policies, budgets, resources, tasks and outputs
Autonomous personnel and funding decisions
Plans versus execution
Alignment of task and funding models
Management reserves (reduced allocations)
Top-level expectations versus realities
Overhead versus direct funded functions
Amount of “technical debt”
Other _______________________________
Other _______________________________
Execution Factors (rate important using scale of 1 to 10 with 10 being most important)
Event-driven requirements and reprioritizations
Short-term mission driven execution schedules
Multiple customers – direct user involvement
Multiple funding streams
No. of releases being done in parallel (using same resources)
Backlog at start of release
Uncertainty of planning parameters
Organizational capability – flexibility – stability
Other _______________________________
Other _______________________________
Page 4 of 6
Software Maintenance Influence Factors – Delphi
Software Maintenance
Term refers to those software activities associated with modifying a software product/system or
component after delivery to correct faults, improve performance or attributes, or adapt it to a
changed environment.
3. Software Maintenance Size Drivers
The size of the software maintenance tasks is directly influenced by factors like the how many
programs/ projects are competing for a limited number of resources (labs, people, etc.) at the
enterprise level and the number of changes and repairs that have to be incorporated into the release
at the program/project level. Please rate these factors in the paragraphs that follow:
Enterprise Size Factors (rate important using scale of 1 to 10 with 10 being most important)
Number of programs/projects competing for resources
Number of releases being supported being supported by program/project
Stability of releases measured by change rate
Stability of core funding
Other _______________________________
Other _______________________________
Other _______________________________
Program/Project Size Factors (rate important using scale of 1 to 10 with 10 being most
important)
Number of change requests (in release)
Number of repairs (in release) by priority and type
Number of emergency repairs being replaced (patches)
Number of COTS packages updated
Other _______________________________
Other _______________________________
Other _______________________________
Page 5 of 6
Software Maintenance Influence Factors – Delphi
4. Software Maintenance Effort Multipliers
The effort needed to perform the software maintenance tasks is influenced first by the work that
needs to be performed and those factors or drivers to which effort is sensitive (effort multipliers).
For software maintenance, please rate the factors which follow to provide us with assessments of
which have the greatest impact on effort. Effort multipliers have been taken from COCOMO II
and SEER and are defined by texts describing them.
People (rate important using scale of 1 to 10 with 10 being most important)
Analyst capability
Applications experience
Programmer capability
Platform experience
Personnel continuity
Language & tool experience
Other _______________
Other _______________
Product (rate important using scale of 1 to 10 with 10 being most important)
Platform/domain type
Required reliability
Application type
Degree of reuse
Language type
Documentation match to needs
Requirements volatility (change)
Execution time constrain
Product complexity
Main storage constrain
Data base size
Platform volatility
Other _______________
Other _______________
Project (rate important using scale of 1 to 10 with 10 being most important)
Degree of precedentedness
Use of software tools
Development flexibility
Multi-site capability
Architecture/risk resolution
Required development schedule
Team cohesion
Resource dedication
Other _______________
Other _______________
Process (rate important using scale of 1 to 10 with 10 being most important)
Acquisition method
Use of modern programming practices
Development method
Process maturity
Development standard
Process volatility (change)
Other _______________
Other _______________
Page 6 of 6