AFFORDABILITY MANAGEMENT ASSESSMENT
Objective
The objective of the Affordability Management Assessment is to offer a standardized approach that
promotes Affordability Management excellence in an organization. The process is to identify specific
areas (tenets) of Affordability Management to be measured, and then to compare performance in these
areas against established qualitative criteria. This process determines the level of achievement of Affordability Management excellence.
This single set of 20 tenets facilitates an assessment of the level of Affordability Management excellence
at both the program and the enterprise levels. All of these principles apply to programs, and approximately half apply to the enterprise. Unique applications of tenets common to both the program and the
enterprise are identified in the descriptions of the levels of achievement to be attained.
The objective of a program-level Affordability Management assessment is to measure the extent to
which a given program is implementing the Affordability Management Tenets. Specific implementation
is tailored to meet customer and stakeholder requirements.
The objective of an enterprise-level Affordability Management assessment is to measure the extent to
which the enterprise supports its programs in implementing the Affordability Management Tenets
across the enterprise. Programs that are deemed highly effective in selected areas can serve as models
for programs that struggle in these same areas. The ultimate goal is improved performance across the
enterprise.
Definition
Each of the 20 tenets (see Appendix A) has five rating levels (Level 1 to Level 5) representing increasing
capabilities with respect to the tenet. The ultimate goal is to achieve a Level 5 in each of the 20 tenets.
Characteristics of each level are written in general terms and are shown conceptually in Figure 1.
Every program or enterprise is able to claim credit for Level 1. On a scorecard (see Table 1), achievement
of a Level 1 in all 20 tenets is worth 20 points with a score of 100 representing total and complete Affordability Management excellence in all tenets. In other words, 20 points are awarded for process participation (20 tenets * Level 1). As capabilities mature, scores increase. Although the approach described here has yet to be used, the current judgment is that, using by systematic improvement efforts,
low initial scores can improve at the rate of twenty points per year while higher scores at the rate of ten
points per year until further improvement is deemed not value added. Therefore, improvement only
comes with concentrated effort. The methodology whereby assessments are made and improvement
planning occurs is covered in the following sections.
1
Level
Level Characteristics
5
Seamless, transparent excellence, automatic, so
good it’s hard to believe it could happen
4
World--class, outstanding, not quite always
automatic, little management action required
3
Systems installed to ensure very good performance
with an occasional glitch (sometimes serious)
2
Awareness established of what good is, first small
steps taken, still many big problems
1
The usual mess, traditional management by
direction, reactive, few or bad systems, many
problems, most accepted as “oh, well....”
Figure 1: Affordability Management Tenet Level Characteristics
2
Table 0: Affordability Management Tenet Scorecard
Tenet #
Title
1
Understanding the Customer/User
2
PTW Process and Technical Approach
Level 1
(1 Point)
3
Requirements Flexibility
4
KPPs and Requirements
5
Cost of Requirements
6
Cost Drivers
7
Cost Targets
8
Architecture and Cost Baselines
9
Subcontractor Involvement
10
Rewards
11
Continuous Cost Reduction
12
Life Cycle Costs and the Trade Process
13
Cost Database and Its Usage
14
Parametric Cost Models
15
Constraint /Risk Identification and Management
16
Integrated Tools
17
Trade Study Levels
18
Selection and Optimization
19
Cost Reviews
20
Affordability Measures
Total Score (100 Points Maximum) >
3
Level 2
(2 Points)
Level 3
(3 Points)
Level 4
(4 Points)
Level 5
(5 Points)
Charting the status of progress toward attaining excellence in Affordability Management in all 20 tenets
brings to focus areas needing attention in order to achieve the improvement goals. Capturing the status
on a chart similar to that shown in Figure 2 illustrates which of the 20 tenets requires more attention.
Note that small improvements in many principles achieve greater over-all ratings over a shorter time
period than do fewer improvements in more difficult areas. Current stages of improvement can be
plotted to show how planned improvement is progressing toward the original goals.
Figure 2: Affordability Management Tenet Status Chart
4
Methodology
Overall Description
The process by which improvement occurs relies on the power of the Affordability Management Tenets
methodology to identify the areas that must be improved by unifying, combining and integrating all implementation activities under a single umbrella. All levels of the organization are given one scorecard
(Table 1) that is easy to understand. Every unit (e.g. group, area, office, or cell) is allowed to develop its
own action plan for Affordability Management improvement, and everyone is encouraged to be involved and active. Long-term management involvement and support is required. The planning and action cycle to apply the Affordability Management Tenets is illustrated in Figure 3. A 15-to-30 minute
orientation session is held with each work group (unit). Each work group then assesses its status. Enterprise leaders, department managers, and program managers then set long-term (3 to 4 years) point
goals that every unit must meet. Each unit selects the tenets and levels on which it will focus during a
specified time. Under the guidance of a supervisor, each unit develops an action plan for attaining each
additional point. Enterprise leaders, department managers, and program managers conduct periodic
audits and visits to congratulate and to coach. This approach can be adjusted as needed, and eventually, improvement is ingrained in the units.
5
The 20 Tenet Planning Cycle
…and Prioritize Our 20
Tenets
1.
Understanding the Customer/User
2.
PTW Process and Technical Approach
3.
Requirements Flexibility
4.
KPPs and Requirements
5.
Cost of Requirements
6.
Cost Drivers
7.
Cost Targets
8.
Architecture and Cost Baselines
9.
Subcontractor Involvement
10. Rewards
11. Continuous Cost Reductions
12. Life Cycle Costs and the Trade Process
13. Cost Database and It’s Usage …..
Figure 3: The Affordability Management Tenet Application Cycle
Specific Measures for Affordability Management Excellence
A complete definition table, outlining the steps to Affordability Management excellence, is provided in
Appendix A. Initial and follow-on capability assessments are made against the description for each tenet. An example of the application of this approach for one of the 20 Affordability Management Tenets,
Integrated Tools, is shown in Figure 4.
6
Tenet #16
Figure 4: Example - Affordability Management Tenet Level Definitions
Organizational Roles
Evaluation Process Organizational Roles
The Affordability Management Tenet evaluation process needs firm commitments from senior-level
management to ensure the assessment process addresses both the enterprise and its programs. This
may include the designation of an overall Affordability Management Tenets application process manager and/or focus team.
Beyond the top-down focus, functional and program management need to ensure that commitments
are made to support the Affordability Management Tenets assessment effort. The functional areas expected to be involved in leading Affordability Management Tenets assessments and process improvements are systems engineering, product development, cost estimating, business development, subcontract management and information technologies. Within a program, the systems engineers and cost
analysts may lead the Affordability Management Tenets assessments and process improvement recommendations.
Evaluation Organizational Roles
A team, consisting of management and functional personnel, perform the Affordability Management
Tenets assessments. A notional sharing of responsibility for this assessment is illustrated in Table 2.
7
This sharing can be augmented by other enterprise, functional and program personnel as necessary.
There is a direct correlation between the commitment that is made by management to this approach
and the improvement associated with the application of the Affordability Management Tenets methodology.
Tenet
#
1
2
Title
Lead
Support
Understanding the Customer/User
PTW Process and Technical
Approach
Program Management
Business Development, Systems Engineering,
Cost Estimating
Program Management, Systems Engineering,
Product Development, Cost Estimating
Business Development
(PTW)
Systems Engineering (Technical Approach)
Systems Engineering
Systems Engineering
Systems Engineering
Cost Estimating
Enterprise Management
3
4
5
6
7
Requirements Flexibility
KPPs and Requirements
Cost of Requirements
Cost Drivers
Cost Targets
8
Architecture and Cost Baselines
9
Subcontractor Involvement
Systems Engineering, Cost
Estimating
Subcontract Management
10
11
Rewards
Continuous Cost Reduction
Enterprise Management
Program Management
12
Enterprise Management
16
Life Cycle Costs and the Trade
Process
Cost Database and Its Usage
Parametric Cost Models
Constraint/Risk Identification
and Management
Integrated Tools
17
18
Trade Study Levels
Selection and Optimization
Systems Engineering
Systems Engineering
19
Cost Reviews
Enterprise Management
20
Affordability Measures
Enterprise Management
13
14
15
Cost Estimating
Cost Estimating
Program Management
Enterprise Management
Program Management, Enterprise Management
Program Management, Business Development
Cost Estimating
Systems Engineering, Product Development
Business Development, Program Management,
Cost Estimating
Program Management
Program Management, Systems Engineering, Cost
Estimating
Program Management
Systems Engineering, Cost Estimating, Product
Development
Systems Engineering, Cost Estimating, Product
Development
Enterprise Management
Information Technologies
Systems Engineering, Cost Estimating, Product
Development
Systems Engineering, Cost Estimating, Information
Technologies
Product Development
Product Development, Cost Estimating, Information
Technologies
Program Management, Systems Engineering, Cost
Estimating
Program Management, Systems Engineering, Cost
Estimating
Table 2: Affordability Management Tenets Assessment Organizational Roles
8
Improvement Plan Development
In defining a plan, the first step is to establish the long-range goals as shown in Figure 2 and the total
number of years for implementation. From this the average number of points to be earned annually in
order to achieve the improvement goals can be calculated.
Average Annual Pts/Year = points assigned to goal – points assigned to current status
Year n
n = year of plan, in which goal to be attained - i.e. 1, 2, 3, etc.
Sorting the 20 tenets by Average Annual Points/Year, from high to low, places tenets where great strides
might be made in relatively short periods of time at the top of the list. Focusing on the tenet at the top
of this list, early in the plan, results in rapid advancement in goal attainment ratings. For each tenet, a
formal plan should be developed which shows the tasks to be completed, the timeline for completion,
and the person(s) responsible for task completion.
An example of a plan to improve Tenet 9 – Subcontractor Involvement, is shown in Figure 5. This plan
delineates the specific activities, by weeks, which are required to improve the tenet level assessment
rating within the schedule constraints.
TTenet
enet 99
Figure 5: Example - Affordability Management Tenet Improvement Plan
Controls and Reviews
Each enterprise establishes its own approach to controls and reviews for the assessment and improvement utilizing the Affordability Management Tenets. This process, as explained earlier, takes 3 to 5
years to advance 50 to 60 points, even with aggressive, organized efforts. Initially monthly, bimonthly,
or quarterly reviews may be required. Once the improvement process is established, semi-annual re9
views may suffice. Whatever schedule of review is adopted, it is incumbent upon enterprise management to be the driving force in seeing that improvement occurs. Enterprise management should expect
both functional management and program management to report at the Affordability Management
Tenets Improvement Reviews. An incentive program commensurate with goal achievement may maintain the impetus for the improvement process. Award criteria may include goal attainment ahead of
schedule.
10
Appendix A Affordability Management Tenet Definition Table
Tenet
Number
1
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Understanding
the Customer/User
Understanding of customer/user values and
needs is derived from
program documentation
(e.g., ORD, O&O, SOO,
spec, etc). Contact is
encouraged to clarify
these documents.
Understanding of customer/user values and
needs is derived from
program documentation
and minimal interaction
with customer program
acquisition personnel. No
direct contact with users
results in a basic understanding of program
needs.
Understanding of customer/user values and
needs is derived from
program documentation,
interaction with customer
program acquisition personnel and some contact
with users. Project team
has solid understanding
of the customer's program acquisition values
and a good understanding of users' needs.
Understanding of customer/user values and
needs is derived from
program documentation
and significant interaction
with customer program
acquisition personnel and
users. Project team understands all customer/user values and needs.
Project team also has
basic understanding of
external elements influencing the program (e.g.,
political, international,
system-of-systems view).
Project team has examined, defined and fully
understands customer/user values and needs.
All customers, users, and
external influences have
been identified and were
engaged early in the
project process. Project
team and customer interaction has resulted in
beneficial adjustments to
the program prior to acquisition.
11
Tenet
Number
2
Title
Level 1
Level 2
Level 3
Level 4
Level 5
PTW Process
and Technical
Approach
Business Development is
solely responsible for
developing the PTW
based on market analyses, perceived customer
funding and desires, and
product familiarity. Minimal contact occurs with
the technical staff prior to
actual proposal writing.
A multi-functional team
led by Business Development prepares the
PTW based on market
analyses, perceived customer funding and desires, and product familiarity. Resources are in
place for engineering and
cost analysts to become
involved pre-proposal to
fine-tune the PTW.
A multi-functional team
led by Business Development prepares the
PTW based on market
analyses, perceived customer funding and desires, product familiarity
and cost iterations. Resources are in place for
engineering and cost
analysts to iterate toward
the PTW value preproposal. The technical
approach is validated by
engineering pre-proposal.
A multi-functional team
led by Business Development prepares the
PTW based on market
analyses, perceived customer funding and desires, product familiarity
and cost iterations. Resources are in place for
engineering and cost
analysts to iterate toward
the PTW value preproposal. The technical
approach is validated by
engineering pre-proposal.
Risk and sensitivity analyses are included to identify potential pitfalls associated with decisions and
the threshold of bid/nobid.
A multi-functional team
led by Business Development prepares the
PTW based on market
analyses, perceived customer funding and desires, product familiarity
and cost iterations. The
market analysis includes
extensive studies to understand the goodness of
competitors’ capabilities.
Resources are in place
for engineering and cost
analysts to iterate toward
the PTW value preproposal. The technical
approach is validated by
engineering pre-proposal.
Risk and sensitivity analyses are included to identify potential pitfalls associated with decisions and
the threshold of bid/nobid.
12
Tenet
Number
3
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Requirements
Flexibility
Requirements are met
during the execution of a
program. While some
may be shown to cause
cost growth, there is not a
documented procedure
for challenging the requirement(s). The enterprise leads program
managers to satisfy customer/user at all times.
A plan to develop the
process to define notional
"threshold" and "goal"
values for requirements
(as appropriate) is formulated. Dialogue is occurring between program
management and the
customer/user to enter
into requirements tailoring
based on costperformance trade studies that address cost
drivers (Tenet 6). Customer/user reluctance to
relax requirements is
noted.
The process to define
notional "threshold" and
"goal" values for requirements (as appropriate) is
being applied to a limited
extent. Dialogue is encouraged (although not
required) between program management and
the customer / user to
enter into requirements
tailoring based on costperformance trade studies that address cost
drivers (Tenet 6). Enterprise rewards program
managers for surfacing
requirements change to
customer/user. Customer/user reluctance to
relax requirements is
noted.
The process to define
notional "threshold" and
"goal" values for requirements (as appropriate) is
being applied across all
programs. Dialogue is
required between program management and
the customer/user to
enter into requirements
tailoring based on costperformance trade studies that address the cost
drivers (Tenet 6). Enterprise rewards program
managers for surfacing
requirements change to
customer/user. Customer/user reluctance to
relax requirements is
strongly challenged.
Customer and user establish "threshold" and
"goal" values to allow
identification of optimal
alternatives. Customer,
user and contractor actively discuss and relax
requirements based on
sound trade studies.
Enterprise rewards program managers for successful requirements
change that have been
surfaced to customer /
user.
13
Tenet
Number
4
Title
Level 1
Level 2
Level 3
Level 4
Level 5
KPPs and Requirements
Customer and user are
aware of performance
specification guidance
(MIL-STD-967) to minimize numbers of KPPs
and fixed constraints.
A plan to identify overly
constrained problems that
may provide insight into
overly constrained requirements and excessive KPPs is formulated.
The optimal solution approach of reducing the
numbers of constraints is
introduced to the customer on a case–by- case
basis. Customer/user is
encouraged to consider
and reveal to the subcontractor what may be unwritten absolute, rigid
requirements
The plan to identify overly
constrained problems that
may provide insight into
overly constrained requirements and excessive KPPs is applied to a
limited extent. Costs of
requirements are beginning to be calculated and
the benefits in the optimal
solution approach can be
shown. Customer/user is
expected to consider and
reveal to the contractor
what may be unwritten
absolute, rigid requirements and show a willingness to limit these.
The plan to identify overly
constrained problems that
may provide insight into
overly constrained requirements and excessive KPPs is applied
across all programs.
Costs of requirements are
calculated for the most
part and the benefits of
the optimal solution approach are clearly explained. Customer/user
is engaged in limiting
unchangeable, fixed constraints, including those
that may be unwritten.
Customer, user and contractor are actively engaged in limiting numbers
of KPPs and fixed constraints to allow best
opportunity for optimal
solution. Cost of requirements is available
for consideration by the
team.
14
Tenet
Number
Title
Level 1
Level 2
Level 3
Level 4
Level 5
5
Cost of Requirements
Cost and requirements
are managed to the subsystem level. However,
no attempt is made to
map the two and prioritization for cost is not
done.
Cost and requirements
are allocated to a common level and mapped to
the program WBS.
Cost and requirements
are allocated to a common level and mapped to
the program WBS. Cost
of requirements is determined and used to prioritize requirements for cost.
Prioritized cost of requirements is used as a
tool to separate true requirements versus “desirements” and to optimize solutions at the
system (not subsystem)
level.
Prioritized cost of requirements is used to
establish firm requirements and to minimize
implementation of “desirements”. Lessons
learned and insights are
provided to the enterprise. The enterprise
facilitates use of these
findings on other programs.
6
Cost Drivers
Identification of cost drivers occurs during program execution. However, the interdependency
of cost and requirements
is not clearly known or
quantified.
Cost drivers are quantified and ranked (prioritized). Sensitivity analyses are performed to
determine relative cost
sensitivity of the driving
requirement, technology,
and schedule.
Cost drivers identified
early. Cost influences
tied to requirements,
technology, and schedule
as appropriate. Risk
identified and managed
based on identified interdependencies.
Cost drivers identified
early. Cost influences
are quantified. Risk mitigation and opportunity
strategies are identified.
Program resources are
tailored to best balance
technical, cost, schedule,
and risk mitigation requirements.
Cost drivers identified
early. Cost influences are
quantified. Risk mitigation and opportunity
strategies are identified.
Resources are adequately applied. Lessons
learned and insights are
provided to the enterprise
for benefit to other programs.
15
Tenet
Number
Title
Level 1
Level 2
Level 3
Level 4
Level 5
7
Cost Targets
Costs are arbitrarily set at
the system and major
subsystem level. No
feedback from IPTs to
validate cost targets.
Design trade studies
quantify system and major subsystem costs during product development.
Program management
adjusts funding between
IPTs using an "equal
pain" approach to meet
targets, and uses the
management reserve as
a last measure to meet
funding shortfalls.
Costs are set at the system and major subsystem
level using data from a
cost database. No feedback from IPTs to validate cost targets. Program management establishes measures to track
costs. Design trade studies quantify system and
major subsystem costs
during product development. Program management uses measures
to adjust funding between
IPTs to meet targets, and
uses the management
reserve as a last measure
to meet funding shortfalls.
Costs are set at the system and major subsystem
level using data from a
cost database. IPTs
validate cost targets with
bottom-up feedback.
Program management
establishes measures to
track costs. Design trade
studies quantify system
and major subsystem
costs during product development. Program
management uses
measures to adjust funding between IPTs to meet
targets, and uses the
management reserve as
a last measure to meet
funding shortfalls.
Costs are set at the system and major subsystem
level using data from a
cost database. IPTs
validate cost targets with
bottom-up feedback.
Program management
establishes measures to
track costs. IPTs work
within an assigned "trade
space" to develop products. Design trade studies quantify system and
major subsystem costs
during product development. Program management uses measures
to adjust funding between
IPTs to meet targets only
when an IPT has exhausted an assigned
trade space without developing a viable product.
Program management
uses the management
reserve as a last measure
to meet funding shortfalls.
Aggressive target costs
are set prior to or during
proposal phase. Cost
targets carry over into
contract execution. Cost
targets are allocated to
the lowest level. Cost
targets provide framework for design tradeoffs.
Cost targets are clearly
owned at an appropriate
level of the organization.
Reasonableness of target
costs is measured
against cost database
and program definition
parameters. Current cost
baseline clearly visible to
appropriate program
personnel. Cost targets
and current assessments
tracked on a regular basis.
16
Tenet
Number
8
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Architecture
and Cost Baselines
Cost baselines are established using a few techniques but may vary from
program to program. The
enterprise does not provide process or procedural guidance.
Cost baselines development processes are well
documented in standard
procedures. Multiple
baselines development
techniques are established and provided by
the enterprise.
Cost baseline is reviewed
by the enterprise when
completed. Feedback
loops are established
between the reviewers
and the baseline creators.
Cost Baseline development training is provided
by the enterprise for all
program managers. Cost
baseline development
templates are provided by
the enterprise.
Tools/models are evaluated and provided by the
enterprise. Customer
requirements are compared the cost baseline.
As a matter of practice,
cost baselines are established early in a program's life cycle (preproposal) and reflect a
convergence of top-down
and bottom-up estimating
techniques. Most probable cost based on multiples of estimates is also
used. Reconciliation
involves assumptions,
risks and areas to be
explored for future cost
reduction activities. The
cost baseline is cascaded
from top-level targets to
lower sub-tiers mapping
to the WBS and maintains traceability to the
WBS, MEL, organization
and structure for the program. Enterprise cost
modeling and estimating
toolset supports all cost
baseline needs.
17
Tenet
Number
9
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Subcontractor
Involvement
Subcontractors are allocated cost and performance targets. Design
engineers and subcontract management see
that these targets are
being met during the
execution of the subcontract.
Subcontractors are allocated cost and performance targets using a
cost and performance
database. Subcontractors validate cost and
performance targets prior
to buy-in. Design engineers and subcontract
management see that
these targets are being
met during the execution
of the subcontract.
Subcontractors are allocated cost and performance targets using a
cost and performance
database. Subcontractors validate cost and
performance targets prior
to buy-in. Program management and subcontractor management establish
measures to track targets. Design engineers
and subcontract management see that these
targets are being met
during the execution of
the subcontract.
Subcontractors are allocated cost and performance targets using a
cost and performance
database. Subcontractors validate cost and
performance targets prior
to buy-in. Program management and subcontractor management establish
measures to track targets. Design engineers
and subcontract management see that these
targets are being met
during the execution of
the subcontract. Subcontractors are active participants during program
reviews.
Subcontractors are integral to the Affordability
Management Process.
Subcontractors are allocated cost and performance targets using a
cost and performance
database. Subcontractors validate cost and
performance targets prior
to buy-in. Program management and subcontractor management establish
measures to track targets. Design engineers
and subcontract management see that these
targets are being met
during the execution of
the subcontract. Subcontractors are active participants during program
reviews.
18
Tenet
Number
Title
Level 1
Level 2
Level 3
Level 4
Level 5
10
Rewards
Rewards are based solely
on the enterprise’s normal reward system that
recognizes outstanding
individual and team effort.
Individual programs within an enterprise establish
a rewards program that is
extended beyond the
normal enterprise reward
system to account for
implementation of “best
value” solutions within the
bounds of affordability.
Individual and team
awards are available.
Best features of individual
programs’ reward program institutionalized at
enterprise level to reward
employees and subcontractors for implemented
“best value” solutions
within bounds of affordability. Individual and team
awards are available.
Reward program institutionalized at enterprise
level to reward employees and subcontractors
for implemented “best
value” solutions within
bounds of affordability.
Individual and team rewards are available.
Customer and user are
encouraged to recognize
and reward their participants in programs.
Reward program institutionalized at enterprise
level to reward employees and subcontractors
for implemented “best
value” solutions within
bounds of affordability.
Freely contributed suggestions and/or ideas are
eligible for some lower
level reward. Individual
and team rewards are
available. Customer and
user actively participate in
the rewards program
through appropriate
means.
19
Tenet
Number
11
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Continuous
Cost Reduction
Continuous cost reduction suggestions and
ideas occur on a random
basis.
Continuous cost reduction suggestions and
ideas are encouraged
through Design to Value
(DTV) training.
On a subset of enterprise
programs, continuous
cost reduction suggestions and ideas are expected of program personnel on a regular basis.
Some program-specific
reward program may be
in place.
On most of an enterprise’s programs, continuous cost reduction suggestions and ideas are
expected of program
personnel on a regular
basis. The enterprise has
instituted a rewards program from the best features of program-specific
rewards programs (Tenet
10).
Continuous cost reduction is an in-bred practice
in the enterprise. Less
expensive ways to accomplish customer or
user requirements are
regularly recommended
and implemented. The
enterprise has a program
that rewards suggestions
and ideas (Tenet10).
20
Tenet
Number
12
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Life Cycle Costs
and the Trade
Process
Ad hoc trade studies are
performed but there is
neither a standard process nor a method of
assigning responsibility.
A process for performing
trade studies is standardized and documented at
the program level. Trade
studies are conducted on
a program-by-program
basis. Responsibility is
assigned and studies are
managed at the program
level.
A standardized trade
study process is defined
and managed at the enterprise level. The standardized process can be
tailored from the enterprise set of standard
practices for each program under guidelines
specified within the process. Deviations beyond
those allowed by tailoring
guidelines are documented, justified, reviewed
and approved.
Measures are assigned to
the standardized trade
study process such that
adherence to the enterprise process is measured. Measures can
consist of the number of
trade studies performed
on a program, the number of options considered
in each trade, cost avoidance realized resulting
from each trade, cost
performance index (CPI)
and schedule performance index (SPI) trend
analysis, etc. Programs
are quantitatively measured and statistics are
logged for program design/cost/schedule influence.
Standard procedure for
trade study management
is in place and expected
to be followed. Costperformance trade study
approach is part of the
design process training.
Standard trade study
formats, derived from
standard procedure, are
followed within each program. Oversight of the
trade study process is
assigned at the enterprise
level, along with the
keeping measures for all
programs. The measures
are analyzed at the enterprise level, and fed
back to the programs for
optimization of the trade
study process. An enterprise repository of trade
study findings is available
for the benefit of all programs.
21
Tenet
Number
13
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Cost Database
and Its Usage
Historical cost data are
available but difficult to
find and use. Greater
use is made of parametric
and program unique data.
The enterprise has not
assigned historical cost
data maintenance.
In each program, the
WBS has been crafted to
allow easy accumulation
of labor cost and purchased product cost data.
Labor cost and purchased cost data are
electronically available
from labor charging system and can be parsed
and imported into simple
office automation software. The enterprise has
assigned historical cost
data planning and collection to a single functional
organization.
Programs continue to use
the WBS to accumulate
and make cost data
available electronically.
The enterprise-assigned
functional organization
has defined requirements
for an historical cost database and requirements
for kinds of costs and
effort measures that will
be tracked. A plan is in
place for improving industry-wide parametric cost
models with enterpriseunique data and factors.
Programs continue to use
the WBS to accumulate
and make cost data
available electronically.
Effort has expanded to
capture program costs
across all phases. Enterprise-wide, parametric
cost models have been
customized and calibrated to reflect program
actuals, rather than industry averages. The
enterprise-wide functional
organization has developed a plan for maintaining currency.
Enterprise historical cost
data is used to develop
bottom-up cost estimates.
The historical database is
continuously expanded.
The enterprise-assigned
functional organization
creates, maps, updates
and disseminates historical cost data.
22
Tenet
Number
14
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Parametric Cost
Models
Commercial parametric
cost modeling and estimating tools are used to
generate cost estimates.
Within the enterprise,
each program is allowed
to choose its own tools
without toolset standardization.
Parametric cost models
are used to generate
program costs. The
company’s historical cost
database is used to calibrate the program cost
estimates.
Parametric cost models
are used to generate
program costs. The
company’s historical cost
database is used to calibrate the program cost
estimates. Supplemental
modeling and estimating
tools are developed to
enable faster, more accurate estimates. Tools are
shared among more than
one program, but not
enterprise-wide.
Parametric cost modeling
and estimating is an integral part of every program. The company's
historical cost data and
CERs are integrated with
CERs derived from parametric estimates through
cost modeling and estimating tools. An enterprise-wide toolset is in
place and used on every
program.
Parametric cost modeling
and estimating is an integral part of every program. The company's
historical cost data and
CERs are integrated with
CERs derived from parametric estimates through
cost modeling and estimating tools. An enterprise-wide toolset is in
place and used on every
program. Lessons
learned and insights are
provided to the enterprise
for benefit to other programs and future enhancements and improvements.
23
Tenet
Number
15
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Constraint /Risk
Identification
and Management
Risk identification and
management are applied
to programs according to
enterprise standard procedures. Constraints are
identified and managed
as appropriate.
Risk and constraint identification and management
are extended beyond the
enterprise standard procedures to address surfacing optimal solutions.
However, risk minimization remains program
focus.
Risk and constraint identification and management
are an integral part of
cost-performance trades.
Acceptance of manageable risk may occur on a
case-by-case basis.
Risk and constraint identification and management
are an integral part of
cost-performance trades.
Acceptance of manageable risk may occur on a
case-by-case basis. Risk
tolerance guidance is
beginning to surface in
the enterprise’s program
leadership training.
Risk and constraint identification and management
are an integral part of
cost-performance trades.
Acceptance of manageable risk occurs on all
programs to enable costeffective solutions. Risk
tolerance guidance is a
critical element of the
enterprise’s program
leadership training.
24
Tenet
Number
Title
Level 1
Level 2
Level 3
Level 4
Level 5
16
Integrated Tools
Cost modeling and estimating tools and requirements management
tools are used. However,
they are not electronically
linked.
Tools for performing cost
modeling and estimating
and requirements management remain at Level
1 maturity. However, a
plan now exists for integrating these tools that
will eventually achieve
Level 5 maturity, including requirements and
specifications for interfacing the disparate cost
modeling and estimating
tools.
The enterprise has begun
to execute the plan for
creating a suite of integrated cost modeling and
estimating tools with the
requirements management tools. Requirements and interface specifications, defined during
Level 2, are agreed upon
and complete, and integration of some of the
more easily integrated
tools has entered the
verification phase. Some
of the more easily integrated tools are now
executing in a cooperative manner thus proving
the integration concept
defined during Level 2.
Some programs have
begun to use the toolsets.
Integration of cost modeling and estimating tools
with the requirements
management tools continues with about 50% of
the tools now executing
cooperatively and exchanging data in a seamless manner. All programs are using the toolset to a degree commensurate with need.
Cost modeling and estimating tools tightly integrated with requirements
management tools. Enterprise has provided the
linkage, preferably electronically. All programs
use the common toolset.
25
Tenet
Number
17
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Trade Study
Levels
Trade studies are performed at the lowest level
of design implementation/component selection.
A methodology for system-level trades is rudimentary at best, perhaps
done by system level
simulations.
A plan to develop a system level trade study
process, optimizing the
results of subsystem and
component trade studies
and selections, has been
formulated. The enterprise continually monitors
industry trends such that
no levels of trade study
are plagued by obsolescence issues. The level
of trade studies is determined at the program
level.
The process of conducting trade studies to
achieve subsystem optimization is standard
throughout the organization. The enterprise plan
to conduct trade studies
such that system designs
are optimized at all levels
has been initiated. Some
pilot programs have begun the process of conducting trade studies from
the lowest to the highest
level, then refining the
result to achieve the best
value at the system level.
The enterprise has documented the methodology of employing trade
studies at all levels.
Measures are applied to
the trade study levels
such that programs can
quantitatively assess the
difference between a
rolled up subsystem trade
result and optimization
among subsystems, for
system level optimization.
Measures can include the
number of subsystems
being studied, the time
needed (cost) to conduct
the various levels of
trade, the number of options considered at each
level of trade, cost avoidance achieved by identifying the best value system, etc. Trade study
methodology, measures
and results are documented by the program.
Trade studies are conducted at all levels of the
system from the system
level down to the lowest
level where trade studies
are realizable. Systemlevel trade study methodology is deployed by the
enterprise. Responsibility
for ensuring uniform application of the systemlevel trade study methodology, along with maintaining a central repository of methodology,
measures and trade results is assigned at the
enterprise level. Results
of trade study level activity is made generally
available, and fed back to
the programs for optimization of the process.
26
Tenet
Number
18
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Selection and
Optimization
Selection and optimization of a system is performed at the subsystem
level. The system reflects a "roll up" of the
subsystems that meet
allocated costs and requirements. Costs, performance, schedule and
risk are monitored on a
regular basis.
A plan to develop a system selection and optimization methodology that
balances cost, performance, schedule and risk
has been formulated. A
procedure to direct the
tradeoff process that
assesses multiple, interrelated factors and determines their impact on
affordability and LCC has
been initiated. Costs,
performance, schedule
and risk continue to be
monitored and applied to
developing utility factors,
as appropriate.
The plan to develop a
system selection and
optimization methodology
that balances cost, performance, schedule and
risk has been initiated
and is being practiced to
a limited extent on pilot
programs. A procedure
to direct the tradeoff process that assesses multiple, interrelated factors
and determines their
impact on affordability
and LCC has been completed and documented.
Costs, performance,
schedule and risk continue to be monitored and
applied to developing
utility factors, as appropriate, for system optimization.
The plan to develop a
system selection and
optimization methodology
that balances cost, performance, schedule and
risk is complete and applied to most programs.
The tradeoff process that
assesses multiple, interrelated factors and determines their impact on
affordability and LCC is
defined and documented
in enterprise procedures.
Costs, performance,
schedule and risk continue to be monitored and
applied to developing
utility factors and
weighting factors, as
appropriate, for system
optimization.
A system selection and
optimization methodology
that balances cost, performance, schedule and
risk has been developed
and is applied to every
program. The tradeoff
process that assesses
multiple, interrelated factors and determines their
impact on affordability
and LCC is defined and
documented in enterprise
procedures. Utility functions and weighting factors are used to optimize
the system in this process
based on collected costs,
performance, schedule,
and risk data.
27
Tenet
Number
Title
Level 1
Level 2
Level 3
Level 4
Level 5
19
Cost Reviews
Cost reviews occur at the
program level in conjunction with normal program
review events and are
usually "statusing"
events. Enterprise review
of costs occurs for programs with cost "problems".
Cost reviews occur at the
program level in conjunction with normal program
review events and are
detail-oriented events.
Enterprise review of costs
occurs for programs with
cost "problems". A dedicated cost control person
is assigned to the program.
Enterprise and programlevel cost reviews are
required at regularly
scheduled intervals beyond the normal program
reviews. Updates are
required on a weekly or
biweekly basis. A dedicated cost control person
is assigned to all programs.
Enterprise and programlevel cost reviews are
required at regularly
scheduled intervals beyond the normal program
reviews. Updates are
required on a weekly or
biweekly basis. A dedicated cost control person
is assigned to all programs. Cost containment
plans for out-of-tolerance
cost items are required.
Cost review procedures
policies are well documented and included in
the program manager’s
guide. Earned Value
Management systems are
established to facilitate
the cost review process.
Enterprise and programlevel cost reviews are
required on a weekly
basis for all programs
including IR&D and company-funded activities.
For programs with an IPT
structure, cost issues are
regularly addressed at
IPT meetings. Cost control persons are assigned
to all programs and IPTs
within programs. Cost
containment plans for
out-of-tolerance cost
items are required. Cost
review procedures policies are well documented
and included in the program manager’s guide.
Earned Value Management systems are established to facilitate the cost
review process. Subcontractors are included in
program/IPT cost reviews.
28
Tenet
Number
20
Title
Level 1
Level 2
Level 3
Level 4
Level 5
Affordability
Measures
Performance, cost,
schedule and risk are
tracked program by program but no standard
exists by which to quantify and measure them. No
automated tools are used
to track these parameters.
Performance, cost,
schedule and risk are
tracked program by program. Standards are
developed to establish a
common format. No
automated tools are used
to track data.
Key performance, cost,
schedule and risk parameters are selected and
tracked program by program. Standards are
developed to establish a
common format. No
automated tools are used
to track data.
Key performance, cost,
schedule and risk parameters are selected and
tracked program by program. Standards are
developed to establish a
common format. Automated tools are used to
track data.
A standard procedure for
identifying and tracking
Affordability measures is
in place. Key performance, cost, schedule
and risk parameters are
selected and tracked
program by program.
Standards are developed
to establish a common
format. Automated tools
are used to track data.
29
Appendix B Acronyms
Acronym
CER
CPI
DTV
IPT
IR&D
KPP
LCC
MEL
O&O
ORD
PTW
SOO
SPI
WBS
Definition
Cost Estimating Relationship
Cost Performance Index
Design to Value
Integrated Product Team
Independent Research and Development
Key Performance Parameter
Life Cycle Cost
Master Equipment List
Organization and Operation
Operational Requirements Document
Price-to-Win
Statement of Objectives
Schedule Performance Index
Work Breakdown Structure
30