EMIS 7307
T&E Part 2
DOD 5000 Changes
• Documents in flux.
• MNS - Mission need statement
– Non system specific, a needed capability.
• Being replaced by Initial Capabilities
Document (ICD).
– ICD identifies the capability gap, describes the
evaluation of different approaches and proposes
an approach to pursue. - Supports the MS A
decision.
1
EMIS 7307
T&E Part 2
DOD 5000 Changes
• ORD - Operational requirements document
• Minimum acceptable capability of possible system.
• ORD is being replaced by Capabilities
Development Document (CDD) at MS B
and Capabilities Production Document
(CPD) for MS C.
• CDD- Contains operational performance
parameters to design and build an affordable and
useful increment includes KPPs (Key performance
parameters).
• CPD - Contains production attributes of a single
increment.
2
EMIS 7307
T&E Part 2
Revised DoD 5000
ICD
CDD
CPD
3
EMIS 7307
T&E Part 2
DoD 5000 with planned upgraded capabilities
called increments. A spiral is used when
upgrades are expected but are TBD.
AoA Analysis of Alternatives
DAB Defense Acquisition Board
JROC Joint Requirements Oversight Council
4
EMIS 7307
T&E Part 2
A significant number of documents have relevancy to T&E.
5
EMIS 7307
T&E Part 2
Test Documentation Relationships
Analogous to A, B and C spec relationships
6
EMIS 7307
T&E Part 2
7
EMIS 7307
T&E Part 2
Test Program Documentation
8
EMIS 7307
NATIONAL POLAR-ORBITING
T&E Part 2 OPERATIONAL
ENVIRONMENTAL SATELLITE SYSTEM (NPOESS) Test and
Evaluation Master Plan (TEMP)
9
EMIS 7307
T&E Part 2
NATIONAL POLAR-ORBITING OPERATIONAL
ENVIRONMENTAL SATELLITE SYSTEM (NPOESS)
TEMP
10
EMIS 7307
T&E Part 2
NATIONAL POLAR-ORBITING OPERATIONAL
ENVIRONMENTAL SATELLITE SYSTEM (NPOESS)
TEMP
11
EMIS 7307
T&E Part 2
Test Program Documentation
• System Test Plan
– Describes ‘big picture’ of all the testing for the whole
system, not a management document.
– This is the foundational technical test document from
which subservient plans are derived.
– Includes laboratory, component, subsystem, system,
software etc.
• Detail Test Plan
– In a large system, each of these topics (laboratory,
component, subsystem, system, software etc.) is likely
to be a volume or a book of its own.
12
EMIS 7307
T&E Part 2
13
EMIS 7307
T&E Part 2
Other Documents
• Test Procedures
– Implements a specific test with carefully
constructed step by step directions.
– Completeness is paramount as unplanned steps
may invalidate the data being collected.
– May be augmented by test cards or scripts.
– Formality in execution may require QC and/or
customer sign-off.
– Dry run - find problems in procedure or UUT
and fix.
14
– Document changes during execution.
EMIS 7307
T&E Part 2
Other Documents
• Test Reports
– Quick look - very important for management
• Based on observation and limited, if any, data
analysis
• May affect next day’s testing - consider email?
– Final report includes:
15
EMIS 7307
T&E Part 2
Test/Program Interrelationships
16
EMIS 7307
T&E Part 2
More about types of testing.
• DT&E again.
• Contractor perspective.
– Demonstrates design and development and in
the end-game, completeness i.e. meets spec.
– Reduce risks.
• High-risk components.
• Intra-subsystems.
• Inter-subsystem.
17
EMIS 7307
T&E Part 2
• DT&E government perspective.
–
–
–
–
Demonstrates spec compliance.
Used for acceptance of the EDM(s).
Estimate the military utility.
Confirms ready for field testing (OT&E).
18
EMIS 7307
T&E Part 2
• DT&E by phase.
– Prior to program start.
• Determine or at least estimate that the technology is
feasible.
– Concept and technology development.
• Assist in selecting between possible alternative
system concepts.
• Info for SRR and program start decision.
19
EMIS 7307
T&E Part 2
• DT&E by phase.
– During System Development and
Demonstration.
• Risks can be reduced to acceptable levels.
• From here on engineering efforts instead of
experimental efforts.
• Supports decision to build an EDM.
• System I&T is a part of this.
• Includes EDM acceptance testing.
• Includes the testing of the EDM to provide data for
the LRIP decision.
20
EMIS 7307
T&E Part 2
• DT&E by phase.
– During production and deployment phase.
• Prelude to IOT&E.
• Navy calls TECHEVAL.
– During full rate production and operations.
• To verify corrections and/or improvements.
• In terms of calendar time and manhours
consumed, DT&E is larger than OT&E and
has mostly contractor involvement.
21
EMIS 7307
T&E Part 2
Another view of DT&E
22
EMIS 7307
T&E Part 2
DT&E For the MSs
23
EMIS 7307
T&E Part 2
OT&E is complex in it’s simplicity!
24
EMIS 7307
T&E Part 2
Types of OT&E
• Early Operational Assessment (EOA).
– Performed for the Program Office and other
decision makers as a indicator of how
operational things are “shaping up”.
– Done by operational testers but may be mostly
analytical based on DT&E provided info.
• Initial OT&E
– Supports full rate production decision.
– Called OPEVAL by Navy.
25
EMIS 7307
T&E Part 2
Types of OT&E cont’d.
• Follow-on OT&E (FOT&E)
– After full rate production.
– Conducted during fielding.
– Checks on correction of IOT&E deficiencies.
26
EMIS 7307
T&E Part 2
More comparisons
27
EMIS 7307
T&E Part 2
Lessons for the PM that you should know too.
28