Breakthroughs in Applying Systems Engineering to
Technology Development
for
International Council on Systems Engineering (INCOSE)
Systems Engineering Conference in DC (SEDC)
Ms. Melanie Klinner
US Army Space and Missile Defense Command/
Army Forces Strategic Command
Mr. Jeffrey Craver
Defense Acquisition University
Mr. Mike Ellis
Senior Systems Engineer
(Formerly with Intrepid)
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
1
Overview
• Describe the problem space of managing the maturation of technology
for transition from the perspective of the Program Manager, User, and
the Technologist
• Provide an overview of the Technology Program Management
Model (TPMM) solution set in terms of Processes, Systems
Engineering, and Transition Management
• Describe the Breakthrough in terms of the Systems Engineering
Module (SEM) that provides online implementation of TPMM
• Describe how the SEM exists within an S&T Enterprise and for the
Management of Technology Programs/Portfolio’s/Project’s
• Describe how the DTRA TPMM Project has applied the model to
improve current processes and add-value to the enterprise
• Provide the Way Forward and the extension of the TPMM process into
the Technology Development Domain from Basic Research to Program
Management.
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
2
The Problem
Fact: There are diminishing resources and belt tightening ongoing
in DoD
Unfortunate Reality: Warfighter Needs are increasing at a
significant rate in an attempt to outpace an Asymmetric Threat
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
3
Faced With this Reality,
What Needs to Be Done?
• If we can't get adequate funding,
 We need to make smarter choices in what to fund.
• If we can't build things faster,
 We need to build them more efficiently with less re-work.
• If we can't afford the full set of performance,
 We should focus on the highest priority needs.
• If we can't afford to fail,
 We must effectively identify and manage the risk.
Culture Change: Adopt a Stage-Gate Process in the Development Phase
and Incorporate a Systems Engineering-Based Criteria Set for Assessing
Technology Maturity
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
4
Technology Development Challenges
Consistency in Technology Development Planning
 Requirements definition and plans for maturity advancement are not
developed consistently or effectively.
 No common set of tools and standards to gauge technology maturity.
Improve the rate for Transitioning of Technologies
 Transition not always considered as a part of Technology Development
 Limited Customer/User identification/involvement
Establish “a management methodology that balances the portfolio by
Incorporating Discipline and Rigor through the use of clear, well-defined
and Measurable Metrics!”
J. Heusmann DTRA-J9 (7 May 09)
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
5
Perspectives
Your next chance for
funding is 5 years down
the road – stud!
Hey Buddy - I
OWN The
Requirements!
Gotta be small,
lightweight,
and 99.99%
reliable
I Want it All!!
I Want it
Cheap!
I Want it Now!
I NEED a
REQUIRMENT
(CDD)!
I’m
governed
by the
JCIDS
My prime can
do that!!
I am governed by
DoD 5000.02
You don’t
understand - This
project is different
from everyone else
S&T does not
require a process –
I have been doing it
for years
Budget???
You forgot
about the
“illities”!!!
•
•
•
•
•
Threat Driven
Soldier-Proof
Fieldable
Meets Mission Needs
DOTMLPF
USER
2014-04-03
If you “Push”
long enough –
they will come!
Customer role
is to integrate
•
•
•
•
•
•
•
Value Added
Capability
Probability of Success
Acquisition Strategy
Budget (LLC/POM)
Schedule - WBS
The System “ approach”
• Technical “breakthrough”
• Performance Goals
• Risk
• Cost Estimate.
• Program Plan
• Build a prototype
PM
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
S&T
6
TPMM Collaboration
The TPMM Collaboration was formed to improve and expand the TPMM
methodology and jointly develop a tool-based process intent on reducing
the risk to technology development.
DAU
DTRA
USASMDC
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
7
Minimize the Risk in Misalignment
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
8
SMDC/DAU/DTRA Collaboration
Project Breakthroughs
• A paradigm shift in Project and Portfolio Management has begun with
•
•
•
•
•
the use of a SharePoint capability to :
Plan
Execute
Communicate progress
Assess technical maturity
Manage risk
DAU has recommended TPMM as a Systems
Engineering best practice since 2008.
Best
Practice
• A significant advance in the consistent SE product set (SE artifacts,
metrics, Exit Criteria, & online SEM reports).
• Revolutionary changes to cultural norms evident by Increasing interest
from (Senior-DoD) early-adopters to a software tool that can overcome
resistance from the S&T community to apply SE principles.
• Enterprise Adoption facilitated by the pervasiveness of SharePoint
throughout the DoD which allows for easy adoption and simplified IA
requirement.
DTRA Process Improvement effort has been recognized by
AT&L Value Engineering Achievement Award.
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
9
Recipient of a USD(AT&L) 2012 Value
Engineering Achievement Award
TPMM-based SharePoint Solution Application:
Systems Engineering Module (SEM)
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
10
Functions Performed by TPMM
• Program Definition
o Identify Activities to consider
o Identify Deliverable Documents
o Provide guidance for Tailoring
o Employ “Best Practice” Tools
o Identify and Mitigate Risk
• Transition Management
o Technology Transition
o Technology Transfer
o Technology Marketing
• Maturity Assessments
o Establishes Entry/Exit Criteria
o Provides a Framework for
Technology Maturity Assessments (TMA)
Legend:
TTA = Technology Transition Agreement TDS = Technology Development Strategy
A TRL-Based, Stage Gate Model Designed for Technology
Development and Transition
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
11
Systems Engineering in TPMM
• Lab Test Strategy
• Operational
Prototype
Validation
• IDD
• Updated Technology Development Strategy (TDS)
• Preliminary System Spec
• Final TTA (Commitment)
• Breadboard Laboratory Test Results
• TDS/Acq Strategy Roadmap
• Updated TTA - Technology
Transition Agreement (Intent)
• Final System Spec
• Relevant Environment Test
Design
• Manufacturing Plan
• Update Tech Requirements
• “-ilities” Documented
• Functionality Analysis
• Initial “-ilities” Plan
• CTE Identification
• Exit Criteria
• System Configuration
formally controlled and
documented
• Risk Mitigation
Strategy
• Interface Spec
• Design Codes
• Brassboard Relevant Environment
Test Results
Systems Engineering principles are embodied in the activities and
associated documentation in each stage of maturity development
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
12
Tech Program Events Gates/TDP’s
[With Sample Entry/Exit Criteria]
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
13
Systems Engineering Module (SEM)
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
14
Staff Meeting Review Graphic
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
15
TPMM Capabilities for Decision Support
Automatically Produce a Project Quad Chart
Provide consistent format for Program/Project Status Review
Report on Portfolio by Performer, Collaborator
Assist Synergy among Performers, Contract Vehicles
Plot Portfolio Matrix to Analyze Distribution
Support for Portfolio Balancing, Clustering, and Gap Filling
Trace to Rqmts (% covered, uncovered, dups)
Provide a platform for Risk Management
Consistency in Risk Mgmt and Promotes use in Project Mgmt
% of Program supported by Inter-Agency
Calculate Program Funding Disbursed to the Inter-Agency
Metrics Analysis for Portfolio balancing
Program/Portfolio balancing based on Risk/Cost Ratio
Project Planning for Execution
Prepare for and Conduct Stage Gate Reviews
2014-04-03
Program/Portfolio Balancing based on Capability Coverage
Assist in Evaluating Technology Road Map, Developing CDRL's
Objectively Validate Technology Maturity to Customers
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
16
Way-Ahead
Use Systems Engineering-Based (TPMM) process to change the culture:
• Balance Technology Portfolio’s so that Priorities are aligned to Mission
Strategy and Customer Needs are clearly identified and understood
• Improve Project Management Performance in the areas of requirements
traceability, system engineering rigor, transition focus, maturity assessment,
and in planning/execution/reporting
• Exchange “informational” calendar-driven Project Technical Reviews with
“decisional” event-driven Reviews where technical progress is examined
"as-needed" in accordance with a Project’s Lifecycle Plan.
• Increase Successful Transition of Technology by engaging Early Systems
Engineering and aligning with Acquisition Authorities.
• Promote consistency in technical interpretation of TRL Maturity levels
throughout the DoD S&T Enterprise and with Program Managers.
• Extend SE-Based practices for use in SBIR and IRAD Programs by tailoring
the TPMM as applicable to that purpose.
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
17
Contact/Consultation
Information
Mr. Jeffrey Craver
Defense Acquisition University
South Region
Huntsville, AL
E-mail: Jeffrey.Craver@dau.mil
Ms. Melanie Klinner
USASMDC/ARSTRAT
TPMM Product Manager
Technical Center/Cyberspace Technology Division
Redstone Arsenal, Huntsville, AL
E-Mail: Melanie.G.Klinner.civ@mail.mil
Mr. Mike Ellis
Systems Engineering Consultant
(Formerly with Intrepid)
E-mail: Michael.Shannon.Ellis@gmail.com
Mr. Alan Barclay
TPMM Industry Project Manager
Intrepid
E-mail: Alan.Barclay@intrepidinc.com
US Government Personnel can request a link to the TPMM Web Site at:
http://www.tpmm.info
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
18
BACKUP
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
19
TPMM Capabilities for Decision Support
Capability
Prepare and execute TRL Checklists
Consider viable interest, intent or commitment to Transition
Automatically Produce a Project Quad Chart
Provide consistent format for Program/Project Status Review
Report on Portfolio by Performer, Collaborator
Assist Synergy among Performers, Contract Vehicles
Plot Portfolio Matrix to Analyze Distribution
Support for Portfolio Balancing, Clustering, and Gap Filling
Trace to Rqmts (% covered, uncovered, dups)
Program/Portfolio Balancing based on Capability Coverage
Provide a platform for Risk Management
Consistency in Risk Mgmt and Promotes use in Project Mgmt
% of Program supported by Inter-Agency
Calculate Program Funding Disbursed to the Inter-Agency
Metrics Analysis for Portfolio balancing
Program/Portfolio balancing based on Risk/Cost Ratio
Project Planning for Execution
Prepare for and Conduct Stage Gate Reviews
2014-04-03
Decision Support
Assist in Evaluating Technology Road Map, Developing CDRL's
Objectively Validate Technology Maturity to Customers
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
20
Project Planning For Execution
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
21
Prepare and Execute TRL Checklists
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
22
TPMM Capability Applicability Matrix
Capability
Prepare and execute TRL Checklists
Consider viable interest, intent or commitment to Transition
Automatically Produce a Project Quad Chart
Provide consistent format for Program/Project Status Review
Report on Portfolio by Performer, Collaborator
Assist Synergy among Performers, Contract Vehicles
Plot Portfolio Matrix to Analyze Distribution
Support for Portfolio Balanacing, Clustering, and Gap Filling
Trace to Rqmts (% covered, uncovered, dups)
2014-04-03
Decision Support
Program/Portfolio Balancing based on Capability Coverage
Provide a platform for Risk Management
Consistency in Risk Mgmt and Promotes use in Project Mgmt
% of Program supported by Inter-Agency
Calculate Program Funding Disbursed to the Inter-Agency
Metrics Analysis for Portfolio balancing
Program/Portfolio balancing based on Risk/Cost Ratio
Project Planning for Execution
Assist in Evaluating Technology Road Map, Developing CDRL's
Prepare for and Conduct Stage Gate Reviews
Objectively Validate Technology Maturity to Customers
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
23
Automatically Produce Project Quad
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
24
Risk Drill-Down from Project Quad
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
25
TPMM Capability Applicability Matrix
Capability
Prepare and execute TRL Checklists
Consider viable interest, intent or commitment to Transition
Automatically Produce a Project Quad Chart
Provide consistent format for Program/Project Status Review
Report on Portfolio by Performer, Collaborator
Assist Synergy among Performers, Contract Vehicles
Plot Portfolio Matrix to Analyze Distribution
Support for Portfolio Balanacing, Clustering, and Gap Filling
Trace to Rqmts (% covered, uncovered, dups)
2014-04-03
Decision Support
Program/Portfolio Balancing based on Capability Coverage
Provide a platform for Risk Management
Consistency in Risk Mgmt and Promotes use in Project Mgmt
% of Program supported by Inter-Agency
Calculate Program Funding Disbursed to the Inter-Agency
Metrics Analysis for Portfolio balancing
Program/Portfolio balancing based on Risk/Cost Ratio
Project Planning for Execution
Assist in Evaluating Technology Road Map, Developing CDRL's
Prepare for and Conduct Stage Gate Reviews
Objectively Validate Technology Maturity to Customers
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
26
Portfolio Reports/Views
Collaboration:
Type
Innovation:
Type
Performer:
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
27
TPMM Capability Applicability Matrix
Capability
Prepare and execute TRL Checklists
Consider viable interest, intent or commitment to Transition
Automatically Produce a Project Quad Chart
Provide consistent format for Program/Project Status Review
Report on Portfolio by Performer, Collaborator
Assist Synergy among Performers, Contract Vehicles
Plot Portfolio Matrix to Analyze Distribution
Support for Portfolio Balanacing, Clustering, and Gap Filling
Trace to Rqmts (% covered, uncovered, dups)
2014-04-03
Decision Support
Program/Portfolio Balancing based on Capability Coverage
Provide a platform for Risk Management
Consistency in Risk Mgmt and Promotes use in Project Mgmt
% of Program supported by Inter-Agency
Calculate Program Funding Disbursed to the Inter-Agency
Metrics Analysis for Portfolio balancing
Program/Portfolio balancing based on Risk/Cost Ratio
Project Planning for Execution
Assist in Evaluating Technology Road Map, Developing CDRL's
Prepare for and Conduct Stage Gate Reviews
Objectively Validate Technology Maturity to Customers
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
28
Portfolio Matrix Analysis
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
29
TPMM Capability Applicability Matrix
Capability
Prepare and execute TRL Checklists
Consider viable interest, intent or commitment to Transition
Automatically Produce a Project Quad Chart
Provide consistent format for Program/Project Status Review
Report on Portfolio by Performer, Collaborator
Assist Synergy among Performers, Contract Vehicles
Plot Portfolio Matrix to Analyze Distribution
Support for Portfolio Balanacing, Clustering, and Gap Filling
Trace to Rqmts (% covered, uncovered, dups)
2014-04-03
Decision Support
Program/Portfolio Balancing based on Capability Coverage
Provide a platform for Risk Management
Consistency in Risk Mgmt and Promotes use in Project Mgmt
% of Program supported by Inter-Agency
Calculate Program Funding Disbursed to the Inter-Agency
Metrics Analysis for Portfolio balancing
Program/Portfolio balancing based on Risk/Cost Ratio
Project Planning for Execution
Assist in Evaluating Technology Road Map, Developing CDRL's
Prepare for and Conduct Stage Gate Reviews
Objectively Validate Technology Maturity to Customers
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
30
Capability Needs Coverage Matrix
TECH
PORTFOLIO
PROJECT
REQUIREMENTS/CAPABILITY NEEDS
Gaps NOT
Covered
Gaps
Potentially
Duplicated
2014-04-03
Gaps
Partially
Covered
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
31
TPMM Capability Applicability Matrix
Capability
Prepare and execute TRL Checklists
Consider viable interest, intent or commitment to Transition
Automatically Produce a Project Quad Chart
Provide consistent format for Program/Project Status Review
Report on Portfolio by Performer, Collaborator
Assist Synergy among Performers, Contract Vehicles
Plot Portfolio Matrix to Analyze Distribution
Support for Portfolio Balanacing, Clustering, and Gap Filling
Trace to Rqmts (% covered, uncovered, dups)
2014-04-03
Decision Support
Program/Portfolio Balancing based on Capability Coverage
Provide a platform for Risk Management
Consistency in Risk Mgmt and Promotes use in Project Mgmt
% of Program supported by Inter-Agency
Calculate Program Funding Disbursed to the Inter-Agency
Metrics Analysis for Portfolio balancing
Program/Portfolio balancing based on Risk/Cost Ratio
Project Planning for Execution
Assist in Evaluating Technology Road Map, Developing CDRL's
Prepare for and Conduct Stage Gate Reviews
Objectively Validate Technology Maturity to Customers
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
32
Staff Meeting Review Graphic
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
33
TPMM Capability Applicability Matrix
Capability
Prepare and execute TRL Checklists
Consider viable interest, intent or commitment to Transition
Automatically Produce a Project Quad Chart
Provide consistent format for Program/Project Status Review
Report on Portfolio by Performer, Collaborator
Assist Synergy among Performers, Contract Vehicles
Plot Portfolio Matrix to Analyze Distribution
Support for Portfolio Balanacing, Clustering, and Gap Filling
Trace to Rqmts (% covered, uncovered, dups)
2014-04-03
Decision Support
Program/Portfolio Balancing based on Capability Coverage
Provide a platform for Risk Management
Consistency in Risk Mgmt and Promotes use in Project Mgmt
% of Program supported by Inter-Agency
Calculate Program Funding Disbursed to the Inter-Agency
Metrics Analysis for Portfolio balancing
Program/Portfolio balancing based on Risk/Cost Ratio
Project Planning for Execution
Assist in Evaluating Technology Road Map, Developing CDRL's
Prepare for and Conduct Stage Gate Reviews
Objectively Validate Technology Maturity to Customers
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
34
Project Gate Review Manager
2014-04-03
DISTRIBUTION A.: Approved for public release; distribution unlimited. SMDC #3139. DTRA #NT-13-612
35