Applying Semantic Technology to Early
Stage Defense Capability Planning
Analysis Based on JCIDS Artifacts
Allen Moulton
amoulton@mit.edu
Sociotechnical Systems Research Center
77 Massachusetts Ave, Cambridge, MA 02139
Dr. Donna Rhodes
MAJ James Enos
Chief, SE Branch, JRAD, J8
Prof. Stuart Madnick
COL Douglas Matty
MIT Sloan School of Management
Chief, PDD, PAED, HQDA G8
Abstract ID 18026
18th NDIA Systems Engineering Conference
29 October 2015
Agenda
• Goals of JCIDS Semantic Architecture
Framework Research
• Joint Capability Enterprise Architecture
• Exploratory Experiments
• Systematizing Method for Manual Use
• Leveraging Semantic Technology
• Next Steps
2
JCIDS (Joint Capabilities Integration and Development System)
A Systematic Process for Warfighters to Develop, Validate, and
Control Capability Requirements for Acquisition
LIMITATIONS OF CURRENT JCIDS PROCESS
• Necessarily Document-Driven • DODAF Architecture Not Fully Integrated
• Silos of Information by Capability/Program and Date of Writing
Warfighters
SMEs
Docs
DODAF
Acquisition
SMEs
MIT Research Goals
Unlock docs
into data
Apply inference to
extend understanding
Connect
text info to
architecture
content
Bridge
info silos
Joint Capability Enterprise Architecture (JCEA)
3
System of Systems Complexity is Inherent in JCIDS
Value Proposition for Capability-Based Planning (Aldrich Study, 2004)
Strategy
Desired
Effects
Capabilities
Fielded
Systems
Capability-Based Planning Works Backwards from Goals to Factor Out Systems Needed
Not as Simple and Linear as it Looks
Investment decisions must be made years or decades in advance
... within limited and changing budget constraints
... to assure that Services will have the capabilities on hand
... to supply resources to combatant commanders
... to be dynamically integrated into joint task forces
... to achieve effects needed to accomplish future missions
... in support of national strategy
Question: How to Manage the Inherent Complexity of the Problem?
• Combinatorics of the solution space vs. need to limit scope of each system
• Dynamic effects of decision lead times and necessity for integration
• Uncertainty on critical factors affecting the design
e.g., strategy, threats, budgets, technology, related program outcomes
4
Joint Capability Enterprise Architecture (JCEA)
JCEA content
extracted from
multiple views
Text
Doc
Views
Views
JCIDS Docs, DODAF
and SMEs each
capture partial
information on
underlying reality as
of a point in time
DODAF
Views
C-M-L
Views
Other
Capabilities,
Systems and
Time-Frames
Search
Views
SME
Views
Other
Capabilities,
Systems and
Time-Frames
Current State and
Planned Future States
Strategic Guidance
Missions ― Threats
Force Capabilities
Functions and Tasks
Materiel Systems
Technology
JCEA used to
generate
other views
Other
Capabilities,
Systems and
Time-Frames
Other
Capabilities,
Systems and
Time-Frames
Other
Capabilities,
Systems and
Time-Frames
Other
Capabilities,
Systems and
Time-Frames
Decision
Views
Other
Capabilities,
Systems and
Time-Frames
JCEA holds
content that
can make
connections
across
capabilities
and time
frames
Other
Capabilities,
Systems and
Time-Frames
Underlying Fabric of Evolving Capabilities and Requirements over Time
Ontology defines slots that structure data extracted from documents and DODAF
Ontology also defines relationships among data elements in the JCEA model
5
Defining Semantics: Empirical Review of Documents
• Broad review of 88 unclassified sample JCIDS documents to build
familiarity, recognize patterns, and discern ‘ground truth’
• Detailed deep-dive into three capability documents (ICD, CDD, CPD)
1) what SHOULD be in document?
2) what WAS in document?
3) what is ESSENTIAL in document?
• Documents selected for deep-dive experiment:
– 3 different stages of development (ICD, CDD, CPD)
– 3 different functional areas staffed by different FCBs
– All in Air domain with documents staffed in 2007-2009
ICD
Logistics
CDD
Force Application
CPD
Battlespace Awareness
Joint Future Theater Lift (JFTL)
Move cavalry with armor
Joint Air-to-Ground Missile (JAGM)
Replace HELLFIRE,TOW and Maverick
Extended Range UAS (MQ 1C)
Dedicated support to Division
Found implicit interdependencies across separately staffed capabilities.
Framing a Joint Capability Enterprise Architecture:
Capability Categories – Joint Capability Areas
“To support needs definition, gap and excess analysis, major trade analyses, and capabilities planning,
DoD’s capabilities must be divided into manageable groups, or capability categories.” – Aldrich Study (2004)
2005 – Original JCAs
• 4 top level categories
(operational, functional,
domain, institutional)
• 22 Tier 1 with 240
subordinate JCAs
2007 – Revised JCAs
• Too many overlaps
and redundancies
• Unnecessary
complexity for use
as a taxonomy
•
•
•
•
9 Tier 1 JCAs, 6 Tiers
Functional only
Aligned with FCBs
Operational dimension
removed
Empirical Observations from Docs Conclusions
• Most JCIDS docs use multiple Tier 1 JCAs
• JCAs are used as a framework for
describing operational attributes of
capabilities not just desired effects
• JCAs alone are insufficient to
categorize capabilities
• A multidimensional category structure
is preferable to a single taxonomy
7
Framing a Joint Capability Enterprise Architecture:
Joint Staff Capability Mission Lattice (CML)
SAP
SAP
System
Architecture(s)
System
Architecture(s)
Deny Adversary
Objectives
Crisis Response / Limited
Contingency Operations
Military Engagement /
Security Cooperation
Counterinsurgency and
Stability Operations
Support to Civil
Authorities
Humanitarian Assistance /
Disaster Response
Policy
Facilities
Personnel
Training
UJTs
(enable decomposition of missions against threats, and
identification of associated capability requirements)
Rogue
Nations
Non-State
Actors
Weapons
of Mass
Destruction
Assesment(s)
Treaties/Alliances
Assesment(s)
Op.
Arch(s)
SAP
SAP
Planning & Direction
Collection
Processing/Exploitation
Analysis/Production
Dissemination/Relay
Operational
Architecture(s)
Force Management
Force Preparation
Human Capital Mgmt
Health Readiness
Ways
Near Peer
Competitors
Environmental /
Natural Events
Operational
Architecture(s)
Ends
Threats
Threat Assessments
Counter Weapons of
Mass Destruction
Critical Intelligence Parameters
Combat
Terrorism
Operational Concepts
Universal Joint Tasks
Service Tasks
Conditions
Force
Support
Battlespace
Awareness
Force
App.
Operational
Architecture(s)
Logistics
Operational
Architecture(s)
Operational
Architecture(s)
Op.
Arch(s)
SAP
SAP
SAP
Command
and Control
Op.
Arch(s)
Operational
Architecture(s)
Advisory & Compliance
Strategy & Assessment
Information Management
Acquisition & Technology
Program / Budget / Finance
Global Stabilizing
Presence
Communicate
Shape
Defeat
Adversaries
Global
Context/
Threats
Prote Build
Net-Centric ction Ptnr
Corporate
Management
Prevent
Joint Strategic
Capabilities Plan
SAP
System
Architecture(s)
Mitigate
Unified Command
Plan
SAP
Information Transport
Enterprise Services
Net Management
Information Assurance
Guidance for
Employment of
the Force
Cyber
Systems
System
Architecture(s)
Organize
Understand
Planning
Decide
Direct
Monitor
Defense Planning
Guidance
Space
Systems
SAP
Deployment/Distribution
Supply
Maintain
Logistic Services
Operational Contract Spt.
Engineering
Base/Installation Spt.
Quadrennial
Roles/Missions
Air
Systems
System
Architecture(s)
Maneuver
National Security
Interests
Sea
Systems
Engagement
Defense of the
Homeland
Integrated Security Constructs / OPLANs / CONPLANs
Maintain Nuclear
Deterrence
Quadrennial Defense Review – Prioritized Missions:
National Security
Strategy
Quadrennial
Defense Review
Logistics
Land
Systems
Means
Planning /
Operations
National Defense
Strategy
National Military
Strategy
Science & Technology / Research & Engineering
Capstone Concept for Joint Operations / Other Concepts and CONOPS
Strategic
Guidance
Organization
Adapted by MIT
from
(RevJoint
0.8.1 /Staff
24 AprConcept
2014)
Materiel Acquisition/Investment, Including Legacy System Sustainment
Doctrine
Capability-Mission
Lattice
Leadership/Education
Materiel and Non-Materiel Capability Solutions
Functions
Capability Requirement Portfolio Management
Basic ontology from Capability Mission Lattice has been expanded to include
elements required in JCIDS Manual and taxonomies/frameworks in use
8
Using C-M-L Ontology to Find Interdependencies
JFTL ICD
JAGM CDD
Phrases from JCIDS Docs
attached to Ontology Slots Interdependencies
JFTL ICD
Inferred
JAGM CDD
ER UAS CPD
ER UAS CPD
MVM: Mounted
Vertical Maneuver
The C-M-L based ontology can help identify interdependencies between
systems that are not apparent in documents or with current taxonomies
9
Systematizing Semantic Architecture Framework
JCIDS Ontology Design Task
Central goal: Define a semantic knowledge base that captures the
portfolio of capabilities & gaps early in development
Ontology and architecture frame the knowledge base
– Ontology also captures and connects essential military and
requirements process subject domain knowledge
Requirements documents provide the content
– Text of documents (interpreted against ontology)
– Structured information in tables and DODAF artifacts attached in
structured form suitable for machine use
– Images such as OV-1 (hard to extract info from)
Additional content will come from SME annotations as an
ontology-based knowledge base is put into use
Data captured and organized in a semantic architecture framework
will continue to be accessible and reusable as
SMEs rotate in and out and as circumstances change
10
Overview of ICD Ontology Design based on
2015 JCIDS Manual and Capability-Mission-Lattice
Cover Page
Metadata
Operational Context
• Time Frame
• Strategic Guidance
• ROMO
• Operational
Concepts
Capability Req’ts
• Define Capability
Requirements in
Lexicon of:
o
o
o
o
o
o
o
o
Time Frame
ROMO
Org / Unit Type
JCAs
UJTL Tasks
Service Tasks
Conditions
Supported and
supporting tasks
Threats
• Threat context
• Expected
operational
environment
• Operational
• Current threats
Attributes
• Anticipated threats o Metrics
o Objective Values
A. References
B. Acronyms
Capability Gaps
• Match to Current
Capabilities
o
o
o
o
Legacy fielded
In Development
Rapidly fielded
Predecessor system
if recap or next gen
• Identify Gaps for
each Operational
Attribute (O/A):
o Current capability
O/A metric value
o Gap from current
to objective value
• Operational
Impact of Gap
C. Glossary
Recommendations
• Materiel Solutions
Suggested for AoA
o Evolution of fielded
system
o Replacement or
recap of fielded
system
o Transformational
capability solution
• Technology
Leverage to reduce
Operational Risk
o Functionality
o Affordability
• DOTmLPF-P
Recommendations
D. DODAF
11
Example: JFTL ICD Extracted Capability Gaps
Gap Functional
Gap Description
Num Concept
1
2
IOM
OMSD
DMSS
3
4
DES
JFEO
Ontology Concept
in Yellow
Document
Data in Blue
Inability to operate into austere, short, unimproved landing areas
Inability to perform operational maneuver with medium weight armored vehicles
and personnel or reposition medium weight armored vehicles and personnel by
airlift
Inability to reposition forces with combat configured medium weight armored
vehicles via air
Inability to operate into austere, short, unimproved landing areas
Deliver cargo weights equivalent to the weight of combat configured medium
weight armored vehicles to austere, short, unimproved landing areas.
Conduct precision air delivery of supplies, to the point of need/point of effect over
strategic and operational distances with required velocity.
Inability to operate into austere, short, unimproved landing areas
Deliver cargo weights equivalent to the weight of combat configured medium
weight armored vehicles to austere, short, unimproved landing areas.
Conduct precision air delivery of supplies, to the point of need/point of effect over
strategic and operational distances with required velocity.
Inability to transport forces over strategic and operational distances to points of
need by passing traditional PODs, and to operate on austere, short, unimproved
landing areas.
Inability to deploy and employ forces, with combat configured medium weight
vehicles, via air across the global battle space from strategic, operational and
tactical distances
Reason for
Gap
Proficiency
Proficiency
Proficiency
Proficiency
Proficiency
Proficiency
Proficiency
Proficiency
Sufficiency
Proficiency
Proficiency
12
Example: Compare Gap Operational Attributes
Operational
attribute
Cargo handling
Combat Radius
Cruise Speed
Fuel efficiency
In-flight Refuel
Speed (as Receiver)
Payload Weight &
Dimensions
Precision Delivery
Gaps by Functional Concept
1
2
3
4
DMSS/
IOM OMSD
JFEO
DES
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Precision Landing
Secure
Communications
Self Deploy
Survivability
X
X
X
X
X
X
X
X
X
X
X
X
X
Operational attribute values
Ontology Concept
in Yellow
No MHE
As determined in AoA
As determined in AoA
Fuel efficiency must be greater than that of the C-130J
As required
Combat configured medium weight armored vehicles
(Army ground combat vehicles, Stryker)
Document
~25 – 50 km of objective
Data in Blue
Point of need/point of effect
Routine 0 ft takeoff & land (VTOL) to routine <1500 ft
takeoff and land (STOL)1 over a 50’ obstacle into austere,
complex, urban or unprepared landing areas independent
of external navigation aids
Interoperable, secure, encrypted, voice and data, beyond
line of sight/over the horizon
2,400 nm
Ability to effectively integrate with future joint forces for
threat suppression/mitigation in a low to medium threat
environment
13
Semantics-Based Inference Can Help Fill in Missing
Data and Inconsistencies in JCIDS Documents
Capturing Implicit Information
Documents reviewed often have
inconsistent data
– Most have current JCAs; some have 2005
JCAs; some have JFCs
– JCAs often used for multiple purposes
– Some have UJTs; most do not
SMEs can make sense of documents
despite gaps & other inconsistencies
Ontology-based data capture –
combined with inference rules – can
allow automation to follow same
logic used by SMEs
Connecting to other Knowledge
Example of how can semantic inference
can help:
• Joint Future Theater Lift (JFTL) ICD has
no UJTs
• JFTL ICD references JP 3-17 (Air
Mobility Operations) and Joint Forcible
Entry by name
• Joint Forcible Entry (JFEO) defined by
JP 3-18
• UJTL database ties UJTs to definitional
docs JP 3-17 and JP 3-18
• By combining these fragments of
information, UJTs for JFTL can be
inferred
Semantic architecture provides the benefits of capturing the true capability
provided by a system by interpreting text within a document.
14
Semantic Ontology Experiments
Developed an ICD ontology containing 150 data slots based on draft
2015 JCIDS Manual, C-M-L, and other frameworks
Manual text extraction experiments
– 6 ICDs as sources, 3 SMEs perform extraction
– Into Excel form structured by the ontology
– Reliability varied: some data were consistently extracted; other data inconsistent
A parallel project showed potential for applying natural language
processing to automate text extraction
SMEs built a practical relational database by focusing on the more
consistent areas and for wider sample of JCIDS documents
Experiment showed that DODAF views can be generated from data
extracted from JCIDS documents
MIT continuing research is focused on formalizing and systematizing
methods to extend the scope and value of the results
15
Research on Technologies and Methods for
Storing and Accessing Semantic Knowledge
1) Documents repository (current as-is state)
2) Relational or spreadsheet data
3) DODAF architecture structured data
– New 2015 JCIDS Manual requires DODAF views to be
submitted with requirements documents for validation
– Research is exploring how to connect text document
content to DODAF data and artifacts
4) Semantic data store with inference rules
– Facts stored as RDF Triples (subject-predicate-value)
– Flexibility from capturing facts in small pieces
– Facts can be combined in multiple ways by inference rules
and semantic query
16
Semantics Technology Proof-of-Concept Prototype
Design Overview
C-M-L
Space
Systems
Cyber
Systems
SAP
SAP
SAP
System
Architecture(s)
System
Architecture(s)
System
Architecture(s)
Policy
Air
Systems
SAP
System
Architecture(s)
Facilities
Logistics
Sea
Systems
SAP
Personnel
Training
Doctrine
Science & Technology / Research & Engineering
Land
Systems
System
Architecture(s)
Critical Intelligence Parameters
UJTs
(enable decomposition of missions against threats, and
identification of associated capability requirements)
Near Peer
Competitors
Rogue
Nations
Non-State
Actors
Weapons
of Mass
Destruction
Environmental /
Natural Events
Assesment(s)
Treaties/Alliances
Force
App.
Logistics
Operational
Architecture(s)
Op.
Arch(s)
SAP
SAP
SAP
Command
and Control
Op.
Arch(s)
Operational
Architecture(s)
Prote Build
Net-Centric ction Ptnr
Corporate
Management
Prevent
Battlespace
Awareness
Operational
Architecture(s)
Mitigate
Force
Support
Operational
Architecture(s)
Information Transport
Enterprise Services
Net Management
Information Assurance
SAP
Force Management
Force Preparation
Human Capital Mgmt
Health Readiness
Op.
Arch(s)
SAP
Planning & Direction
Collection
Processing/Exploitation
Analysis/Production
Dissemination/Relay
Operational
Architecture(s)
Communicate
Shape
Assesment(s)
Operational
Architecture(s)
Advisory & Compliance
Strategy & Assessment
Information Management
Acquisition & Technology
Program / Budget / Finance
Humanitarian Assistance /
Disaster Response
Organize
Understand
Planning
Decide
Direct
Monitor
Counterinsurgency and
Stability Operations
Support to Civil
Authorities
Maneuver
Counter Weapons of
Mass Destruction
Deny Adversary
Objectives
Crisis Response / Limited
Contingency Operations
Military Engagement /
Security Cooperation
Global
Context/
Threats
Engagement
Combat
Terrorism
Deployment/Distribution
Supply
Maintain
Logistic Services
Operational Contract Spt.
Engineering
Base/Installation Spt.
Guidance for
Employment of
the Force
Joint Strategic
Capabilities Plan
Quadrennial Defense Review – Prioritized Missions:
Quadrennial
Roles/Missions
Defense Planning
Guidance
Unified Command
Plan
Defeat
Adversaries
Global Stabilizing
Presence
Integrated Security Constructs / OPLANs / CONPLANs
Quadrennial
Defense Review
Capstone Concept for Joint Operations / Other Concepts and CONOPS
Maintain Nuclear
Deterrence
Defense of the
Homeland
National Military
Strategy
National Security
Interests
Materiel Acquisition/Investment, Including Legacy System Sustainment
Planning /
Operations
National Security
Strategy
National Defense
Strategy
Threat Assessments
(Rev 0.8.1 / 24 Apr 2014)
Strategic
Guidance
Organization
Capability-Mission
Lattice
Leadership/Education
Materiel and Non-Materiel Capability Solutions
JCIDS
Manual
Design
Ontology
Design
Other
Sources
Capability Requirement Portfolio Management
JCIDS
Docs
DODAF
Data
Manual Extraction
Automated Extraction
Semantic Technology Tools
RDF
Graph • Built on Semantic Web industry
Store standards such as OWL, RDF,
Semantic
Technology
Platform
Other
Sources
Semantic
Query
Updates
to ontology
and methods
Ontology – design based on
• JCIDS Manual
• Capability-Mission-Lattice
• other terminology frameworks
Dashboard
Viewer
Semantics
Experiments
Data
Export
DODAF
Generation
Evaluation of experimental results
SPARQL & cyber-security
• Includes tools for working with
ontology and data
• Highly flexible data store and
semantic query/search
• Technology used allows
research results to be ported to
other COTS product sets
DODAF Generation Tools
• COTS/GOTS tools, such as
NoMagic/MagicDraw/CAMEO
• UPDM interface (probable)
• Python to convert data format
17
Connections in Capability Requirements Ontology
Category Frameworks
Value of capability comes
from effect produced
Service &
Universal
Joint Tasks
JCA – Joint
Capability
Areas
supports
Strategic
Guidance
Mission
Areas
• Universal Joint Task
supports
Mission
Effects
Operational
Concept
Performing
Org/Unit
• Universal Joint Task
Desired
Effects
Operational
Activity
Operational
Activity
• Joint Capability Area
• Joint Capability Area
specifies
Operational
Attributes
Mission
Conditions
Expected
Operational
Environment
Threats to
Capability
Threats to
Mission
Capability
Conditions
Operational
Attributes
Time Frame
Operational Attribute
describes
Threat
Context
Capability Requirement
Mission Operational Context
Generic
Operational
Attribute
Capability
Gap
Required Initial
O/A Objective
Value
Difference
Metric for
Operational
Attribute
Current
Attribute
Value
Current
Capability
18
JCIDS Semantic Architecture Framework
Enables Capability Enterprise Architecture
– Multi-dimensional grouping of capabilities by category framework properties
– Logically deriving capability dimensions and similarities from operational
attributes
– Capturing and retaining SME knowledge across silos and over time
Identifies Capabilities Dependencies
– Tracing capabilities to assumptions, conditions, and threats
– Tracking interfaces and connections among capabilities
– Inferring dependencies based on effects produced and effects needed
Supports Systems Engineering
– Trade space identification for capability requirements planning
– Trade space exploration at the capabilities portfolio level
MIT Research is investigating and developing methods to apply semantic
technology to Joint Capability Enterprise Architecture
19
Goals for Semantic Architecture (2016)
Unlocking Knowledge
• Decompose documents into
conceptual elements independent of
language, to enable translation of across
terminology, frameworks, and
taxonomies.
• Identify implicit interconnections
and interdependencies across
Supporting Decisions
• Provenance: Maintain time-varying
continuity of requirements across
development stages and across separate
branching threads.
• Drill down: Make conceptual connections
across different levels of architecture (e.g.
SoS vs. Systems, KPPs vs. DODAF) as
designs evolve.
separately staffed capability requirements
(including different time periods, different • Track changes to assumptions (e.g.,
functional areas, and different services or
strategic direction, mission profiles,
components).
threats, operational concepts, technology
available).
• Connect text to architecture to create
a more complete picture in a form
• Support systems engineering
suitable for inference.
methods such as Trade Space Exploration
• Generate DODAF artifacts from
ontology-based data extracted from text
documents.
and Epoch-Era Analysis.
20
References
Aldridge, Pete et al. (2004). Improving DOD Strategic Planning, Resourcing and Execution to
Satisfy Joint Capabilities. Joint Defense Capabilities Studies, Jan 2004.
Ahmed, Col. L. Najeeb (2014) Improving Trade Visibility and Fidelity in Defense Requirements
Portfolio Management: A Formative Study of the Joint Capabilities Integration and
Development System using Enterprise Strategic Analysis and Semantic Architecture
Engineering. Unpublished MIT SDM Thesis.
Allemang, Dean & Hendler, Jim (2011). Semantic Web for the Working Ontologist. Waltham,
MA: Morgan Kaufman.
U.S. Dept of Defense. JCIDS Manual (12 Febuary 2015)
Acknowledgements
The work presented here was supported, in part, by the MIT Lincoln Laboratories and the US Army
under the "Study of JCIDS Semantic Architecture Framework" project. Any opinions, findings, and
conclusions or recommendations expressed in this material are those of the authors and do not reflect
the official policy or position of MIT Lincoln Laboratory, the US Army, the Department of Defense.
All research and results reported are unclassified
21