Cognitive Systems
High Performance
Embedded Computing Workshop
Robert Graybill
DARPA IPTO
Sept 28, 2004
7/12/2016 RBG
1
The Challenge

Computer systems are the backbone of key
national infrastructure and critical DoD systems



While computational performance is increasing,
productivity and effectiveness are not keeping up





Virtually all important transactions involve massive amounts of software
and multiple computer networks
DoD future vision is “network-centric warfare”
Cost of building and maintaining systems is growing out of control
Systems have short lifespans with decreasing ROI
Demands on expertise of users are constantly increasing
Users have to adapt to system interfaces, rather than vice versa
As a result, systems have grown more complex,
more fragile, and more difficult to develop
We need to change the game
completely
7/12/2016 RBG
2
Capability Provided by Software in DoD Systems
is Increasing but so are the Challenges…
90
F-22
80
70
B-2
60
50
F-16
F-16
40
F-15
F-15
Multi-year
delays
associated
with software
and system
stability
Software and
testing delays
push costs up
30
F-111
F-111
20
10
F-4
A-7
1960
1964
0
Ref: Defense Systems Management College
7/12/2016 RBG
1970
1975
3
1982
1990
2000
IPTO’s Approach
Developing Cognitive Systems:
Systems that know what they’re doing

A cognitive system is one that
can reason, using substantial amounts of appropriately
represented knowledge
 can learn from its experience so that it performs better
tomorrow than it did today
 can explain itself and be told what to do
 can be aware of its own capabilities and reflect on its own
behavior
 can respond robustly to surprise

7/12/2016 RBG
4
Computing Systems
that know what they’re doing can…
7/12/2016 RBG

…reflect on what goes wrong when an anomaly occurs
and anticipate its occurrence in the future

…respond to naturally-expressed user directives to
change behavior or increase functionality

…be configured and maintained by non-experts

…reconfigure themselves in response to environmental
changes and mission events

…reduce the effort to develop and maintain software

…thwart adversarial systems that don’t know what
they’re doing

…preserve “corporate memory” to ease transitions for
rotational personnel
5
Four Tiers of Agile Processing
Systems That Know What They’re Doing
+ Cognitive Processing
Hardware Elements

SBIRs
Intelligent Systems
 - Architectures for Cognitive Information Processing
(ACIP)

+ HECURA
High-End Application
Responsive Computing
 High Productivity Computing Systems Program (HPCS)
+ OneSAF

Objective
System
Mission Responsive
Architectures
+ XPCA??
 Polymorphous Computing Architectures Program (PCA)

Power Management
 Power Aware Computing and Communications Program
(PAC/C)
7/12/2016 RBG
6
M
i
s
s
i
o
n
Protocols
Micro Architectures
Vdd Scaling
Clock Gating
Compilers/OS
Algorithms
ACIP Program Vision
Biological Clues
Cognitive Algorithm Clues
DoD Mission Challenge Clues
MTO
IPTO
Cognitive Efforts,
PAL, Real, SRS…
Functional
Demonstrations
& Algorithm
Developments
ACIP Phase 1
Early Architecture
Concepts & InContext Evaluation
- 2 Years -
ACIP Phase 2
Full Scale
Implementation and
Demonstration
- 4 Years -
ACIP Phase 3
Cognitive
Technology DOD
System Transitions
- 2 Years 7/12/2016 RBG
7
3-D Interconnects,
Optical, Nano
Physical Devices,
Interconnect, and
Packaging
ACIP Phase I
Study Considerations
Cognitive
Reasoning,
Learning &
Knowledge
Technologies
In-Context
Cognitive
DoD Applications
HW/SW
Architectures &
Technologies
Metric Evaluation
Cognitive
Processing
Requirements
DoD
Applications
Requirements
& Real-time
Constraints
In-Context
Evaluations
Analyze Cognitive
Techniques,
Computing,
Memory and
Control
Requirements
Early
Architecture
Concepts
Innovative
Architecture
Concepts
Technology
Assessments
(2 Year Study)
Multiple Disciplinary Teams
MTO Technology
7/12/2016 RBG
IPTO Projects
8
Deliverables
System architecture
concepts, models, &
evaluations-to-SDR
Concept device
specification and
technology roadmap
Cognitive computing
requirements, analysis,
specifications and
runtime characterization
Living Framework Draft
Composable runtime
concepts
Phase II Challenge
Problem, Metrics, & Go
No/Go Definition
ACIP Phase I Response

Fantastic Response!!!

Participation Mix (Including
Subs)
9 Defense Contractors
 11 Research Laboratories
 51 Universities
 30 Commercial Companies
Broad MultiDisciplinary
Coverage required
for System
Innovation

Large Diverse and
Robust Teams
Resulted in the Best
Concepts
Study Technical Framework Concept has Emerged
7/12/2016 RBG
9
Funded ACIP Efforts

COGnitive ENGine Technology (COGENT)


Polymorphous Cognitive Agent Architecture (PCAA)


Raytheon Company - Network Centric Systems
Lockheed Martin Advanced Technology Labs
CEARCH: Cognition Enabled Architecture

7/12/2016 RBG
University of Southern California/ISI
10
COGENT
ACIP Related SBIRs

Reservoir Labs Inc – Cognitive Processing Hardware and
Software elements

Intelligent Automation Inc. – Hardware Architectures for
Flexible Component Based Hybrid Cognitive Systems

Hoplite Systems LLC – Cognitive Processing Hardware
Elements

Cardinal Research LLC – Cognitive Processing Hardware
Elements

Saffron Technologies – Associative Memory Hardware
Elements for Cognitive Systems (Funded by AFRL)
7/12/2016 RBG
11
Cognitive Technology
Classification
Reasoning Algorithms
1st Order Reasoning
Abductive Reasoning
Analogical Reasoning
Bayesian Networks
Case-based Reasoning
Causal Reasoning
Common Sense Reasoning
Counterfactual Reasoning
Deductive Inference
Defeasible Reasoning
Forward & Backward Chaining
Fuzzy Reasoning
Game Theory - Optimization
Goal-oriented Planning
Heuristic Meta-reasoning
Inductive Reasoning
Logical Pattern Matching
Logical Unification
Markov Processes
Mathematical Programming
Maximum Likelihood
Meta-meta Reasoning
Modal Intuitionistic, Higher Order Reasoning
Model-based Reasoning
Non-monotonic Reasoning
Optimal decisions - Min-Max, Auctions
Pattern Matching
Probabilistic Constraint Satisfaction
Resource-limited Theorem Proving
SAT - Constraint Satisfaction
Special Purpose Reasoning Algorithms
Temporal Reasoning
Utility Theory
Well-formedness Reasoning
7/12/2016 RBG
Symbolic (S)
Probabilistic (P)
Hybrid (H) Ray
S
S,P,H
X
S,H
X
P
S,H
P
X
S
X
S
S
X
H
S
H
H
S
H
S
S
S
P
H
P
X
S
S
H
X
S
P
X
H
H
X
S
S
X
S
S,P,H
X
P
X
S
LM ISI
X
X
X
X
X
X
X
X
X
X
X
X
X
Learning Algorithms
Abductive Learning
Abstraction
Analogical Learning
Artificial Neural Networks
Associative
Bayesian Learning
Chunking
Classification Learning
Clustering
Constructing Analogies
Co-training
Data Mining
Decision Trees
Dimensionality Reduction
Evolutionary Search
Genetic Algorithms
Inductive Learning
Instance-based Learning
Learning from Advice
Network Construction
Parameter Learning
Plan recognition
Reinforcement Learning
Relational Learning
Rule Generation Composition & Specialization
Statistical Clustering
Statistical Learing (nearest neighbor, approx)
Supervised Learning
Support Vector Machine
X
X
12
Symbolic (S)
Probabilistic (P)
Hybrid (H) Ray
H
H
X
S
P
H
X
P
H
X
H
P,H
X
S
X
H
H
H
H
H
P
S
P
H
X
P
P
X
H
P,H
X
S
X
S
P
P
P
P
X
LM ISI
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Knowledge Representation
Algorithms
1st Order Logic (with extensions)
Bayesian Classifier
Bayesian Networks
Case -based
Causal Networks
Conceptual Graphs
Decision Trees
Episodic
Frames
Fuzzy Logic
Horn Clause Program
Influence Diagrams
Knowledge Acquisition
Logical (Prop., FOL, Frame-based)
Logical Rules
Markov Models
Multi-layer Neural Net
Ontologies
Production System
Propositional Logic
Reactive Plan
Relational Models
Rule-based Systems
Self-knowledge
Semantic Nets
Situation Calculus
Taxonomic Hierarchy
Temporal Networks
Type Ontologies and Constraints
Symbolic (S)
Probabilistic (P)
Hybrid (H) Ray
S
X
P
P,H
X
S
H
X
H
X
H
X
H
H
X
H
S
H
X
H
S
S
P
X
P
H
X
S
H
X
S
H
X
H
H
S
S
X
P
H
X
S
LM ISI
X
X
X
X
X
X
X
X
X
Cognitive Services
Cognitive_Services
Reasoning_Strategy
Learning
Control_Regime
Semantic
Analogy
JTMS
Production_System
ILP
Forward_Chaining
Episodic
Backward_Chaining
Abduction
Subsymbolic
Reactive_Execution
Induction
Bayesian
Deliberation
Deduction
Future
Resolution
Future
Future
Compilation
EBG
Planning
Future
Future
Reflection
S_R_Rules
Preference_Based
Utility_Based
HTN
UC_POP
Future
Future
Knowledge_Domains
Knowledge_Representation
Future
First_Order_Logic
Logical
Space
Description_Logic
Probabilistic
Times
Episodic_Trace
Resource
Semantic_Network
Preferences
Associative
Hopfield_Network
Future
Future
Fuzzy
Future
7/12/2016 RBG
Future
Future
13
Lockheed Martin
Cognitive Computing
Requires Innovation
Classical Computing
Cognitive Processing
Markovian –current state
only
 Processor-oriented; favors
regular addressing





Procedural, results
oriented – apply this
function next



Key operations: arithmetic
& simple scalar decision
making



Single deterministic result
 Parallelism difficult to
extract
 Functional composition
determined at compile time
 Largely static resource
management
7/12/2016 RBG



14
History of prior results
guides next: “learning”
Memory-oriented;
unpredictable access
patterns, with metadata
guiding access
Goal oriented – with
multiple, possibly
incompatible objectives,
Process oriented – history +
new perceptions => new
knowledge
Context oriented –
computation based on
metadata from prior results
Key operations: wide
spectrum including complex
pattern matching
Often multiple “acceptable”
results
Speculation, futures a first
class activity
Functional composition
determined at run-time
Dynamic resource
management (Reasoning vs
Learning Balance)
ACIP Strawman Framework
Cognitive Functional Architecture
Reactive……………………….…..………..……..Deliberative
Tolerant……..…….………………..……………..……Intolerant
Transparent …………………………………………………Opaque
Embedded
Ground-Based
DoD Mission Requirements
Goals for Mission
Manage
Goals
Manage to Goals
Meta
ResourceManager
Manager
MetaCognitive
Cognitive Resource
Cognitive Algorithms
Learning
Knowledge
Reasoning
Probabilistic
Symbolic
Hybrid
Common Set of Services
Run-Time High
Level Compiler
Low Level Resource
Allocation & Meta Data
Management - Agents
Run-Time Low
Level Compiler
Micro Hardware
Architecture
GISC
C
V
M
PCA (SVM+TVM) + CVM = ACIP???
7/12/2016 RBG
15
Potential New Research Ideas!
Leveraging Embedded Computing Workshop Ideas
Chaired by
MIT LL and ISI
Future Role of Embedded Computing Devices:
GP, DSP, GPU, NIC, FPGA, ASIC
7/12/2016 RBG
16
Physical (COTS) PCA Systems
The Solution
The Problem
War Fighter
Software Development
Computation Density
Mission Application Software
D
Mission Software
E
High Level Software
Development Environment
C
PCA Morphware
Technology
B
Stable Architecture
Abstraction Layer (SAAL)
SVM
A
TVM
X
DTVM
Low Level Device
Vendor Specific
Game
Chips
FPGA
PCA
GPU &
NICs
Computational Efficiency
Developed under PCA program
Manual Low Visibility Stove Pipe
SW Development Environments
7/12/2016 RBG
Physical (COTS) PCA Systems Concept
17
Embedded Computing
Complexity Challenge
Embedded
Software Developer
The Solution:
Cognitive Software
Developer’s Assistant
Cognitive SW Development & Runtime Assistant
Software Developer’s Assistant
War Fighter
Mission Software
High Level Software
Development Environment
Stable Architecture
Abstraction Layer (SAAL)
SVM
TVM
X
DTVM
Low Level Device
Vendor Specific
Game
Chips
FPGA
PCA
GPU &
NICs
Developed under PCA program
7/12/2016 RBG
18
The Future is Yours
Become an DARPA Program
Manager!!
7/12/2016 RBG
19