Applying Category Theory to
Improve the Performance of a
Neural Architecture
Michael J. Healy Richard D. Olinger
Robert J. Young Thomas P. Caudell
University of New Mexico
Kurt W. Larson
Sandia National Laboratories
This work was supported in part by Sandia National Laboratories, Albuquerque,
New Mexico, under contract no. 238984. Sandia is a multiprogram laboratory operated
by Sandia Corporation, a Lockheed Martin Company, for the United States Department
of Energy's National Nuclear Security Administration under
Contract DE-AC04-94AL85000.
Semantic Representation
T’
M (T’)
P3
Functor M
m
P2
M (m)
M (T)
T
Concept
category
Neural
category
Neural
network
P1
T’
Model-Space Morphisms ==>
Reciprocal Connections
Mod(T’)
Instances
of
M(T’)
P3
Functor M
m
M(m)
Functor Mod
Mod(m)
Instances
of
T
P2
Mod(T)
M(T)
P1
Colimits Express Specialization Limits Express Abstraction
Least specialization
T5
T3
T4
T3
T1
T2
T2
T1
T5
Maximally specific abstraction
Classifying Pixels by Spectral Similarity
Neural network
classifier
Intensities for
10 optical bands
…
.
.
.
Multispectral
camera data
Pixel class
(color)
Colored
pixel i
Data for pixel i
( = one input pattern)
Multispectral
image
Stack Interval Network
Positive stack nodes
-
-
Complement stack nodes
-
-
StimLB0
- - -
-
0
+
N-1
+
+ +
StimVal
+
N
+
+
StimUBN-1
2 N-1
+
Stack Interval Patterns
Represent Real Intervals
Positive stack
Complement
0 < v <= 1
Width 1 unit
Positive stack
1 < v <= 2
Complement
Width 1 unit
Intersection of stack patterns (in template patterns)
0 < v <= 2
Width 2 units
ART-1 with Stack Interval Inputs
...
F2
Template
pattern
GC
F1
+
Band 1
b1
F0
+
b1c
Band 2
b2
b2c
Composite input pattern
(two stack
Intervals)
V
+
ART-1 + F1 Colimits, Limits
...
F2
-
+
L
F+1
F1
-
+
S
F+-1
...
F0
...
+
V
+
Panchromatic Image - 1 m Resolution
Multispectral Image - Generic ART-1
r = 0.795
Template density ordering
Multispectral Image - ART-1 with Limits
r = 0.55 F-1 tol = 0.55 Template density ordering
References
M. J. Healy, R. D. Olinger, R. J. Young, T. P. Caudell,
and K. W. Larson, “Applying Category Theory to
Improve the Performance of a Neural Architecture”
(under review).
M. J. Healy and T. P. Caudell (2006) “Ontologies and Worlds
in Category Theory: Implications for Neural Systems”,
Axiomathes, 16 (1), pp. 165-214.
M. J. Healy and T, P. Caudell (2004) “Neural Networks,
Knowledge, and Cognition: A Mathematical
Semantic Model Based upon Category Theory”,
UNM Technical Report EECE-TR-04-020,
University of New Mexico, Albuquerque, NM, USA .
Template Patterns
Template 1
Band
2
Template 2
Band 1
Stack Numeral Quanta
v= 0
Width 0
units
0 < v <= 1
Width 1 unit
..
.
..
.
..
.
3 < v <= 4
Width 1 unit
0 <= v <= 1
Width 1 unit
0 < v <= 2
Width 2 units
2 < v <= 4
..
.
..
.
..
.
Width 2 units
Neural Network Research Objective:
Associate an Evolving Knowledge Structure with
Neural Structure and Activity
Concept
hierarchy
Learning and
representation
Modality-specific
input streams
Motor functions
Neural network
Sensors and
actuators
Event stream
Environment
Limits Express Abstraction
T3
T1
T2
T5 … maximally specific abstraction
Colimits Express Specialization
T5
… least specialization
T4
T3
T1
T2