Cognitive Computing via Synaptronics and Supercomputing © 2008 IBM Corporation

Cognitive Computing via

Synaptronics and Supercomputing

© 2008 IBM Corporation

"The information that comes from deep in the evolutionary past we call genetics. The information passed along from hundreds of years ago we call culture. The information passed along from decades ago we call family, and the information offered months ago we call education. But it is all information that flows through us. The brain is adapted to the river of knowledge and exists only as a creature in that river. Our thoughts are profoundly molded by this long historic flow, and none of us exists, self-made, in isolation from it."


Inflection Point 1:

Neuroscience has matured

1414 pages


Inflection Point 2:

Supercomputing meets Brain


Mammalian-scale simulation in near real-time?

Memory Computation

Communication


BlueGene Meets Brain

Mouse

N: 16 x 10 6

S: 128 x 10 9

Rat

56 x 10 6

448 x 10 9

Cat

763 x 10 6

6.1 x 10 12

Monkey

2 x 10 9

20 x 10 12

Human

22 x 10 9

220 x 10 12

New results for SC09

Almaden

BG/L

December, 2006

Watson

BG/L

April, 2007

WatsonShaheen

BG/P

March, 2009

LLNL Dawn

BG/P

May, 2009

Latest simulations achieve unprecedented scale of

10 9 neurons and 10 13 synapses


Inflection Point 3:

Nanotechnology meets Brain


Power

Space

Rat

50 mW

6 cm 2

Human

20 W

2,400 cm 2

Novel non-von Neumann Architectures are necessary

Brain

~10 10 synapses/cm 2

Data from Todd Hylton

Neuromorphic Electronics

10 10 intersection/cm 2 in 100 nm crossbar

~10 6 Neurons/cm 2 ~5x10 8 transistors/cm 2 in state of the art CMOS

~5 x 10 8 long range axons @ ~1 Hz

~30 Gbit/sec multiplexed digital addressing

Brain can be realized in modern electronics


Turning Back the Clock

Digital, synchronous conventional, 5GHz

(compare Power 6 , 2008)

Digital, semi-synchronous, 5 MHz

(compare IBM PC/8088 , 1978)

Digital, asynchronous, 100 kHz

(compare ENIAC , 1946)

Digital-Analog, asynchronous, clockless

(compare the Brain )

Commandment:

Do what is necessary, when it is necessary, and only that which is necessary.


Dharmendra S Modha

IBM Research – Almaden

Raghavendra Singh

IBM Research – India

Network Architecture of the

White Matter Pathways in the Macaque Brain

PNAS (July 2010)


The connection model

 Cortex has evolved such that it is organized into areas with distinct structural and functional properties

•

•

•

Primary sensory areas

Association areas

Motor areas

 The white matter (myelinated nerve cell) underneath the outer covering of gray matter (nerve cell bodies), interconnects different regions of the central nervous system and carries nerve impulses between neuron

 Model each area as a node and each connection as an edge in a graph

– Analysis and Visualization of the brain

•

• Wire length minimization

Organizational model that suggest the flow of information from input of sensory signals to the eventual output by motor neurons

– Use model to simulate dynamics in the simulator


CoCoMac: Connectivity data on the Macaque brain

Rolf Kotter, Klass Stephen, 2000

413 literature reports

7007 brain sites

8003 mapping details

2508 tracer injections

39748 connection details


Divergent Nomenclature and Abundant Conflicts

FV91-V1

AP84-TE

PG91b-IT

FV91-V4

BP82-46

SP89a-46

FV91-TF

RV99-TF

FV91-TH

BR98-TH

RV99-CA1

BR98-CA1

כ כ

PFCd

=

9

CCa

=

כ

PCi

=

24c

=

24

=

כ

24d

=

IPL

=

24c

=

PF#1

=

CMAr

7b












Bundling Algorithm by Holten, 2006





Kaiser, Hilgetag, 2006


Notable Collations

Species Study

Monkey Felleman, Van Essen, 91

Cat

Young, 93


Scannell et al., 95

Scannell et al., 99

Areas Connections

32 305

70

95

65

95

700

2,402

1,139

1,500


Notable Collations

Species Study

Monkey Felleman, Van Essen, 91

Young, 93


Cat Scannell et al., 95

Scannell et al., 99

Monkey This Collection

Areas Connections

32 305

70

95

65

95

383

700

2,402

1,139

1,500

6,602



Cingulum Bundle


Uncinate Fasciculus


C, C, C, C, C, and C

Complete Cortex, Thalamus, Basal Ganglia

Comprehensive Includes every study in CoCoMac

Consistent Every connection can be tracked back

Concise

Coherent

Colossal

6,877 areas to 383

Unified hierarchical parcellation

3 times larger than previous network wetware to software


Aggregate Statistics

Nodes

Edges

383

6,602

Density 4.5% of possible connections exist

Reciprocity 42%

SCC 351 areas, 6,491 edges


Diameter

Brain is small-world

SCC: 351 areas, 6,491 connections

Brain

6

Random

(100 trials)

3.93

Characteristic Path Lengh 2.62

2.30

Average Clustering Coefficient

0.33

0.0528

Reciprocity

42% 5.34%


“Organized Complexity” – Weaver, 1948


Degree Distribution Consistent with Exponential


Prefrontal Cortex is Topologically Central



Brain is small-world, Core is “tiny”-world!

Brain Core

Diameter 6 4

Characteristic Path Lengh 2.62

1.95

Average Clustering Coefficient

0.33

0.39

Core contains only 32% of vertices yet 88% of all edges originate or terminate in the core


Core contains correlated-anti-correlated networks and may be a key to consciousness

Fox, Snyder, Vincent, Corbetta, Van Essen, and Raichle, 2005


Inter-chip Connectivity


Rent ’s Rule

 Rent's rule pertains to the organization of computing logic, specifically the relationship between the number of external signal connections (C) to a logic block with the number of logic gates (N) in the logic block

 E.F. Rent observed a powerlaw relationship in the 1960’s - the law has been shown to hold true for small circuits upto mainframe computers

C

 kN p

0 ≤ p≤1 is the Rent parameter and k is the Rent coefficient.

 Intrinsically it ’s a surface area (wire) to volume (number of nodes) relationship

– Represents a cost-efficient solution to the challenge of embedding a high dimensional functional interconnect topology in a relatively low dimensional physical space with economical wiring costs



–



Microprocessor (0.45), Gates Arrays (0.5), High speed Computers (0.63)

 For 2D layouts p> 0.5 implies that wires must grow longer as circuit size increases; global connections dominate over local connections for large p

– The relative contribution of wiring to layout area will grow with the size of circuit to allow space for a greater number of wires to pass between adjacent nodes, increasing the node-to-node spacing

 Allometric scaling

– Gray (physical) to white (logical) matter scaling - Zhang & Sejnowski


 High value of p

– Topological dimensionality of network greater than

3, i.e., greater than the dimensionality of the Euclidean space in which the network is embedded

– Communication is a significant factor of power and space

–

Tradeoff between wiring costs and greater logical capacity.

•

Rewiring the network so as to reduce its topological dimension results in loss of functional modularity

Rent ’s Rule



“white matter is nature’s finest masterpiece”

Nicolaus Steno, 1669


Owing both to limitations in hardware and architecture, these (convential) machines are of limited utility in complex, real-world environments, which demand an intelligence that has not yet been captured in an algorithmiccomputational paradigm. As compared to biological systems for example, today’s programmable machines are less efficient by a factor of one million to one billion in complex, real-world environments. The SyNAPSE program seeks to break the programmable machine paradigm and define a new path forward for creating useful, intelligent machines.

The vision for the anticipated DARPA SyNAPSE program is the enabling of electronic neuromorphic machine technology that is scalable to biological levels . Programmable machines are limited not only by their computational capacity, but also by an architecture requiring (human-derived) algorithms to both describe and process information from their environment. In contrast, biological neural systems (e.g., brains) autonomously process information in complex environments by automatically learning relevant and probabilistically stable features and associations. Since real world systems are always many body problems with infinite combinatorial complexity, neuromorphic electronic machines would be preferable in a host of applications —but useful and practical implementations do not yet exist.


Cognitive Computing via Synaptronics and Supercomputing © 2008 IBM Corporation

Cognitive Computing via

Synaptronics and Supercomputing

Inflection Point 1:

Neuroscience has matured

Inflection Point 2:

Supercomputing meets Brain

Mammalian-scale simulation in near real-time?

BlueGene Meets Brain

Inflection Point 3:

Nanotechnology meets Brain

Turning Back the Clock

Network Architecture of the

White Matter Pathways in the Macaque Brain

Notable Collations

Notable Collations

C, C, C, C, C, and C

Aggregate Statistics

Brain is small-world

0.33

0.0528

42% 5.34%

Prefrontal Cortex is Topologically Central

Brain is small-world, Core is “tiny”-world!

0.33

0.39

“white matter is nature’s finest masterpiece”

Related documents

Products

Support

Cognitive Computing via Synaptronics and Supercomputing © 2008 IBM Corporation