Cracking the Population Code
Dario Ringach
University of California, Los Angeles
The Questions
Two basic questions in cortical computation:
How is information represented?
How is information processed?
Representation by Neuronal Populations
How is information encoded in populations of neurons?
Representation by Neuronal Populations
How is information encoded in populations of neurons?
1. Quantities are encoded as rate codes in ensembles of 50100 neurons (eg, Shadlen and Newsome, 1998).
Representation by Neuronal Populations
How is information encoded in populations of neurons?
1. Quantities are encoded as rate codes in ensembles of 50100 neurons (eg, Shadlen and Newsome, 1998).
2. Quantities are encoded as precise temporal patterns of
spiking across a population of cells (e.g, Abeles, 1991).
Representation by Neuronal Populations
How is information encoded in populations of neurons?
1. Quantities are encoded as rate codes in ensembles of 50100 neurons (eg, Shadlen and Newsome, 1998).
2. Quantities are encoded as precise temporal patterns of
spiking across a population of cells (e.g, Abeles, 1991).
3. Quantities might be encoded as the variance of responses
across ensembles of neurons (Shamir & Sompolinsky, 2001;
Abbott & Dayan, 1999)
Coding by Mean and Covariance
Responses of two neurons to the repeated presentation of two stimuli:
Neuron #2
Mean only
B
A
Neuron #1
Averbeck et al, Nat Rev Neurosci, 2006
Coding by Mean and Covariance
Responses of two neurons to the repeated presentation of two stimuli:
Neuron #2
Mean only
Covariance only
B
B
A
Neuron #1
A
Neuron #1
Averbeck et al, Nat Rev Neurosci, 2006
Coding by Mean and Covariance
Responses of two neurons to the repeated presentation of two stimuli:
Mean only
Covariance only
Both
Neuron #2
A
B
B
B
A
Neuron #1
A
Neuron #1
Neuron #1
Averbeck et al, Nat Rev Neurosci, 2006
Macaque Primary Visual Cortex
Orientation Tuning
Receptive field
Orientation Columns
Primary Visual Cortex
V1 surface and vasculature under green illumination
4mm
Orientation Columns and Array Recordings
Optical imaging of intrinsic signals under 700nm light
1mm
Alignment of Orientation Map and Array
Find the optimal translation and rotation of
the array on the cortex that maximizes the
agreement between the electrical and optical
measurements of preferred orientation.
qelec
(3 parameters and 96 data points!)
qoptical
Error surfaces:
0.4
ty
ty
f
0.0
tx
tx
f
Micro-machined Electrode Arrays
Array Insertion Sequence
1
2
3
4
Basic Experiment
Input
q (t )Î S 1
Output
r (t + t )Î Rd
qk = k p / N
k = 0, L , N - 1
We record single unit activity (12-50 cells), multi-unit activity (50-80 sites)
and local field potentials (96 sites). What can we say about:
P (r (t + t )| q (t ))
Dynamics of Mean States
ìïï 1 if q (t )= qi
qi (t )º í
ïïî 0 otherwise
mi (t )= E {r (t + t )| qi (t )= 1}
m1
m18
m2
m3
Dynamics of Mean Responses
Multidimensional scaling to d=3 (for visualization only)
Dynamics of Mean Responses
Multidimensional scaling to d=3 (for visualization only)
Stimulus Triggered Covariance
S i (t ) = E {D r (t + t )D rT (t + t )| qi (t ) = 1}
m1
m18
S2
m2
m3
S3
Covariance matrices are low-dimensional
Average spectrum for co-variance matrices in two experiments
li
Covariance matrices are low-dimensional (!)
Two Examples
Bhattacharyya Distance and Error Bounds
m1
m
P (r | qi ) : N 18
(mi , S i )
Si
mi
mj
Sj
Bhattacharyya distance:
S1 + S 2
1
1
- 1
T
BD = D m (S 1 + S 2 ) D m+ log
4
2
2 S 1S 2
Differences in mean
Differences in co-variance
P (error )< exp (- BD / 2)
Information in Covariance
?
Information in Mean
Bayes’ Decision Boundaries – N-category classification
Si
P (r | qi ) : N (mi , S i )
Hyperquadratic decision surfaces
t
t
i
gi (x)= x Wi x + w x + wi 0
Where:
1 -1
Wi = - S i
2
wi = S -i 1mi
1 t -1
1
wi 0 = - mi S i mi - log S -i 1 + log P (qi )
2
2
mi
mj
Sj
Confusion Matrix and Probability of Classification
Confusion Matrix and Probability of Classification
Stimulus-Triggered Responses
n=41 channels ordered according their preferred orientation
Channel # (orientation)
2.0
Dr / r
0.0
150ms
Stimulus-Triggered Responses
n=32 channels ordered according their preferred orientation
Channel # (orientation)
2.0
Dr / r
0.0
150ms
Mean Population Responses
Mean Population Responses
Population Mean and Variance Tuning
(l , s 2 )
Population Mean and Variance Tuning
(l , s 2 )
Population Mean and Variance Tuning
(l , s 2 )
Population Mean and Variance Tuning
Population Mean and Variance Tuning
Population Mean and Variance Tuning
Bandwidth of Mean and Variance Signals
Estimates of Mean and Variance in Single Trials
Population of independent Poisson spiking cells:
1
l = å li
n i
1
s = å (l i - l
n i
2
{l i }
2
)
+l
Estimating Mean and Variances Trial-to-Trial
Noise correlation = 0.0
mean
variance
Estimating Mean and Variances Trial-to-Trial
Noise correlation = 0.1
variance
mean
Estimating Mean and Variances Trial-to-Trial
Noise correlation = 0.2
variance
mean
Dimension #2
Tiling the Stimulus Space and Response Heterogeneity
Dimension #1
Orientation
Tiling the Stimulus Space and Response Heterogeneity
Dimension #2
Population response to the
same stimulus
Dimension #1
Orientation
Tiling the Stimulus Space and Response Heterogeneity
Dimension #2
Population response to the
same stimulus
Dimension #1
Orientation
Tiling the Stimulus Space and Response Heterogeneity
Dimension #2
Population response from independent
single cell measurements
Dimension #1
Orientation
Tiling the Stimulus Space and Response Heterogeneity
Dimension #2
Population response from independent
single cell measurements
Dimension #1
Orientation
Can single cells respond to input variance?
Silberberg et al, J Neurophysiol., 2004
Can single cells respond to input variance?
Silberberg et al, J Neurophysiol., 2004
Summary
• Heterogeneity leads to population variance as a natural
coding signal in the cortex.
• Response variance has as smaller bandwidth than the mean
response.
• For small values of noise correlation variance is already a
more reliable signal than the mean.
Summary
• In a two-category classification problem the variance
signal carries about 95% of the total information (carried by
mean and variance together.)
• The covariance of the class-conditional population
responses is low dimensional, with the first eigenvector most
likely indicating fluctuations in cortical excitability (or gain).
• Cells may be perfectly capable of decoding the variance
across their inputs (Silberberg et al, 2004)
• In prostheses, the use of linear decoding based on
population rates may be sub-optimal. Quadratic models
y = xt Hx may work better.
Acknowledgements
V1 imaging/electrophysiology (NIH/NEI)
Neovision phase 2 (DARPA)
Brian Malone
Andy Henrie
Ian Nauhaus
Frank Werblin (Berkeley)
Volkan Ozguz (Irvine Sensors)
Suresh Subramanian (Irvine Sensors)
James DiCarlo (MIT)
Bob Desimone (MIT)
Tommy Poggio (MIT)
Dean Scribner (Naval Research Labs)
Topological Data Analysis (DARPA)
Gunnar Carlsson (Stanford)
Guillermo Sapiro (UMN)
Tigran Ishakov (Stanford)
Facundo Memoli (Stanford)
Bayesian Analysis of Motion in MT (NSF/ONR)
Alan Yuille (UCLA)
HongJing Lu (Hong Kong)