Using natural stimuli
Reductionism
Reducing complex things into simpler components
Explaining the “whole” as a sum of its parts
Duck behavior is the
sum of its automatically
behaving parts.
Descartes 1662
Vision
Decompose visual experience into:
Locations in visual field
Contrast
Orientation
Spatial frequency
Direction of motion
Categories – objects, faces, houses
Visual experiment
Change a stimulus attribute in a controlled manner and see
how the neural response changes (easy to do!).
Repeat many times and average.
Visual system
Different neurons in different visual system areas
process specific components:
Spatial receptive field
Contrast and spatial frequency sensitivity
Color sensitivity
Selectivity for orientation, direction of motion, visual
category.
Temporal dependence, adaptation.
What happens in real life?
Inter-subject correlation
Correlate the responses across subjects/runs
Hasson et. al. Science 2004
Global component
General response across many
areas – similar across subjects.
Due to structure of movie?
General arousal?
Reverse correlation
Reverse correlation
Natural stimulus
Despite the complexity of the stimulus:
1. Categorical visual areas maintain selectivity.
2. Responses are similar across different subjects.
3. Reverse engineering works: can go from brain to
stimulus.
4. Or “map” multiple areas at once by breaking down
the stimulus into visual components (e.g.
retinotopic mapping).
Temporal receptive windows
Hasson et. al. J Neurosci. 2008
Temporal receptive windows
Do the neurons care
about the momentary
stimulus being
presented, or about
its context within a
certain temporal
history…
Temporal receptive windows
Scrambling the movie
at different segment
lengths.
Inter-subject
correlation in some
cortical areas
depended on the
“temporal continuity”
of the movie.
In autism
Individuals with autism show weak inter-subject correlations
Hasson et. al. Aut. Res. 2009
In autism
Individual variability, but group averages were similar…
Total = Sum of components?
Data driven multivariate analyses
Our variables are voxels
We assume that voxel fMRI measurements represent a sum
of separate linear components (separate “brain processes”)
that are mixed in some unknown way.
Find a mathematical criteria to separate the data into
meaningful components:
Principle component analysis (PCA)
Independent component analysis (ICA)
Clustering algorithms: K means, spectral clustering, etc…
Principal component analysis
PCA is a way of representing the data in components that
are orthogonal (dot product = 0, correlation = 0), while
ordering them by the amount of variability that they explain.
In two dimensions
1. Normalize the data (% sig change).
2. Find the direction with largest
variability (1st component). Data are
most correlated along this
direction….
3. Add it’s orthogonal direction (2nd
component).
Principal component analysis
fMRI data has as n by m dimensions (n voxels and m
time-points). Perform PCA on the temporal dimensions.
Num voxels
Num TRs
Computing the components
One way of computing a PCA is using singular value
decomposition (SVD):
Data = U * S * VT
n by m matrix
n by m matrix
Eigenvectors
n by n matrix
Eigenvalues
n by n matrix
n = num of trs
m = num of voxels
Data structure
Variance
If the data is correlated, it will have components
(eigenvectors) that will explain a large part of the
variability…
Reduce the dimensionality of the data?
Spatio-temporal components
Brain areas that “work together” will be correlated in
time
The “weighting” of a particular component in the
different voxels:
The dot product of a
voxel’s “weights” and
the components matrix
will give the original
voxel’s timecourse
Principle component analysis
Are orthogonality and “variability explained” good
criteria for separating independent brain processes?
How many simultaneous processes are there?
Are they correlated in time?
Reliability across scans and subjects?
Typically more than one PCA solution….
Has mostly been used for dimensionality reduction
(good for compressing data).
Independent component analysis
A different algorithm that separates components such that
they are “statistically independent”.
Pr(A ∩ B) = Pr(A) * Pr(B)
Cov(X,Y) = 0
Independent variables are uncorrelated, but not all
uncorrelated variables are independent…
They need to have a joint distribution fulfilling:
Good for separating audio
Two microphones:
Microphone 1
Microphone 2
Two sources:
Source 1
Source 2
Good for cleaning EEG data
Spatially consistent “sparse” processes.
Separating fMRI components
Independent component analysis
And with fMRI “free viewing” movie data:
Independent component analysis
Things to think about:
1. You decide how many independent components to split
the data into (arbitrary choice).
2. Reliability across scans and subjects.
3. How can we tell whether the components are
biologically meaningful?
To the lab!