Connectivity Modeling
Anthony Randal McIntosh
Department of Psychology
University of Toronto
Overview
• Theoretical issues and basis for analytic approach
• Structural equation modeling
– Integrating anatomy and function
• Partial Least Squares
– Identification of distributed systems
• Applications
– Sensory learning
– Working memory
Theoretical focus
• All behavioural and cognitive operations in the
brain come about through the the action of
distributed networks
• In order to assess this, need methods that can
measure the whole brain and analyses that look at
more than one region at a time
• Ideally, we would like to analyze spatial and
temporal patterns of brain function at the same
time
Causal patterns in brain research
Design/Task
Brain
Behaviour
Causal patterns in brain research
Response
Stimulus
Causal patterns in brain research
Response
Stimulus
Theory to Analysis
• Examine the influences between brain areas
– Interregional correlation (Horwitz, et al, 1984)
– Structural equation modeling (McIntosh & Gonzalez-Lima, 1991,
Buchel & Friston, 1997)
– Multiple regression and extensions (e.g., Kalman filters, Buchel &
Friston, 1998)
– Bayes networks (Dynamic Causal Modeling, Friston, Penny, et al,
2003)
• Identification of interacting regions
– Partial Least Squares (McIntosh, Bookstein, et al, 1996)
– Canonical Variates Analysis (Strother et al, 1995)
– Independent Components Analysis - 32 flavours (McKeown et al,
1998, Calhoun et al, 2001, Beckmann, Smith, et al., 2002)
Functional and Effective Connectivity
Structural Equation Modeling
• Multivariate multiple regression
• Combines interregional covariances with
anatomical framework
• Provides means to assess whether effective
connections are modified by task-demands or
differ between groups
• Is not meant to be a model test in the coventional
sense
– Goodness of fit not as relevant
Structural Equation Modeling
w = 0.611
A
x = 0.011
y = 0.614
C
A
B
C
D
B
z = -0.553
D
A
1.00
B
0.48
1.00
C
0.62
0.16
1.00
D
0.24
-.41
0.06
Structural Equations
A = xB + yC + 

B = wA + zD + 
B
1.00
Dorsal vs. Ventral Cortical Visual
Streams
Spatial Location
Object Identification
7
19d
46/47
46/47
17
18
17
18
19v
37
19v
37
21
Path Coefficients
Positive
Negative
0.7 to 1.0
0.4 to 0.6
0.1 to 0.3
0
McIntosh et al, J. Neurosci, 1994
21
What inferences does Structual
Equation Modeling allow?
Space
Object
21
21
21
7
37
19d
19v
7
46
46
46
46
21
7
37
37
19d
19d
19v
19v
Negative
Positive
0.65 - 1.0
0.35 - 0.65
0.1 - 0.35
0
7
19d
19v
37
What inferences does Structual
Equation Modeling allow?
Space
Object
21
21
21
7
37
19d
19v
7
46
46
46
46
21
7
37
37
19d
19d
19v
19v
Negative
Positive
0.65 - 1.0
0.35 - 0.65
0.1 - 0.35
0
7
19d
19v
37
Partial Least Squares
• “Least-squares” decomposition of “part” of a covariance
matrix
• PLS is optimized to explain the relation between two or
more blocks of data
– what pattern in one block most strongly covaries with a pattern in
another block?
• Ignores the relation among items within data blocks
• Statistical assessment through resampling algorithms
– Permutation test and bootstrap estimation of standard error
McIntosh, Bookstein, et al, Neuroimage, 1996
Task PLS
Compute matrix Mdev which is n*k by m
n is the number of subjects or repetitions,
k is the number of scans,
m is the number of voxels.
Each voxel is centered relative to the grand mean.
• Compute matrix X, a matrix of scan means expressed as
deviations from the grand mean.
Alternative: project (correlate) set of orthonormal contrasts
(design matrix) on to the data matrix
Matrix X now the covariance of image activity with the
experimental design.
Task PLS
• Perform a singular value decomposition (SVD) on X to
define the latent variables (LV):
SVD(X) = [U,S,V] where:
X = U*S*VT
U is the k by m orthonormal matrix containing voxel
weights (singular or eigen image).
S is a diagonal matrix of k singular (“eigen”) values. (The
kth singular value is zero because we use deviation
values in X, i.e., we eliminate the grand mean to create
X).
VT is the transpose of matrix V, a k by k orthonormal
matrix of scan weights.
• Project singular image on to original data to obtain “brain
scores”
Index of how well each subject shows the effect
Task PLS
Matrix M
Brain Images
Mean
Grand
Matrix X
Average deviation within scan,
of Matrix X
Singular Value Decomposition
V1
Matrix X
Average deviation within scan,
Matrix M
Brain Images
(Singular Image)
U1
(Singular Image)
U1
S1
Behavior PLS
• Compute matrix M which is n*k by m
n is the number of subjects/repetitions,
k is the number of scans,
m is the number of voxels.
• Create vector B, which is an n* k by 1 vector of
performance measures for each scan.
• Create matrix Y, which contains the scan-specific
correlations of voxel activity (Yk) and behavior
(Bk).
Behavior PLS
• Perform a singular value decomposition (SVD) on
Y to define the latent variables (LV):
SVD(X) = [U,S,V] where:
X = U*S*VT
U is the k by m orthonormal matrix containing
voxel weights (singular or eigen image).
S is a diagonal matrix of k singular (“eigen”)
values.
VT is the transpose of matrix V, a k by k
orthonormal matrix of scan weights.
Behavior PLS
Cross-correlation of B and M,
Matrix Y
Brain Images
Matrix M
Behavior
Measures
Matrix B
S1
Cross-correlation of B and M,
Matrix Y
V1
U1
(Singular Image)
Singular Value Decomposition
of Matrix M
4
3
2
0
Brain Scores
-2
-0.6
4
-0.4
-0.2
0.0
0.2
0.4
0.6
2
0
-2
-4
-1.0
-0.5
0.0
0.5
1.0
1.5
8
6
U1
4
(Singular Image)
2
0
Brain Images
Matrix M
-2
-4
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
Behavior Measure
Correlation of Brain
Scores & Behavior
1
-1
Statistical Assessment
• Assessment of omnibus/latent variable structure
through permutation tests
– Is the latent variable significantly different from
“noise”?
• Assessment of the precision of estimates derived
from PLS through bootstrap estimation of
standard errors
– How reliable is the answer?
• Procrustes rotation to original solution space used
to correct for axis rotation and reflection during
resampling
Milan & Whittaker, 1995, Royal Stat Society J
Why use bootstrap?
0.02
Permutation
0.015
• Estimation of standard errors is a direct
assessment of the stability of your data
0.01
Est Standard Error
0.005
0
0.02
Bootstrap
– A signal can be significantly different
from
noise (e.g., P<0.01), but not be reliable
0.015
0.01
0.005
0
0.02
ANOVA
0.015
0.01
0.005
0
-0.04
0
Singular Vector Weight
0.04
Causal patterns in brain research
Response
Stimulus
Multiblock PLS
Brain Images
Matrix M
Behavior
Measures
Matrix B
Average deviation within scan,
Matrix X
Normalize rows to unit-length
Cross-correlation of B and M,
Matrix Y
Normalize rows to unit-length
Grand
Mean
Brain Images
Matrix M
Multiblock PLS
Average deviation within scan,
Matrix X
S1
Cross-correlation of B and M,
Matrix Y
V1
U1
(Singular Image)
Matrix Z
U1
(Singular Image)
Brain Images
Matrix M
Within-task correlation of
brain scores and behavior
Average Brain Score
within-task
Singular Value Decomposition
of Matrix Z
How do we use this?
Target
Tone = 1 kHz FM ~ 65 dB - 500 ms duration
1
2
Scans
3
Distractor
Tone
lowP
P(Visual/Tone2)=0.2
Tone
highP
Reaction Time (msec)
Distractor
Unpaired trials
Paired trials
560
540
520
500
480
460
440
420
4
5
Tone
highP
Tone
lowP
Tone
highP
Tone
highP
Tone
highP
2
3
4
5
Scan
Tone
highP
P(Visual/Tone1)=0.7
6
Distractor
McIntosh, Cabeza & Lobaugh, J Neurophys 1998
Distractor
6
Sensory Associative Learning
1 Identify system(s) that respond to change
in significance of the tone
2 Identify system(s) that relate to (effect) a
change in behavior as a result of learning
3 Identify the overlap between 1 and 2
Task PLS
-28
-4
+20
16
Brain scores
14
12
10
8
6
4
2
VD1
TLP
THP1
THP2
Scan
THP3
VD2
Behaviour PLS
-28
-4
Brain Scores
+20
TLP
THP2
VD2
THP1
THP3
Behavior
Multiblock PLS
TLP
2
Brain Scores
Brain Scores
3
1
0
-1
THP2
VD2
THP1
THP3
-2
-3
TLP
THP1
THP2
THP3
VD2
Behavior Scores
Explaining regional activation
Behaviour
Adjusted rCBF
Adjusted rCBF
Task
Visual
Tone
Scan 1
Tone
Scan 2
Tone
Scan 3
Scan
Tone
Scan 4
Tone - scan 1
Tone - scan 2
Tone- scan 3
Visual
Tone scan 4
Visual
RT - Difference (unpaired - paired)
Seed voxel PLS
-4
+20
Adjusted rCBF
TLP
r = -.57
THP2 r = -0.22
Brain Scores
THP1 r = -.15
THP3 r = 0.72
Structural Equation Model
Positive
10
0.1 - 0.35
0.35 - 0.65
6
0.65 - 1.0
Negative
42
0.1 - 0.35
0.35 - 0.65
0.65 - 1.0
18
10
18
18
TLP
0
10
10
6
6
6
42
42
42
18
THP1
18
18
THP2
18
18
THP3
Occasions, Trials,
Subjects, Groups
ERP/MEG/fMRI Data Sets
SPACE
Voxels, detectors, electrodes
Occasions,Trials,
Subjects, Groups
ERP/MEG/fMRI Data
Flatten the matrix
Occasions,Trials,
Subjects, Groups
Space: Voxels/Channels
Time and Space
Task PLS
Matrix M
Brain Images
Mean
Grand
Matrix X
Average deviation within scan,
of Matrix X
Singular Value Decomposition
V1
Matrix X
Average deviation within scan,
Matrix M
Brain Images
(Singular Image)
U1
(Singular Image)
U1
S1
Spatiotemporal PLS- fMRI
Voxel
Saliences
Brain Images
Matrix Y
Temporal Brain Scores
How strongly does
the brain differentiate
tasks at each
timepoint???
McIntosh, Protzner & Chau, Neuroimage, in press
N-back Task Variant
Motivation
• Working memory may be conceived as the
interplay of sustained attention and memory
• For a given WM task - is there a
dissociation that would reflect the relative
contribution of an attentional component
v.s. recruitment of memory retrieval?
Lenartowicz & McIntosh, submitted
N-Back Task Variant
2-back (standard - Std)
0-back (detection)
1-back
Time
2-back (Cued)
Task PLS
0
8
2
6
6
0
8
4
Brain Score
Time (sec)
4
Bootstrap Ratio
4
2
0
-2
-4
-4
10
-6
-8
12
-8
14
+16
+24
+32
+40
+48
+56
+64
0
2
4
6
8
10
Time (sec)
0
-b
a
c
k
1
-b
a
c
k
C
u
e
d2
-b
a
c
k
S
td
2
-b
a
c
k
12
14
Task PLS
0
10
2
8
6
0
8
6
Brain Score
Time (sec)
5
Bootstrap Ratio
4
4
2
0
-2-
10
-5
0
2
4
6
8
10
Time (sec)
12
-10
14
-12
0
+12
+24
+36
+48
+60
0
-b
a
c
k
1
-b
a
c
k
C
u
e
d2
-b
a
c
k
S
td
2
-b
a
c
k
12
14
Task PLS
0.3
0.2
0.1
0
-0.1
-0.2
-0.3
-0.4
0
2
4
6
8
10
12
14
0
-b
a
c
k
1
-b
a
c
k
C
u
e
d2
-b
a
c
k
S
td
2
-b
a
c
k
Task PLS
0.25
0.2
0.15
0.1
0.05
0
-0.05
-0.1
-0.15
0
2
4
6
8
10
12
14
0
-b
a
c
k
1
-b
a
c
k
C
u
e
d2
-b
a
c
k
S
td
2
-b
a
c
k
Task PLS
• Dominant effect on anterior cingulate
activity
• What is the functional connectivity of the
anterior cingulate?
– Does it change between the 2-back tasks?
• Is the pattern of functional connectivity
related to behavior?
Behaviour/Seed PLS
15
10
1
5
0
Correlation
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
Std 2-back
Cued 2-back
-5
-10
-16
-4
+8
+20
+32 +44
+56 +68
AC
RT
Hits
Behaviour/Seed PLS
10
1
5
0
Correlation
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-5
-10
-16
-4
+8
+20 +32 +44
+56 +68
-1
Std 2-back
Cued 2-back
AC
RT
Hits
Summary & Implications
• Anterior cingulate activity differentiates tasks
based on attentional demands
• Functional connectivity varies with attentional
demand
• Relation of functional connectivity patterns to
performance also varies with attentional demand
• Behavioural relevance of a region to a cognitive
operation depends on its pattern of functional
connectivity
– Neural Context - McIntosh, Neural Networks 2001
Evaluating the analytic tools:
How do we know when the math is right?
• Neurobiological interpretation
– Identification of new principles
• Psychological interpretation
– What is the level of nervous system operation that best
relates to the cognitive operation?
• What is the question?
– Level of the answer
– Does the experimental design require a certain analytic
tool?
– Is causality necessary or will correlation suffice?
Thank You
Acknowledgements:
Collaborators: NJ Lobaugh, MN
Rajah, CL Grady
Funding: CIHR, NSERC, JS
McDonnell Fnd
http://www.rotman-baycrest.on.ca
What inferences does Structual
Equation Modeling allow?