Classical inference and
design efficiency
Zurich SPM Course 2014
Jakob Heinzle
heinzle@biomed.ee.ethz.ch
Translational Neuromodeling Unit (TNU)
Institute for Biomedical Engineering (IBT)
University and ETH Zürich
Translational Neuromodeling Unit
Many thanks to K. E.
Stephan, G. Flandin and
others for material
Overview
Image time-series
Realignment
Spatial filter
Design matrix
Smoothing
General Linear Model
Statistical Parametric Map
Statistical
Inference
Normalisation
Anatomical
reference Parameter estimates
RFT
p <0.05
Classical Inference and Design Efficiency
2
A mass-univariate approach
Classical Inference and Design Efficiency
3
Estimation of the parameters
i.i.d. assumptions:
OLS estimates:
𝜀~𝑁(0, 𝜎 2 𝐼)
𝛽 = (𝑋 𝑇 𝑋)−1 𝑋 𝑇 𝑦
𝛽1 = 3.9831
𝛽2−7 = {0.6871, 1.9598, 1.3902, 166.1007, 76.4770, −64.8189}
𝛽
𝑦 =
+𝜀
𝛽8 = 131.0040
𝜀=
𝛽~𝑁
𝛽, 𝜎 2 (𝑋 𝑇 𝑋)−1
𝜎2
=
𝜀𝑇 𝜀
𝑁−𝑝
Classical Inference and Design Efficiency
4
Contrasts
 A contrast selects a specific effect of interest.
[1 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
0-1
 A contrast 𝑐 is a vector of length 𝑝.
 𝑐 𝑇 𝛽 is a linear combination of regression
coefficients 𝛽.
𝑐 = [1 0 0 0 … ]𝑇
𝑐 𝑇 𝛽 = 𝟏 × 𝛽1 + 𝟎 × 𝛽2 + 𝟎 × 𝛽3 + 𝟎 × 𝛽4 + ⋯
= 𝜷𝟏
𝑐 = [1 0 0 − 1 0 … ]𝑇
𝑐 𝑇 𝛽 = 𝟏 × 𝛽1 + 𝟎 × 𝛽2 + 𝟎 × 𝛽3 + −𝟏 × 𝛽4 + ⋯
= 𝜷𝟏 − 𝜷𝟒
𝑐 𝑇 𝛽~𝑁 𝑐 𝑇 𝛽, 𝜎 2 𝑐 𝑇 (𝑋 𝑇 𝑋)−1 𝑐
Classical Inference and Design Efficiency
5
Hypothesis Testing - Introduction
Is the mean of a measurement different from zero?
Mean of several
measurements
Many experiments
𝑇=𝜎
0
Exp A
Exp B
m1, s1 m2, s2
𝜎𝜇 =
𝜎1,2
𝑛
𝜇
𝑛
Null distribution
Null Distribution of T
Ratio of
effect vs. noise
 t-statistic
What distribution of T
would we
get for m = 0?
Classical Inference and Design Efficiency
6
Hypothesis Testing
To test an hypothesis, we construct “test statistics”.
 Null Hypothesis H0
Typically what we want to disprove (no effect).
 The Alternative Hypothesis HA expresses outcome of interest.
 Test Statistic T
The test statistic summarises evidence
about H0.
Typically, test statistic is small in
magnitude when the hypothesis H0 is true
and large when false.
 We need to know the distribution of T
under the null hypothesis.
Null Distribution of T
Classical Inference and Design Efficiency
7
Hypothesis Testing
u
 Significance level α:
Acceptable false positive rate α.
 threshold uα
Threshold uα controls the false positive rate
t

  p (T  u  | H 0 )
Null Distribution of T
 Conclusion about the hypothesis:
We reject the null hypothesis in favour of the
alternative hypothesis if t > uα
 p-value:
A p-value summarises evidence against H0.
This is the chance of observing a value more
extreme than t under the null hypothesis.
𝑝 𝑇 > 𝑡|𝐻0
t
p-value
Null Distribution of T
Classical Inference and Design Efficiency
8
T-test - one dimensional contrasts – SPM{t}
cT = 1 0 0 0 0 0 0 0
effect of interest > 0 ?
=
amplitude > 0 ?
=
b1 = c T b > 0 ?
Question:
b1 b2 b3 b4 b5 ...
H0: cTb=0
Null hypothesis:
contrast of
estimated
parameters
T=
Test statistic:
T 
T
c bˆ
T
var( c bˆ )
variance
estimate
T
c bˆ

sˆ c
2
T
X
T
X

1
~ tN  p
c
Classical Inference and Design Efficiency
9
T-contrast in SPM
 For a given contrast c:
ResMS image
beta_???? images
bˆ  ( X T X ) 1 X T y
con_???? image
c T bˆ
2
sˆ 
T
ˆ ˆ
N  p
spmT_???? image
SPM{t}
Classical Inference and Design Efficiency
10
T-test: a simple example
 Passive word listening versus rest
Q: activation during
listening ?
Null hypothesis: b1 = 0
𝑡=
𝑐𝑇 𝛽
var
𝑐𝑇 𝛽
T
Threshold T = 3.2057 {p<0.001}
SPMresults:
voxel-level
( Z)
13.94
12.04
11.82
13.72
12.29
9.89
7.39
6.84
6.36
Inf
Inf
Inf
Inf
Inf
7.83
6.36
5.99
5.65
p uncorrected
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
Mm mm
-63
-48
-66
57
63
57
36
51
-63
mm
-27 15
-33 12
-21
6
-21 12
-12 -3
-39
6
-30 -15
0 48
-54 -3
Classical Inference and Design Efficiency
11
T-test: summary
T-test is a signal-to-noise measure (ratio of
estimate to standard deviation of estimate).
 Alternative hypothesis:
H0: c T b  0
vs
HA: c T b  0
 T-contrasts are simple combinations of the betas; the Tstatistic does not depend on the scaling of the regressors
or the scaling of the contrast.
Classical Inference and Design Efficiency
12
Scaling issue
[1
1
1
1
]/ 4
T 
T
c bˆ
var( c bˆ )
Subject 1
T
T
c bˆ

sˆ c
2
T
X
T
X

1
c
The T-statistic does not depend on the
scaling of the regressors.
 The T-statistic does not depend on
the scaling of the contrast.
Subject 5
[1
1
1
]/ 3
𝛽 = (𝑋 𝑇 𝑋)−1 𝑋 𝑇 𝑦
T
 Contrast c bˆ depends on scaling.
 Be careful of the interpretation of the
T
contrasts c bˆ themselves (eg, for a
second level analysis):
sum ≠ average
Classical Inference and Design Efficiency
13
F-test - the extra-sum-of-squares principle
Model comparison:
Null Hypothesis H0: True model is X0 (reduced model)
X0
X1
X0
Test statistic: ratio of
explained variability and
unexplained variability (error)
F 
RSS

ˆ full
2
RSS
ˆ reduced
2
F 
ESS
RSS
Full model ?
or Reduced model?
 RSS
RSS
RSS0

0
~ F 1 , 2
1 = rank(X) – rank(X0)
2 = N – rank(X)
Classical Inference and Design Efficiency
14
F-test - multidimensional contrasts – SPM{F}
Test multiple linear hypotheses:
Null Hypothesis H0: b3 = b4 = b5 = b6 = b7 = b8 = 0
X0
X1 (b3-8)
cT =
Full model ?
c Tb = 0
00100000
00010000
00001000
00000100
00000010
00000001
Is any of b3-8 not equal 0?
Classical Inference and Design Efficiency
15
F-contrast in SPM
ResMS image
beta_???? images
bˆ  ( X T X ) 1 X T y
2
sˆ 
T
ˆ ˆ
N  p
ess_???? images
spmF_???? images
( RSS0 - RSS )
SPM{F}
Classical Inference and Design Efficiency
16
F-test example: movement related effects
contrast(s)
contrast(s)
10
20
30
40
50
60
70
80
10
20
2
4
6
8
Design matrix
30
40
50
60
70
80
2
4
6
8
Design matrix
Classical Inference and Design Efficiency
17
F-test: summary
F-tests can be viewed as testing for the additional variance
explained by a larger model wrt a simpler (nested) model 
model comparison.
 F tests a weighted sum of squares of one or several
combinations of the regression coefficients b.
 In practice, we don’t have to explicitly separate X into [X1X2]
thanks to multidimensional contrasts.
 Hypotheses:
1

0

0

0
0
0
1
0
0
1
0
0
0

0

0

0
Null Hypothesis
Alternativ
H 0 : b1  b 2  b 3  0
e Hypothesis
H A : at least one b k  0
 In testing uni-dimensional contrast with an F-test, for example
b1 – b2, the result will be the same as testing b2 – b1. It will be
exactly the square of the t-test, testing for both positive and
negative effects.
Classical Inference and Design Efficiency
18
Orthogonal regressors
Variability described by 𝑋1
Testing for 𝑋1
Variability described by 𝑋2
Testing for 𝑋2
Variability in Y
Classical Inference and Design Efficiency
19
Correlated regressors
Variability described by
Variability described by
Shared variance
Variability in Y
Classical Inference and Design Efficiency
20
Correlated regressors
Variability described by
Variability described by
Testing for 𝑋1
Variability in Y
Classical Inference and Design Efficiency
21
Correlated regressors
Variability described by
Variability described by
Testing for 𝑋2
Variability in Y
Classical Inference and Design Efficiency
22
Variability described by
Variability described by
Correlated regressors
Variability in Y
Classical Inference and Design Efficiency
23
Correlated regressors
Variability described by
Variability described by
Testing for 𝑋1
Variability in Y
Classical Inference and Design Efficiency
24
Correlated regressors
Variability described by
Variability described by
Testing for 𝑋2
Variability in Y
Classical Inference and Design Efficiency
25
Correlated regressors
Variability described by
Variability described by
Testing for 𝑋1 and/or 𝑋2
Variability in Y
Classical Inference and Design Efficiency
26
Design orthogonality
For each pair of columns of the design
matrix, the orthogonality matrix depicts
the magnitude of the cosine of the
angle between them, with the range 0 to
1 mapped from white to black.
 If both vectors have zero mean then
the cosine of the angle between the
vectors is the same as the correlation
between the two variates.
Classical Inference and Design Efficiency
27
Correlated regressors: summary
●
We implicitly test for an additional effect only. When testing for the
first regressor, we are effectively removing the part of the signal that
can be accounted for by the second regressor:
 implicit orthogonalisation.
x^2
x2
x2
x1
x1
x^
x2
x^2 = x2 – x1.x2 x1
x1
1
●
●
Orthogonalisation = decorrelation. Parameters and test on the non
modified regressor change.
Rarely solves the problem as it requires assumptions about which
regressor to uniquely attribute the common variance.
 change regressors (i.e. design) instead, e.g. factorial designs.
 use F-tests to assess overall significance.
Original regressors may not matter: it’s the contrast you are testing
which should be as decorrelated as possible from the rest of the
design matrix
Classical Inference and Design Efficiency
28
Design efficiency
 The aim is to minimize the standard error of a t-contrast
T 
(i.e. the denominator of a t-statistic).
T
c bˆ
T
var( c bˆ )
T
2 T
T
1
var( c bˆ )  sˆ c ( X X ) c
 This is equivalent to maximizing the efficiency e:
1
e (sˆ , c , X )  (sˆ c ( X X ) c )
2
2
T
Noise variance
T
1
Design variance
 If we assume that the noise variance is independent of the specific
design:
1
e(c, X )  (c ( X X ) c )
T
T
1
 This is a relative measure: all we can really say is that one design is
more efficient than another (for a given contrast).
Classical Inference and Design Efficiency
29
Design efficiency
A
B
𝑋𝑇𝑋 =
1
−0.9
−0.9
1
𝑐 = [1 0]𝑇 :
𝑒 𝑐, 𝑋 = 18.1
𝑇
𝑐 = [0.5 0.5] : 𝑒 𝑐, 𝑋 = 19.0
𝑐 = [1 − 1]𝑇 : 𝑒 𝑐, 𝑋 = 95.2
A+B
A-B
 High correlation between regressors leads to
low sensitivity to each regressor alone.
 We can still estimate efficiently the difference
between them.
Classical Inference and Design Efficiency
30
Bibliography:
Statistical Parametric Mapping: The Analysis of
Functional Brain Images. Elsevier, 2007.
 Plane Answers to Complex Questions: The Theory of Linear Models. R.
Christensen, Springer, 1996.
 Statistical parametric maps in functional imaging: a general linear approach.
K.J. Friston et al, Human Brain Mapping, 1995.
 Ambiguous results in functional neuroimaging data analysis due to covariate
correlation. A. Andrade et al., NeuroImage, 1999.
 Estimating efficiency a priori: a comparison of blocked and randomized
designs. A. Mechelli et al., NeuroImage, 2003.
Classical Inference and Design Efficiency
31