Basics of fMRI Inference
Douglas N. Greve
Overview
• Inference
• False Positives and False Negatives
• Problem of Multiple Comparisons
• Bonferroni Correction
• Cluster Correction (voxel-wise threshold)
• False Discovery Rate
• Selection Bias
Statistical Inference
• Can your conclusions be extended to data you have
not seen?
– Subjects, Time Points, Groups, Scanners
• Or are your results the product of a chance
occurrence that is unlikely to be repeated?
• Generalizability, Repeatability,
Reproducibility, Predictability
• Uncertainty
• Beyond good Experimental Design
Group Population
(All members)
Hundreds?
Thousands?
Billions?
Sample
18 Subjects
3
Truth Table
Reality
Conclusion
Effect Is Not Effect Is
Present (Neg) Present (Pos)
True Negative False Positive
Effect Is Not
Present (Neg)
Effect Is
False Negative True Positive
Present (Pos)
4
Error Rate
Conclusion
Reality
Effect Is Not
Present (Neg)
Effect Is Not
Present (Neg)
Effect Is
Present (Pos)
Effect Is
Present (Pos)
True Negative False Positive
TNR=1-a
FPR = a
False Negative True Positive
FNR = b
TPR = 1-b (Power)
False Positive Rate (FPR) – probability that you declare an effect to be
present when there is no effect
False Negative Rate (FNR) - probability that you declare no effect to be
present when there is an effect
5
How Do You Draw Conclusions?
Protocol: reduce all your data to one number (the “test
statistic” T).
• If T is greater than some threshold (q) then conclude that
an effect is present (ie, a positive)
• Otherwise conclude that an effect is not present (ie, a
negative).
Every protocol has some FPR and some FNR, though it is
not always easy to figure out!
6
Noise Causes Uncertainty
Voxel 1
Voxel 2
7
GLM Inference

bOFF
2
OFF
bON
2
 ON
T=8
T=
T=1
b ON  b OFF
2
2
( N ON  1) ON
 ( N OFF  1) OFF
( N ON  N OFF  2) 2
2
b ON ,  ON
, N ON  Mean,Var, N in ON
2
b OFF ,  OFF
, N OFF  Mean,Var, N in OFF
8
Example Protocol
• Collect data
• Motion Correct
• Smooth by 5mm FWHM
• Extract Voxel 1 (throw away rest of data)
• Compute Mean and StdDev of ON time points
• Compute Mean and StdDev of OFF time points
• Compute test statistic T
• If T > 3.41, Conclude that the voxel is active
Test Statistic (T) is the t-ratio
Threshold (q) is 3.41
What is the FPR (a) and FNR (b) for this protocol?
9
Example Protocol: False Positive Rate
b b
• “NULL” Distribution Student’s t-Distribution T =
( N  1)  ( N  1)
• p-value is area under curve to the right of T
( N  N  2)
DOF = N  N  2
• For T = 3.4, FPR = p =.01
• For T=8, FPR = p = 10-11
• For T=1, FPR = p = 0.1
FPR=area
Student’s t Distribution
under curve
• Assumptions:
to the right
• Gaussian noise
of line
(p-value)
• Independent noise
• Homoskedastic (equal variances)
• Violation of assumptions change FPR
ON
ON
2
ON
OFF
2
OFF
OFF
2
ON
ON
OFF
OFF
10
Example Protocol: False Negative Rate
• Need to know what the effect size is
• Previous data
• Guess
• Power Analysis
• Grants require a power analysis!
11
Trade Off of Error Rates
FPR=.10
FPR=.01
• Inverse relationship between error rates
• As False Positives (a) are reduced, the False Negatives
(b) increase
• Increase sample size decreases b, does not affect a
• Which Error is more important? Depends ..
• Science? FPR=.05ish, TNR<0.2
• Pre-operative surgery?
FPR=10-7
12
What conclusions to draw from this?
• Brain is activated?
• Visual Cortex?
• Auditory Cortex?
• False Positive Rate?
Need a protocol!
13
Possible Protocol
• First Level Analysis
• Compute t-ratio for each voxel
• Compute p-value for each voxel
• If any brain voxel has p < .01, declare a positive
• Same as
• Test Statistic: T = max(Ti)
• Threshold: q=3.4
What is the False Positive Rate for this protocol?
14
What does p<.01 mean?
Rand(0,1)
100x100
10,000 vox
p < 0.1
1000 vox
p < 0.01
100 vox
p < 0.001
10 vox
• p<.01 means one expects 1% of voxels will be active purely
by chance
• Protocol gives a False Positive any time even a single voxel
has p<.01
• What is the probability that at least one voxel has p<.01?
15
The “Problem of Multiple Comparisons”
a FWE = 1  (1  aVox )
N
N = 10,000
aVox
• aVox = voxel-wise threshold (p< aVox)
• aFWE = Protocol False Positive Rate
(FWE = Family-wise Error)
• N = Number of voxels (“Search Space”)
aVox =.10
aVox =.01
aVox =10-7
aFWE
0.00001 0.095
0.0001
0.632
0.001
1.000
0.01
1.000
16
Bonferroni Correction
aVox = 1  1  a FWE 
N
a FWE
N
= f (a FWE , N )
Compute Voxel-wise threshold needed to achieve
a desired Family-wise FPR.
To achieve aFWE = 0.01 with N = 10,000 voxels
Need aVox = 0.000001 (10-6)
17
Search Space
• Set of voxels over which positives are searched
• Severity of correction increases with size of search
space (regardless of method)
• Reduce Search Space
• Reduce the area to a ROI (eg, superior temp gyrus)
• Increase voxel size (cover same volume with fewer voxels)
• Spatial Smoothing
18
Spatial Smoothing
• Spatially convolve image with Gaussian kernel.
• Kernel sums to 1
• Full-Width/Half-max: FWHM = /sqrt(log(256))
 = standard deviation of the Gaussian
0 FWHM
5 FWHM
10 FWHM
Full-Width/Half-max
Full Max
2mm FWHM
Half Max
5mm FWHM
Smoothing causes irreversible loss of
information (resolution)
10mm FWHM
Spatial Smoothing
Smoothing causes irreversible loss of information (resolution),
similar to increasing voxel size.
0mm
5mm
10mm
Smoothing
1mm
Increased
Voxel Size
4mm
8mm
Resel
• Pixel = picture element
• Voxel = volume element
• Resel = resolution element (depends on smoothing level)
Resel = (FWHM)3 for volumes
Resel = (FWHM)2 for surfaces
If FWHM>Voxel Size, fewer Resels than Voxels.
Correct based on the number of Resels instead of number of
voxels (math is more complicated, need Random Field Theory)
aVox = f (a FWE , N , FWHM )
Bonferroni
aVox = 1  N 1  a FWE 
aVox
N
Clusters
aVox =.10
aVox =.01
aVox =10-7
• True signal tends to be clustered
• False Positives tend to be randomly distributed in space
• Cluster – set of spatially contiguous voxels that are above a
given threshold.
Cluster-wise Correction
• Cluster – set of spatially contiguous voxels that are
above a given threshold.
• Protocol
• Perform 1st level analysis.
• Threshold volume at aVox
• Find clusters.
• If Cluster Size > Threshold (q), Declare a Positive
• Test Statistic: Cluster Size
• What is the FPR (aFWE) for this protocol?
Random Field Theory
aFWE = f(aVox,N,FWHM,ClusterSize)
p=.05
p=.05
Smoothing increases size of random clusters
FWHM 0
Z
Z>2.3
p<.01
FWHM 2
FWHM 4
FWHM 6
Cluster Images
Sig Map
pVox < .001
Cluster Map
pCluster < .05
Some small clusters do not “survive”
Cluster Table
MNI305
Size Cluster Atlas
Cluster X Y Z (mm3) p-value Location
1
40 -67 -11 41368
~0 Right Lateral
Occipital
2
3
4
-40 -85 -13 51184
-6 17
-50
7
~0
Left Lateral
Occipital
45
2784 .00026
Left Superior
Frontal
23
3768 .00002
Left
Precentral
R
L
Radiological
Orientation
ROI Atlas
Cluster Data Extraction
• Spatial average over cluster of each subject’s contrast
• Can correlate with other measures (age, test score, etc)
• Be careful of “Selection Bias” (“Voodoo Correlations”)
Cluster Correction Summary
• Cluster – set of supra-threshold voxels (size)
• Critical Size Threshold given by Random Field Theory
• Search Space
• Voxel-wise threshold (arbitrary)
• FWHM (smoothing level)
• Assumptions on each
• Loose small clusters (False Negatives)
False Discovery Rate (FDR)
p < 0.1
1000 vox
p < 0.01
100 vox
p < 0.001
10 vox
• Given the voxel-wise threshold, know expected
number of False Positives
• If there are more Positives than this, then some of
them must be True Positives
False Discovery Rate (FDR)
Number of False P ositives
Number of False P ositives Number of T rue P ositives
Number of False P ositives
=
T otalNumber P ositives
FDR =
• Number of False Positives = N*aVox
• Total Number of Positives = Count from image
• aVox = f(FDR,N,Data)
False Discovery Rate (FDR)
• FDR = .05 means that 5% of Positives are False Positives
• Which 5%, no one knows
• How to interpret?
FDR = .05
aVox = .0070
FDR = .01
aVox = .0070
False Discovery Rate (FDR)
• FDR = .05 means that 5% of Positives are False Positives
• Which 5%, no one knows
• How to interpret?
FDR = .05
aVox = .0070
FDR = .01
aVox = .0070
Would you change your
opinion of this blob if 50
of the voxels were False
Positives?
False Discovery Rate (FDR)
• FDR = .05 means that 5% of Positives are False Positives
• Which 5%, no one knows
• How to interpret?
FDR = .05
aVox = .0070
FDR = .01
aVox = .0070
Would you change your
opinion of this blob if 50
of the voxels were False
Positives?
False Discovery Rate Summary
• False Discoveries
• FDR does not control FPR (False Positive Rate)
• Careful when interpreting
• Voxel-wise threshold is Data Dependent
Summary
• Can your conclusions be extended to data you have not seen?
• Truth Table: False Positives (a) and False Negatives (b)
• Protocol – describes how you will draw conclusions
• Problem of Multiple Comparisons (Family-wise Error)
• Search Space, Search Space reduction
• Larger voxels (less resolution)
• Smoothing (Resels)
• Bonferroni Correction
• Cluster Correction (voxel-wise threshold)
• False Discovery Rate
• Selection Bias
37