Simplified power and sample size
calculations using prevalence &
magnitude of active peaks.
a
a
a
b
Joke Durnez , Beatrijs Moerkerke , Ruth Seurinck and Thomas Nichols
a
b
Department of Data Analysis, Ghent University, Belgium
Department of Statistics & Warwick Manufacturing Group, University of Warwick, United Kingdom
Simulations
100 full-brain datasets with smooth Gaussian noise (3 voxels)
superimposed with activation (3 % BOLD change) in 4 foci (3% of
total brain volume)
Prevalence of activation: For small sample sizes: very conservative
estimates (close to 0).
Boxplot of bias of
volume of activation3
●
1.0
Estimation of π1
●
0.2
Sample size
10
15
16
17
18
19
20
21
22
●23
24
25
0.8
●
●
●
0.4
●
●●
●
●● ● ● ●
●
●
●
●
●●
● ●●●● ● ●
●
●
●
●
●
●●●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●●●
●●
●
●
●
●●
●
●
●
●
● ●●
●● ●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●●●●
●
●
●
●●
●
●
●
●●●●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●● ●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●●
●●●● ● ●
●●●
●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●●
●●
●●●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●●
●●
●
● ●●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●●
●●●
●
●●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●●●
● ● ●●
●
●
●
●●
●
●
●●●
●●●● ● ●
●
●
●●
●
●
●
●
●●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●●●●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●●●
●
● ●●●
●
●
●
●
●
●
●●
●
●
●●●●●●
●●●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●●
●
●●
●
●
● ●
●
●
●
●●
●●
●
●
●
● ●● ●
●
●
●
●
●●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●●●
●
●●
●
● ●
●
●
● ●
●
●●
●
●
●●●●
●
●●
●●
●
●●●
●●
●
● ●●
●●●
●
●
●
●
● ●●
●●●
●
●●●●
●●
●●
●
●
●
●
● ● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
● ●●●
●
●
●
●● ●●●●
●●
●
● ● ●●● ● ●● ●
● ●
●
● ● ● ●
●● ● ●
● ●●
0.2
●
●
●
●
●
●
● ●
●
● ●●
●
●● ● ●
● ●
●●
●
●
●
●
● ● ●●
●●● ●●
●
●
●
● ●●
●●
●●● ● ● ●
●
● ●
●
●●
●
● ●●
●●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
● ● ●
●
●
●
● ●
●
●
●
●
●●
●
0.0
●
●
−0.2
●
●
●
●
−0.4
●
Bias (π^1 − π1)
0.6
●
0.0
There is increasing concern about statistical power in neuroscience
research:
Low power decreases the chance of detecting a true effect
Low power reduces the chance that a statistically significant result
indicates a true effect (Ioannidis, 2005).
⇒ A power analysis is a critical component of any study
Power analyses for fMRI are difficult: need to specify magnitude, spatial
extent and location of a hypothesized effect.
We present a simple way to characterize the spatial signal in a fMRI study,
and a direct way to estimate power based on an existing pilot study.
We must estimate (or have prior knowledge of):
the volume of the brain that is activated
the average effect size in activated brain regions
This allows the calculation of power for given sample size, brain volume
and smoothness.
Estimated π^1
Introduction
●
●
0.0
0.2
0.4
0.6
0.8
10 15 16 17 18 19 20 21 22 23 24 25
1.0
●
True π1
sample size
Effect size estimation: Rather conservative estimates.
Boxplot of bias of
estimated effect size
Effect size estimation
Sample
size
u−µ1
σ1
0.6
13
15
20
25
Sample size
Department of Data analysis - H. Dunantlaan 1 - 9000 Gent
30
35
0.0
^
Bias (δ − δ)
−0.2
−0.4
●
10 15 16 17 18 19 20 21 22 23 24 25
1.2
●
sample size
2.0
1.5
1.0
δ
2.97
3.54
3.95
4.28
µ σ=δ n
0.94
0.92
0.88
0.86
6
7
5
●
1.0
0.6
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
Sample size
0.5
●
●
●
15
16
17
18
19
20●
21
22
23
24
25
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0.0
●
●
15 16 17 18 19 20 21 22 23 24 25
15 16 17 18 19 20 21 22 23 24 25
Sample size
Sample size
●
References and acknowledgements
Durnez, Moerkerke, Nichols (2014). NeuroImage, 84.
Ioannidis (2005). PLOS Medicine, 2:6.
Seurinck, de Lange, Achten, Vingerhoets (2011). Journal of Cognitive Neuroscience, 23:6.
Worsley (2006). Random Field Theory in Friston, Ashburner, Kiebel, Nichols and Penny, Statistical Parametric
Mapping. Elsevier, London
This work was carried out using the STEVIN Supercomputer Infrastructure at Ghent University,
funded by Ghent University, the Flemish Supercomputer Center (VSC), the Hercules
Foundation and the Flemish Government â department EWI.
0.6
0.4
Power to detect effect
0.8
1.0
10
●
0.0
π1 = 0.7
π1 = 0.6
π1 = 0.5
π1 = 0.4
π1 = 0.32
π1 = 0.2
π1 = 0.1
current study
0.2
1.0
0.8
●
●
●
Power with varying sample size and effect size
0.6
0.4
0.2
0.0
Power to detect effect
●
●
●
0.5
●
●
peak heights
Power with varying sample size and π1
●
●
●
●
●
●
Expected peak height under activation µ1 = δ = 4.04
√
Effect size µ/σ = δ/ n = 1.13
Step 3: estimate power for given sample size, estimated peak height under
activation and prevalence of activation (π1)
●
Bias on power estimation
●
●
0.8
6
●
●
4
True
power
●
0.2
peak p−values
5
●
Power calculations: Good estimates for realistic values of power with
FWER control at the 5% significance level.
0.4
4
●
peak heights
Estimated power
3
1.1
3
0.4
0.2
2
●
●
●
0.0
2
●
1.0
●
●
●
●
Estimated power from
pilot study (n=15)
0.0
0.8
●
●
●
10
15
20
25
0.6
Density
0.5
0.6
●
●
●
●
1.0
1.0
2.0
1.5
1.0
Density
0.0
0.4
●
1.0
●
0.2
●
●
●
●
●
Sample size
●
0.0
●
●
●
●
Average estimated distribution over all simulations
for different sample sizes
Estimated alternative distribution
Estimated null distribution
Estimated total distribution
0.8
2.5
Estimated null distribution
Estimated alternative distribution
●
●
●
●
●
●
●
0.0
π1 = 0.31 / δ = 4.04 / Power = 0.22
3.0
π1 = 0.31
0.9
●
Alternative distribution: Constant effect size (µ/σ) over different
sample sizes to be used in power analyses.
Density
Data from Seurinck et al. (2011). The original study was analyzed
voxelwise with FWER-control at the 5% significance level. We reanalyzed
the data peakwise with FDR-control at the 5% significance level.
Sample size: 13
Step 1: compute prevalence of activation
Step 2: estimate truncated distributions
Distribution of peak heights
0.8
●
●
●
True effect size in activated peaks (δ)
µ1 and σ1 can be estimated using maximum likelihood, where µ1 is the
expected peak height in activated regions.
Power can be estimated for a given thresholdt as P (T > t|Ha) with T the
T -statistic of the peak.
Distribution of 48 peak p−values
0.7
Example
0.2
1.1
0.6
0.7
0.8
0.9
1.0
●
−0.5
1−Φ
10
15
16
17
18
19
20
21
22
23
●
24
25
−1.0
f (x|π0, µ1, σ1, u) = (1 − π1)u exp(−u(x − u)) + π1
x−µ1
σ1
●●
●
●
●
●
● ●
● ●
●
●
●●
● ●
●
●
●
●●
●
●
●● ●
●
●
● ● ●●
● ●●
●●
●
●
●●
●
● ● ● ●●
● ●●
●
●●●
●
●
● ● ●
● ●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●● ●
●●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●● ●
●
●●
●
●●
●
●●
●
● ● ●●
●
● ●●
●●
●
●●
●●●● ●●
●
●
●
●● ●
●
●
●
●
● ●
●●●
●
●
●
●●
●
●
● ● ●●
●●
●
●
●
●
●
●
●
●
● ●
●● ●
●
●●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
Estimated power − True power
1
σ1 ϕ
^
Estimated effect size in activated peaks(δ)
We start from peak p-values in a group level analysis. We estimate π1,
proportion of peak p-values thare are non-null, as described in Durnez,
Moerkerke & Nichols (2014).
We assume that the null distribution of peak values is an exponential
distribution (Worsley, 2007).
We assume that the alternative distribution of peak values is a truncated
normal distribution (truncated at excursion threshold u).
Therefore, the distribution of peak values can be written as a mixture:
0.4
1.2
Methods
●
●
40
10
13
15
20
25
30
δ = 0.7
δ = 0.8
δ = 0.9
δ=1
δ = 1.12
δ = 1.2
δ = 1.3
current study
35
40
Sample size
Joke.Durnez@UGent.be
da.ugent.be