What is Random Effects
Meta-Analysis?
Pantelis Samartsidis, Thomas Nichols
The University of Warwick
Simulation of valid studies
For group fMRI analyses, random effects inferences are essential for drawing conclusions that
generalise to the population of subjects studied.
In (non-imaging) meta-analysis, random effects
models are also regarded as best practice, as
they account for random differences among
each study’s population or methods. However,
outside the context of linear regression, the
meaning of what exactly is a ‘fixed effects’ or
‘random effects’ is ambiguous (Gelman, 2006).
●
I 2 foci with probability 35%
●
I 3 foci with probability 25%
• Average 14 foci
Figure 1: Simulated valid study.
• Eight true population centers
0.2
0.4
0.6
0.8
Proportion of valid studies
• Foci normally distributed around population centers
• Each simulated meta-analysis includes:
0
20
40
60
80
100
0.6
0.8
8
6
4
2
0
(3) Mean number of centers detected
8
6
4
2
0.4
1.0
0
120
Total number of valid studies
0.2
0.4
0.6
0.8
40
60
80
100
120
1.0
1.0
0.8
1.0
N=20
N=40
N=60
N=80
N=100
N=120
0.0
20
Total number of valid studies
0.8
1.0
0.8
0.6
0.4
0.2
1.0
0.2
Proportion of valid studies
N=20
N=40
N=60
N=80
N=100
N=120
0.0
(2) Probability all 8 centers detected
1.0
0.8
0.6
0.4
0.2
0.0
(2) Probability all 8 centers detected
N=20
N=40
N=60
N=80
N=100
N=120
0.0
0.0
Total number of valid studies
1. Sensitivity properties
Simulation study
0
120
0.6
100
N=20
N=40
N=60
N=80
N=100
N=120
0.4
80
0.2
60
N=20
N=40
N=60
N=80
N=100
N=120
0.0
40
0.6
Proportion of valid studies
20
(4) Probability at least one center detected
0
(3) Mean number of centers detected
1.0
0.8
0.6
0.4
1.0
0.4
0.8
0.2
0.6
N=20
N=40
N=60
N=80
N=100
N=120
0.0
0.4
(4) Probability at least one center detected
0.2
N=20
N=40
N=60
N=80
N=100
N=120
0.2
0.2
0.4
N=20
N=40
N=60
N=80
N=100
N=120
0.0
0.6
0.8
1.0
Results
Study various aspects of the ALE algorithm:
3. Random effects nature
●
●
I 1 foci with probability 40%
0.0
2. Robustness to noise
●
• Conditional on center being present, the
study has:
(1) Mean voxel−wise true positive rate
Objectives
Pop. centers
Foci
●
(1) Mean voxel−wise true positive rate
Existing neuroimaging coordinate based metaanalysis (CBMA) methods, Activation Likelihood Estimation (ALE) in particular, report
‘random effects’ inferences (Eickhoff et al., 2009,
Eickhoff et al., 2012), on the basis of a resampling inference method that distinguishes coordinates within a study from those between studies. We perform a set of simulations to study the
sensitivity properties of ALE in the presence of
varying number of all-noise studies.
●
• Each center present in a study with probability 0.8
0.0
Introduction
0
Proportion of valid studies
20
40
60
80
100
120
Total number of valid studies
Figure 2: Simulation results.
• All measures of power converge to the maximal values with increasing number of studies
• For given N , power increases with p (columns 1 & 3)
• For given p, power increases with N (columns 1 & 3)
N studies
• For fixed total number of valid studies, pN , lines coincide (columns 2 & 4)
p
1−p
Conclusions
• Consistency: sensitivity increases with sample size
Valid
Noisy
• Robustness to noise: more noisy studies do not degrade sensitivity
– Potential drawback: a small proportion of studies can drive inference
Parameter Specifications:
• N = 20, 40, 60, 80, 100 and 120
• p = 0, 0.05, 0.1, ..., 0.95, 1
• B = 1000 datasets for each N, p
Power measures considered:
– Fixed effects inference characteristic
Future work
• Randomness in both location & total number of foci can be expressed via spatial Poisson model.
At each voxel v (or ROI) total number of foci K(v) ∼ Pois (µ(v))
• Study random effects αi can be incorporated:
EK(v) = µ(v) + αi , αi ∼ N (0, τ 2 )
1. Probability at least one center detected
2. Probability all 8 centers detected
3. Mean number of centers detected
4. Mean voxel-wise true positive rate
References
• S. Eickhoff et al. (2009). Human Brain Mapping, 30(9), 2907-26.
• S. Eickhoff et al. (2012). NeuroImage, 59(3), 2349-61.
• A. Gelman (2005). The Annals of Statistics, 33(1), 1-53.
Noisy studies:
• Foci uniformly drawn from brain mask
Acknowledgements
Authors are grateful to Timothy Johnson for the valuable comments.