How to Evaluate the Effects of
Potential Bias in Meta-analysis in
R
Load, Prep, and Check
library(ggplot2)
library(metafor)
#load the data
marine <- read.csv("marine_meta_short.csv",
na.strings=c("NA", ".", ""))
#check variable types
summary(marine)
Calculating Effect Sizes by Hand
#Log Ratio
marine$LR <- log(marine$Y_Poly) –
log(marine$Y_Avg_Mono)
marine$VLR <- with(marine, {
SD_Poly^2 / (N_Poly * Y_Poly^2) +
SD_Avg_Mono^2 / (N_Avg_Mono * Y_Avg_Mono^2)
})
Fit a Model (we’ll talk about this
soon)
mod <- rma(LR, VLR, data=marine)
Warning message:
In rma(LR, VLR, data = marine) :
Studies with NAs omitted from model
fitting.
What did we find?
Random-Effects Model (k = 168; tau^2 estimator: REML)
…
Model Results:
estimate
0.1324
se
0.0429
--Signif. codes:
zval
3.0851
pval
0.0020
ci.lb
0.0483
ci.ub
0.2165
**
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
funnel(mod)
Many funnel types
funnel(mod, main="Standard Error")
funnel(mod, yaxis="vi", main="Sampling
Variance")
funnel(mod, yaxis="seinv", main="Inverse
Standard Error")
funnel(mod, yaxis="vinv", main="Inverse
Sampling Variance")
Many funnel types
trimfill(mod, side="right")
Model Results:
estimate
0.2957
se
0.0493
--Signif. codes:
zval
5.9994
pval
<.0001
ci.lb
0.1991
ci.ub
0.3923
***
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
What is Trim and Fill Doing?
par(mfrow=c(1,2))
funnel(mod)
funnel(trimfill(mod, side="right"))
par(mfrow=c(1,1))
What is Trim and Fill Doing?
Fail-Safe: fsn(LR,
VLR, data=marine)
Fail-safe N Calculation Using the
Rosenthal Approach
Observed Significance Level: <.0001
Target Significance Level:
0.05
Fail-safe N: 12681
Other Types of Fail-Safe Numbers
> fsn(LR, VLR, data=marine, type="Rosenberg")
#based on weighted analysis
Fail-safe N Calculation Using the Rosenberg
Approach
Average Effect Size:
0.0384
Observed Significance Level: <.0001
Target Significance Level:
0.05
Fail-safe N: 3733
Other Types of Fail-Safe Numbers
> fsn(LR, VLR, data=marine, type="Orwin")
#based on unweighted analysis and target effect
size
Fail-safe N Calculation Using the Orwin
Approach
Average Effect Size: 0.1091
Target Effect Size: 0.0546
Fail-safe N: 168
Influence: inf(mod)