Understanding Study Design
& Statistics
Dr Malachy O. Columb FRCA, FFICM
University Hospital of South Manchester
NWRAG Workshop, Bolton, May 2015
COIs: Interesting Confllicts!




Editorial Board Roles:
European Journal of Anaesthesiology
British Journal of Anaesthesia
International Journal of Obstetric Anesthesia
Manuscript Types







(7)
(6)
(5)
(4)
(3)
(2)
(1)
Meta-analysis & systematic reviews
Original research – PDBRCT
Original research – other RCT
Original research – observational
Original research – retrospective
Narrative reviews – including editorials
Case reports, abstracts & letters
Manuscript Types







(7)
(6)
(5)
(4)
(3)
(2)
(1)
Original research – PDBRCT
Original research – other RCT
Original research – observational
Original research – retrospective
Meta-analysis & systematic reviews
Narrative reviews – including editorials
Case reports, abstracts & letters
Statistics: Definition
…the discipline concerned with the treatment of
numerical data derived from groups of individuals…
Data
…are always plural…
‘Datum’ is the singular…
Types of Data


Numerical – continuous & discrete
Categorical – binary, nominal, ordinal
Hypotheses







Null hypothesis (HO)
Alternative hypothesis (HA)
P value and 95% confidence interval
Two-sided by convention
One-sided are rarely appropriate
Equivalence, Non-inferiority, Superiority (Margins)
Inequality is the usual HA
Potencies and probabilities: One-sided P values suggest a one-sided story!
Columb MO, Polley LS. Anesthesia & Analgesia 2001;92:278-9
Controlling Bias - Design





Prospective > Retrospective
Double Blind > Single Blind > Unblinded
Randomised Controlled Trial > Unrandomised
PDBRCT > Propensity Score Matching!
PROBE (Single Blind)
Sample Size
Power analysis and sample size calculations.
Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.
Sample size



Minimum difference that is (clinically) important
Defines primary outcome!
Multiple comparisons!
Power analysis and sample size calculations.
Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.
Estimate of SD




Published research
Pilot data
Empirical approach
1/5th – ‘one fifth’ of the range
Power analysis and sample size calculations.
Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.
One-Fifth Range




4 SD = 95.4% of values
6 SD = 99.7% of values
Take 1/5th range to approximate SD
20% of the range
Power analysis and sample size calculations.
Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.
Standardised Difference

Difference / SD
Power analysis and sample size calculations.
Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.
Standardised Difference = 1.0
Power and sample size estimation
Armitage & Berry pp 195-202
Sample size estimate for comparison of 2 equal groups
1 minimum difference to be significant
1 SD
0.05 type 1 error rate, P value
0.8 power
1.959964 Z 2 alpha 2 sided
0.841621 Z 2 beta 1 sided
18 = minimum n for each group
20 = nonparametric
Nonparametric Adjustment

Add 16% more subjects per group!
Power analysis and sample size calculations.
Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.
Sample Size - Proportions
Power analysis and sample size calculations.
Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.
Standardised Difference = 1.0
Sample size estimate for comparison of 2 proportions
0.75 proportion 1
0.25 proportion 2
0.05 type 1 error rate, P value
0.8 power
1.959964 Z 2 alpha 2 sided
0.841621 Z 2 beta 1 sided
0.5 pooled proportion
14 = minimum n per group for uncorrected Chi square test
18 = Fleiss continuity correction, minimum n per group
Descriptive Statistics




Sample
Mean (SD) – 68% of data
Median [interquartiles, range]
Count/frequency
Inferential Statistics - Precision




Population estimates; precision
Differences in means, medians, proportions
Mean or mean difference
Sampling theory!
Population
(variable X)
x
µ
Randomization
Population
of means
(variable )
Sample 1
x1,x2.....xn
1
Sample 2
x1,x2.....xn
Sample 3
x1,x2.....xn
2
3
Distribution of
sample means
(variable )
µ
.........
Sample j
x1,x2.....xn
j
.....
100 random
samples of size
4
100 random
samples of size
20
100 random
samples of size
50
100 random
samples of size
250
Inferential Statistics - Precision






SD of sampled means is the SE of mean
SE mean = SD / n
SEM = 68%CI, (precision)
SEM x 1.96 = 95%CI (precision)
Test statistic = difference / SE difference
P value
Significance





P value – ‘probability of the observed difference
or greater assuming the null hypothesis’
Type I or alpha error <0.05; false +ve
Type II or beta error <0.20; false -ve
Multiple comparisons - Bonferroni correction
Corrections to 95% CI of difference
Group Tests
Groups
Parametric
1
2
One
sample t
Unpaired t
>2
ANOVA
Parametric
repeated
One
sample t
Paired t
Repeated
measures
ANOVA
Nonparametric
Wilcoxon
signed rank
MannWhitney U
KruskalWallis
Nonparametric
repeated
Wilcoxon
signed rank
Wilcoxon
matched pairs
Friedman’s
Proportions
Binomial
Chi square or
Fisher’s exact
Chi square or
expanded
Fisher
Statistical Analyses





Correlation – Pearson, Spearman, intraclass
Regression – linear, logistic, probit, survival
Diagnostics – sensitivity, specificity, ROC curves
Reference intervals – normal range
Agreement – kappa, Bland-Altman plots
Transformations
Table. Some useful transformations
Types of data
Transformations
Positive skew, increasing variance
Logarithmic = ln x, log10 x
Negative skew, decreasing variance
Square power = x 2
Count; Poisson distribution
Square root =x
Time; survival data
Inverse = 1/x
Proportion; p, Binomial distribution
Logit = ln (p/1-p) Arcsin = Arcsinp
Probit = Cumulative Normal Deviate
Time-to-Event: Log Transformation
Analyses for RCT

Per-Protocol (PP)




Treatment-Received (TR)




Received allocated treatment and completed protocol
Largest estimate of effect size
Selection bias for post-treatment withdrawals
Received allocated treatment
May not have completed the protocol
Selection bias for pre-treatment withdrawals
Intention-to-Treat (ITT)




All randomised subjects – NO WITHDRAWALS
May or may not have received the intervention
Underestimates true effect size of treatment
Most robust analysis
MOCPASS – columbmo@msn.com
Power and sample size estimation
Armitage & Berry pp 195-202
Sample size estimate for comparison of 2 equal groups
20 minimum difference to be significant
20 SD
0.05 type 1 error rate, P value
0.8 power
14 = minimum n per group for uncorrected Chi square test
18 = Fleiss continuity correction, minimum n per group
20 = nonparametric
Power estimation for comparison of 2 equal groups
20 minimum difference to be significant
20 SD
0.05 type 1 error rate, P value
20 sample size n in each group
1.959964 Z 2 alpha 2 sided
1.202314 Z 2 beta 1 sided
0.25 proportion 1
0.75 proportion 2
0.05 type 1 error rate, P value
0.8 power
1.959964 Z 2 alpha 2 sided
0.841621 Z 2 beta 1 sided
0.5 pooled proportion
1.959964 Z 2 alpha 2 sided
0.841621 Z 2 beta 1 sided
18 = minimum n for each group
Sample size estimate for comparison of 2 proportions
Power estimation for comparison of 2 equal groups
0.25 proportion 1
0.75 proportion 2
0.05 type 1 error rate, P value
20 n per group
1.959964 Z 2 alpha 2 sided
1.388312 Z 2 beta 1 sided
1.002815 Z 2 beta, corrected
0.5 pooled proportion
0.917479 = power for uncorrected Chi square test
0.89 = power of study
0.84 = power with Fleiss continuity correction