To Prevent Selection Bias
Minimal Balance is Sufficient
Wenle Zhao, PhD
Medical University of South Carolina, Charleston, SC, 29425, USA
Society for Clinical Trials 36th Annual Meeting
Arlington, VA, USA - May 17-20, 2015
Contents
1. Where does Selection Bias Come From?
2. How to prevent selection bias?
3. How to avoid random serious imbalance?
The Worst Thing in the World of Clinical Trials
Funding?
Recruitment?
A Completed Trial with
Suspicious SELECTION BIAS.
Defense Measurements against Selection Bias
Real-time
Treatment
Subject
Allocation
Randomization
Random
Allocation
Subject
Enrollment
Allocation
Concealment
Outcome
Assessment
Treatment
Masking
The only reliable protection left against selection bias.
Allocation Randomness
Ti  F  W, Ti 1 , Xi , R 
Target allocation ratio
Random variable ~U(0,1)
To balance baseline covariate
Stratified Randomization
Minimization
Allocation Randomness
To balance treatment distribution
Permuted Block Randomization
Biased Coin, Urn Design
Complete
Randomization
Permuted Block
Randomization
Minimization
Predictability Defeats Concealment & Masking
Pr(Ti  A)  1
Deterministic Assignment
~ 87%
20
%
B=2
25%
0
33%
50%
Proportion of DA
100%
B=4
B=6
B=8
Permuted Block Randomization
Minimization
Evidence of Selection Bias in Randomized Trials
E 1. Heparin for myocardial infarction
E 2. University Group Diabetes Program
P 3. Talc and mustine for pleural effusions
P 4. Tonsillectomy for recurrent throat infection in children
P 5. Oxytocin and amniotomy for induction of labor
P 6. Western Washington Intracoronary Streptokinese Trial
E 7. RSV immune globulin in infants and young children
E 8. A trial to assess episiotomy
P 9. Canadian National Breast Cancer Screening Study
Selection bias
P 10. Surgical Trial
evidence
P 11. Lifestyle Heart Trial
identified
E 12. Coronary Artery Surgery Trial
P 13. Etanercept for children with juvenile rheumatoid arthritis
E 14. Edinburgh Randomized Trial of Breast Cancer Screening
P 15. Captopril Prevention Project
P 16. Göteborg (Swedish) Mammography Trial
E 17. HIP Mammography Trial
E 18. Hypertension Detection and Follow-up Program
P 19. Randomized Trial to prevent vertical transmission of HIV-1
P 20. Effectiveness trial of diagnostic test
E 21. S African trial of high-dose chemotherapy for metastatic breast cancer
P 22. Randomized study of a culturally sensitive AIDS education program
P 23. Runaway Youth Study
P 24. Cluster randomized trial of palliative care
P 25. Randomized trial of methadone with or without heroin
P 26. Randomized NINDS trial of tissue plasminogen activator for acute ischemic stroke
P 27. Norwegian Timolol Trial
P 28. Laparoscopic versus open appendectomy
P 29. The Losartan Intervention for Endpoint reduction in Hypertension Study
P 30. The Heart Outcomes Prevention Evaluation Study
E
P
Suspicious
election bias
due to
p-value < 0.05
Protect Trials Against Selection Bias
Selection bias will result small p-values
Complete randomization may (5% chance) see a p-value < 0.05
The Logic
 Real-time complete randomization
 Eliminates selection bias due to allocation predictability
 Eliminates selection bias due to allocation concealment failure
 Totally eliminates selection bias
 Without selection bias, complete randomization may still have
 Imbalance in treatment distribution
 Power loss is trivial
 Imbalance is baseline covariate distribution
 Adjustment, not balancing, is the solution
 Serious baseline covariate imbalance with p-value < 0.05
 5% chance for any covariate
 60% chance for at least one in 10 covariates
 Suspicion of selection bias
 Trouble in trial result interpretation
Options We Have
Stratified Restricted Randomization

Permuted Block Randomization

Biased Coin Design - Efron

Urn Design - Wei

Big Stick Design – Soares & Wu

Maximal Procedure – Berger et al.

Block Urn Design – Zhao & Weng
Unnecessarily tighten control imbalances.
Disabled when number of strata getting large.
Minimization
Most assignments are deterministic.
Dynamic Hierarchy Balancing
Hierarchy order is hard to justify.
Minimal Sufficient Balance Procedure
Subject ready for
randomization
N
Any
T-test for
continuous var.
χ2 test for
categorical var.
Any serious
p-value
< 0.2?
imbalance?
Y
Current assignment
can effectively
reduce imbalances?
Y
N
Complete randomization
Next subject
Biased coin assignment
The proportion
depends on pvalue threshold
and biased coin
probability
Example : NINDS rt-PA Stroke Study
p-value
Site
Observed in the
Original Study
NIHSS
Age
OTT
Glucose
0.9987 0.1398 0.0289 0.8662 0.7804
Stroke
Subtype
Sex
0.0733
0.6265
Fibrinogen Weight
0.1808
0.0111
Serious imbalances found
in 2 of the 11 baseline covariates.
Systolic Diastolic
BP
BP
0.5968
0.2810
Example : NINDS rt-PA Stroke Study
NINDS rt-PA Stroke study data.
Simulations = 5000
Baseline covariates:
• Severity (NIHSS)
• Age
• Onset to treat
• Glucose
• Center
Example : NINDS rt-PA Stroke Study
Balance 11 baseline covariates
Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled*
NINDS rt-PA data, Imbalance control limit p-Value ≥ 0.3. ξ= 0.65, simulation = 1000/scenario.
p-value
NIHSS
Age
OTT
Glucose
Stroke
Subtype
Sex
0.259
0.237
0.262
0.244
0.262
0.214
0.245
0.239
0.252
0.246
0.276 0.295 0.279 0.288
0.280
0.292
0.277
0.288
0.278
0.281
0.287
Site
Low 2.5% boundary 0.226
Low 5% boundary
Low 10% boundary 0.309
Median
Observed in the
Original Study
Fibrinogen Weight
Systolic Diastolic
BP
BP
0.341
0.316
0.323
0.315
0.326
0.309
0.322
0.319
0.322
0.330
0.631
0.626
0.624
0.624
0.638
0.609
0.634
0.626
0.638
0.610
0.9987 0.1398 0.0289 0.8662 0.7804
0.0733
0.6265
0.1808
0.0111
0.5968
0.2810
0.605
Summary
 Complete randomization eliminates selection bias due to
allocation predictability.
 Real-time randomization eliminates selection bias due to
allocation concealment failures.
 Minimization method has the highest proportion of deterministic
assignments, and therefore is vulnerable to selection bias.
 Power loss due to treatment imbalance is trivial.
 Justification, not balancing, is the solution for covariate
confounding effects.
 Using Minimal Sufficient Balancing to prevent random serious
imbalances, while maintaining a high level of allocation
randomness.
Thank You!
Contact me at:
zhaow@musc.edu
Some of my works on Randomization
.
 Zhao W, Ciolino J, Palesch Y. Step-forward randomization in multicenter emergency treatment clinical trials. Acad
Emerg Med. 2010 Jun;17(6):659-65. doi: 10.1111/j.1553-2712.2010.00746.x. PMID: 20624149
 Zhao W, Weng Y, Wu Q, Palesch Y. Quantitative comparison of randomization designs in sequential clinical trials
based on treatment balance and allocation randomness. Pharm Stat. 2012 Jan-Feb;11(1):39-48. doi: 10.1002/pst.493.
PMID: 21544929
 Zhao W, Weng Y. Block urn design - a new randomization algorithm for sequential trials with two or more
treatments and balanced or unbalanced allocation. Contemp Clin Trials. 2011 Nov;32(6):953-61. doi:
10.1016/j.cct.2011.08.004. PMID: 21893215
 Zhao W, Hill MD, Palesch Y. Minimal sufficient balance--a new strategy to balance baseline covariates and preserve
randomness of treatment allocation. Stat Methods Med Res. 2012 Jan 26. [Epub ahead of print] PMID: 22287602
 Zhao W. Selection bias, allocation concealment and randomization design in clinical trials. Contemp Clin Trials. 2013
Sep;36(1):263-5. doi: 10.1016/j.cct.2013.07.005. Epub 2013 Jul 19. No abstract available. PMID: 23871796
 Zhao W. A better alternative to stratified permuted block design for subject randomization in clinical trials. Stat Med.
2014 Dec 30;33(30):5239-48. doi: 10.1002/sim.6266. PMID: 25043719
 Zhao W, Durkalski V. Managing competing demands in the implementation of response-adaptive randomization in
a large multicenter phase III acute stroke trial. Stat Med. 2014 Oct 15;33(23):4043-52. doi: 10.1002/sim.6213. Epub
2014 May 22. PMID: 24849843
 Zhao W, Mu Y, Tayama D, Yeatts SD. Comparison of statistical and operational properties of subject randomization
procedures for large multicenter clinical trial treating medical emergencies. Contemp Clin Trials. 2015 Mar;41:211-8.
doi: 10.1016/j.cct.2015.01.013. Epub 2015 Jan 29. PMID: 25638754