What could be going wrong?
Randomisation and blinding
CAMARADES: Bringing evidence to translational medicine
CAMARADES: Bringing evidence to translational medicine
You can usually find what you’re
looking for …
• 12 graduate psychology students
• 5 day experiment: rats in T maze with dark arm alternating at random, and the
dark arm always reinforced
• 2 groups – “Maze Bright” and “Maze dull”
Group
Day
1
Day
2
Day
3
Day
4
Day
5
“Maze
bright”
1.33
1.60
2.60
2.83
3.26
“Maze
dull”
0.72
1.10
2.23
1.83
1.83
Δ
+0.60 +0.50 +0.37 +1.00 +1.43
Rosenthal and Fode (1963), Behav Sci 8, 183-9
CAMARADES: Bringing evidence to translational medicine
Drugs in animal stroke models
NXY-059 (Astra Zeneca)
– 11 publications, 29 experiments, 408 animals
– Improved outcome by 44% (35-53%)
Efficacy 
•
Randomisation
Blinded conduct
of experiment
Blinded
assessment of
outcome
Macleod et al, 2008
CAMARADES: Bringing evidence to translational medicine
What’s a stroke drug worth?
CAMARADES: Bringing evidence to translational medicine
The same thing happens for
other diseases …
Alzheimer´s disease
Improvement in behavioural outcome
(Standardised Effect Size)
Stroke
1.2
1.0
0.8
0.6
0.4
0.2
0.0
No
Yes
Blinded assessment of behavioural outcome
Multiple Sclerosis
Parkinson´s disease
CAMARADES: Bringing evidence to translational medicine
Are many studies at risk of bias?
Disease
Randomisatio
n
(%)
Blinding
(%)
Power
calculation
(%)
Yes
No
Yes
No
Yes
No
Stroke
37
63
37
63
3
97
Alzheimer’s
16
84
22
78
0
100
Parkinson’s
16
84
15
85
0
100
Multiple
Sclerosis
9
91
16
84
0
100
Pain
12
88
26
74
0
100
CAMARADES: Bringing evidence to translational medicine
But it’s just the neurosciences,
right?
Reporting of measures to reduce bias in 254 reports of in vivo, ex vivo or in
vitro studies involving non human animals, identified from random sample of
2000 publications from PubMed
Ioannidis et al, 2014
CAMARADES: Bringing evidence to translational medicine
But not in leading institutions,
right?
“an outstanding contribution to
the internationally excellent
position of the UK in biomedical
science
and
clinical/translational research.”
“impressed by the strength
within the basic neurosciences
that were returned …particular
in the areas of behavioural,
cellular and molecular
neuroscience”
1173 publications using non
human animals, published in 2009
or 2010, from 5 leading UK
universities
Rand
Blind
I/E
SSC
CAMARADES: Bringing evidence to translational medicine
But not in journals of repute,
right?
Bottom 10%
Bottom 10% TopBottom
10% 10%Top 10%
Bottom 10% Top
10%
Top 10%
CAMARADES: Bringing evidence to translational medicine
Reporting research details
The Morris Water Maze in Alzheimer’s research
CAMARADES: Bringing evidence to translational medicine
Reporting research details
Reporting drop outs
minocycline
control
minocycline
control
CAMARADES: Bringing evidence to translational medicine