Statistical stuff happens
Vioxx, placebo power and other wonders
A few headlines
Low fat diet breast cancer hope (BBC May 2005)
Breast cancer link to high fat foods (The Scotsman, July 2003)
Low- Fat Diet May Control Prostate Cancer (Health News, August 2005)
Low-fat diet, not wine, fights heart disease in France (CNN May, 1999)
High-Fat Meal May Raise Risk Of Blood Clotting
-- Increasing Heart Attack And Stroke Risk (American Heart Association, November 1997)
National study finds no effect
from reducing total dietary fat
The study, a project of the National Institutes of Health, had taken eight years,
cost $415 million, and involved nearly 49,000 older women, 40 percent of whom
were assigned to a diet that kept their intake of calories from fat significantly
below that of the other 60 percent. Researchers had expected to confirm what
earlier studies and conventional medical wisdom had long suggested -- that
consuming less fat is good for your health.
Researchers found no difference between the two groups in terms of risk of
breast cancer, colon cancer, heart disease or stroke.
Public release date: 7- Feb-2006
Three Great Men
• Dr John Snow (1813 -1858)
• Sir Ronald Aylmer Fisher (1890 -1962)
• Sir Austin Bradford Hill (1897-1991)
Dr John Snow
The father of modern
epidemiology
Sir Ronald Aylmer Fisher
• “perhaps the most original mathematical scientist of the [twentieth]
century” Bradley Efron Annals of Statistics (1976)
• “Fisher was a genius who almost single-handedly created the
foundations for modern statistical science ….” Anders Hald A History
of Mathematical Statistics (1998)
• “Sir Ronald Fisher … could be regarded as Darwin’s greatest
twentieth-century successor.” Richard Dawkins River out of Eden
(1995)
•
“I occasionally meet geneticists who ask me whether it is true that
the great geneticist R. A. Fisher was also an important statistician.”
Leonard J. Savage Annals of Statistics (1976)
Randomisation!
Let it not be thought that Fisher in any way
regarded Randomisation as an optional extra to
significance testing. One of the earliest subheadings in his seminal book is Randomisation;
the Physical Basis of the validity of the Test.
Fisher’s many words on the subject can be boiled
down to a simple statement – no randomisation
means no significance!
The one-in-twenty lottery!
Publication
Bias!
Sir Austin Bradford Hill
• Association of cigarettes with lung cancer
• Over-emphasis of significance testing
• The Hill Criteria
– Not about causation!
Relative Risk
• If X% of people exposed to a putative
cause suffer a certain effect and Y% not
exposed to the cause (or alternatively the
general population) suffer the same effect,
the RR is X/Y
Relative Risk
• In epidemiologic research, [increases in risk of
less than 100 percent] are considered small and
are usually difficult to interpret. Such increases
may be due to chance, statistical bias, or the
effects of confounding factors that are
sometimes not evident .[Source: National
Cancer Institute, Press Release, October 26,
1994.]
• 100% increase is a relative risk of 2.0
The little red book
Some major causes of error
•
•
•
•
•
•
Statistical variation
Confusing correlation with causation
Confounding factors
Trend fitting
Extrapolation
Ignoring inconvenient data
Some major causes of error
•
•
•
•
•
•
Statistical variation
Confusing correlation with causation
Confounding factors
Trend fitting
Extrapolation
Ignoring inconvenient data
Diet Coke is mainly drunk by fat
people.
Therefore Diet Coke causes obesity.
Correlation is
not
causation!
Some major causes of error
•
•
•
•
•
•
Statistical variation
Confusing correlation with causation
Confounding factors
Trend fitting
Extrapolation
Ignoring inconvenient data
Example of confounding
• The Henry Ford Hospital in Detroit
published a report linking the use of
antibiotics in children with the subsequent
development of asthma.
• They ignored the fact that the antibiotics
must have been prescribed for serious
conditions, including lung infections.
Some major causes of error
•
•
•
•
•
•
Statistical variation
Confusing correlation with causation
Confounding factors
Trend fitting
Extrapolation
Ignoring inconvenient data
Some major causes of error
•
•
•
•
•
•
Statistical variation
Confusing correlation with causation
Confounding factors
Trend fitting
Extrapolation
Ignoring inconvenient data
What happens next?
1975
Now that’s extrapolation!
Some major causes of error
•
•
•
•
•
•
Statistical variation
Confusing correlation with causation
Confounding factors
Trend fitting
Extrapolation
Ignoring inconvenient data
Keys
Keys redrawn
The data dredge
The data dredge is a method of producing
impressive results by examining in one
cohort a number of combinations of
diseases and causes, but treating them as
though they were all independent trials.
e.g. for ten combinations P<0.05 is claimed,
whereas in reality it is P<0.401.
The Harvard Nurses Health
Study
Vitamin and carotenoid intake and risk of squamous cell carcinoma of the skin.
Vitamin D, milk consumption, and hip fractures: a prospective study among postmenopausal
women.
Prospective study of sleep duration and coronary heart disease in women.
Prospective study of self-reported sleep duration and incident diabetes in women.
Meat, fish and egg intake and risk of breast cancer.
Major dietary patterns and the risk of colorectal cancer in women.
Adolescent diet and risk of breast cancer.
Plasma folate, vitamin B6, vitamin B12, homocysteine, and risk of breast cancer.
High-dose antioxidant supplements and cognitive function in community-dwelling elderly women.
Caffeine, postmenopausal estrogen, and risk of Parkinson’s disease.
Television watching and other sedentary behaviors in relation to risk of obesity and type 2 diabetes
mellitus in women
………………..
Just a dozen of the 500 plus papers
Correlation is
not
causation!
Does
size
Matter?
21st April 2006
Jury: Merck liable in elderly man's death
RIO GRANDE CITY, Texas -- A state jury found Merck & Co. liable Friday for
the death of a 71-year-old man who had a fatal heart attack within a month
of taking its since-withdrawn painkiller Vioxx and ordered the company to
pay $32 million. Merck said it would appeal.
Vioxx was found to greatly increase the risk
of heart attacks in people who took the
painkiller for 18 months or longer.
Rofecoxib 1
FDA Advisory Committee
A Kaplan-Meier analysis (Figure 31) of the cumulative incidence of confirmed
thrombotic cardiovascular serious adverse events over time shows that the separation of
the trend lines for rofecoxib and placebo did not begin until after 18 months of
continuous daily treatment. Prior to 18 months there was no apparent difference in the
cumulative incidence of these events in the two groups as evidenced by the overlapping
lines. The changing pattern of treatment effect over time was confirmed by the failed test
for proportionality of hazards (p=0.006). The difference between rofecoxib and placebo
beginning after 18 months appears to primarily reflect a relative flattening of the placebo
curve after 18 months compared with the preceding 18 months (Figure 31). Baseline
characteristics of those patients with events beginning after 18 months were comparable
between the treatment groups (data not shown).
• The difference between rofecoxib and
placebo beginning after 18 months
appears to primarily reflect a relative
flattening of the placebo curve after 18
months compared with the preceding 18
months (Figure 31).
What they are actually saying
• Regardless of the age of the
patient, taking a placebo for
more than eighteen months
effectively prevents heart
disease!
Electric breakdown (1964)
The law of experiments
The first trial always produces a result that is
bizarre and points to a great scientific
breakthrough.
First corollary
The effect never reappears in any subsequent
trials.
The law of experiments
The first trial always produces a result that is
bizarre and points to a great scientific
breakthrough.
First corollary
The effect never reappears in any subsequent
trials.
Second corollary
In fields such as epidemiology and drug
testing there is only one trial.
And finally
This lecture has dealt with the lunatic fringe
of certain branches of science in general and
epidemiology in particular. It is important to
remember, however, that the tradition of Dr
John Snow lives on in the huge organisation
of disease detectives, who monitor potential
epidemics and organise the international
response to them.
www.numberwatch.co.uk