Statistics don’t lie – do people?
Janez Stare
Faculty of Medicine, Ljubljana
USA Today has come out with a new survey
– apparently, three out of four people make
up 75% of the population.
David Letterman
On the other hand
It's amazing how authoritative you can sound
just by quoting some statistics ...
And certainly
Without data it is anyone’s opinion ...
(In God we trust; all others must bring data.)
2
So – statisticians
don’t lie?
3
A researcher viewed 107 published
studies comparing a new drug and a traditional
therapy and found "studies
of new drugs sponsored by drug companies
were more likely to favor those drugs than
studies supported by noncommercial entities".
In not a single case was a drug or treatment
manufactured by the sponsoring company
found inferior to another company's product.
4
A typical lie
Cigarette manufacturer Lorillard claimed
that "TRIUMPH BEATS MERIT" because "an
amazing 60 percent said Triumph tastes as
good or better than Merit.“
Actually, 36 percent preferred Triumph, 24
percent said they were equal, and 40
percent preferred Merit.
5
Phases of research
•
•
•
•
Planning
Collecting data
Data Analysis (together with description)
Interpretation of results
We can ‘lie’ in every phase!
6
Planning of research and data
collection
Example:
100 measurements on one sheet of paper
100 measurements on another sheet
But – measurements are paired!
And the guy doesn’t know how!
When we plan our research, we must know
what methods of analysis will be used!
7
Missing data!
Example: duration of labour
Two phases
Measured variables:
Duration of the first phase
Dur. of the second phase
Total duration
We got:
x1
x2
x3
x1  x3 !!!
8
Some lying graphs
‘Figures don’t lie, but liars can figure’
New York Times
9
Washington
Post
10
11
What a fall!!
12
Lower rang is better!!
13
And some desperately bad graphs
?
14
Analysis
hospital
n
1
2
3
4
5
6
7
8
9
10
11
64
49
67
68
70
45
73
97
125
80
46
dead
3
6
1
1
5
1
7
3
10
2
4
% dead
4,7
12,2
1,5
1,5
7,1
2,2
9,6
3,1
8,0
2,5
8,7
Does hospital 2 stand out?
And what if hospitals are compared to some standard (say 5%)?
15
PID-PAB ANALIZA
COOP WONCA vprašalnik:
SKUPNO: sešteti točke iz posameznih vprašanj (minimalno število
je 6, maksimalno pa 30). Primerjati skupini s PAB in brez PAB glede
na skupno število točk.
ANALIZA
Analizirati, kako posamezne spremenljivke vplivajo na kvaliteto
življenja (COOP WONCA vprašalnik), tako na posamezne vidike
kvalitete življenja kot na skupno oceno (seštevek točk).
Analizirati ločeno za bolnike s PAB in ločeno za paciente brez
PAB, ter za celo skupino pacientov skupaj. Analizirati vsaj:
starost, spol, BMI, pas, sistolični in diastolični tlak,
hemoglobin, s-glukoza, s-K, urea, kreatinin, CRP, celokupni
holesterol, HDL, LDL, trigliceridi, u-proteini, u-glukoza, SCORE,
minimalni GI, znižan GI min, aterosklerotična bolezen, angina
pectoris, akutni koronarni sindrom, zožitev karotidne arterije,
16
ishemični napad, možganska kap, intermitentna klavdikacija,
klavdikacijska razdalja, ishemija uda, bolezni v družini, sladkorna
bolezen, hipirlipidemije, arterijska hipertenzija, kajenje,
razdražljivost, spanje, alkohol, sadje, zelenjava, zmerno gibanje,
intenzivno gibanje, individualno svetovanje, skupinsko svetovanje,
antiagregacijska terapija-skupaj, lipolitiki-skupaj, ACE in
sartani-skupaj, antihipertenzivi, diuretiki, število
zdravil-skupaj (to naj bo nova spremenljivka)
The guy wanted 1114 tables with corresponding tests!
17
10
8
6
4
2
And something here
12
14
Do the assumptions hold?
2
4
6
8
10
Something here
12
14
18
When we need to know a bit
more
Example: Somebody was ‘explaining’ GDP for
eleven years with seven variables in a
regression equation. He got R2 = 0,95.
Wow! Bravo!
But:
The expected value of R2 = 0,7 (under the null
R2 = 0)!!
19
The famous 5 percent (or 1%)
• Examples of reviews:
– Please state that side effects were NOT
different (p = 0.058).
– Either something IS significantly
different or IT IS NOT.
– Please delete discussion of nonstatistically significant results from
the text.
• Fisher
• How much is 5%?
• What is the difference between 5,1% and 4,9%?
20
400
200
New law
0
dead
600
800
Interpretation of results
1990
1995
2000
year
2005
21
0.4
0.6
men
0.2
women
0.0
Survival Probability
0.8
1.0
Survival after AMI by sex
0
1
2
3
4
5
6
7
8
Years
9
10
11
12
13
14
15
16
22
0.6
0.4
men
0.2
women
0.0
Survival Probability
0.8
1.0
Predicted survival by sex after controlling for age
0
1
Adjusted to: age=61
2
3
4
5
6
7
8
Years
9
10
11
12
13
14
15
16
23
Relative survival of men and women
24
There are no routine statistical questions,
there are only questionable statistical
routines
D.R. Cox
25
lie –– people
people can
do.!
can’t lie
Statistics don’t
26