Validity
Generalization
Hein Stigum
Presentation, data and programs at:
http://folk.uio.no/heins/
May-16
H.S.
1
Precision and validity
• Measures of populations
– precision - random error - statistics
– validity - systematic error – epidemiology
• Lack of validity = measures are biased
– type of bias
– direction of the bias
Precision
Bias
True
value
May-16
H.S.
Estimate
2
Type of bias
• Selection bias
– Are those who answer different?
• Information bias
– Do they tell the truth?
• Confounding
– Is the association a cause?
May-16
H.S.
3
Selection bias
May-16
H.S.
4
Sources of selection bias
• Selective response
– sexual survey
• Self selection
– Nevada atom test and leukemia
• Loss to follow up connected to disease
– air pollution and astma
• Healthy worker effect
– aluminium workers and lung disease
May-16
H.S.
5
Selection bias
Population
Sample
Respons
Responders
Non-responders
Outcome
May-16
H.S.
6
Selective survival
Selective survival
6
Disease Death
5
4
3
2
1
0
0
May-16
2
4
6
H.S.
8
10
7
Information bias
May-16
H.S.
8
Non-differential misclassification
Heart disease
+
+ 200
800
Smoke
- 100
2 900
RR=
True smoking
6.0
Heart disease
+
+ 180
720
Smoke
- 120
2 980
RR=
1 000
3 000
4 000
10% of smokers report
no smoking
900
3 100
4 000
5.2
Non-differential:
May-16
H.S.
RR
9
Sources of information bias
• Not true
– males report more partners than females
• Not blinded
– passive smoking and astma
• Selective recall
– alcohol in pregnancy and malformations
May-16
H.S.
10
CONFOUNDER DEFINITION
May-16
H.S.
11
Confounding
• Ideal:
Exposed
– Same subjects are both
exposed and unexposed at the
same time, (counterfactual)
• Practice:
Unexposed
– As equal as possible
• Comparison bias:
– Confounding
May-16
H.S.
12
Associations
• E and D associated
E
D
– E causes D
C
– E and D have common cause
E
D
C
– Both
E
May-16
D
• Overall E-D association =
spurious effect from C
+ causal E-D effect
H.S.
13
Classic confounder
C is the confounder
RRED
is biased
RRED|C is unbiased
C
E
D
age
+
+
vitamins
-
birth defects
RRED =0.8 positive bias
Adjust for age:
RRED|C=0.5 is unbiased
true biased
0
May-16

1
H.S.
14
Confounding: Downs syndrom by parity
Downs syndrom
0.3
0.25
0.2
%
0.15
0.1
0.05
0
0
1
S1
2
Parity
May-16
H.S.
3+
15
Downs by parity and mothers age
0.5
0.4
0.3
%
0.2
0.1
0
0
May-16
1
>32
<32
2
H.S.
3+
16
Downs syndrom, logistic regression
Crude
Adjusted
Odds Confidence
Ratio interval
Variable
Parity
+1 child
1.4
Adjusted
Odds Confidence
Ratio interval
Variable
Parity
+1 child
Mothers age
+1 year
May-16
(1.2 - 1.6)
H.S.
1.0
(0.8 - 1.2)
1.3
(1.2 - 1.3)
17
Generalization
May-16
H.S.
18
Generalization
• Do the result apply outside the sample?
• Statistical generalization
• Smoking among males, generalize to females?
– Representative sample
• Biological generalization
• Drug effect on males, generalize to females?
– Information from outside the study
• Animal studies, generalize to humans?
– Homogenous sample
May-16
H.S.
19