Using
r and 2
Statistics
for
Risk Analysis
Copyright 2002
David M. Hassenzahl
Objectives
• Purpose: to compare model to data
– “validate model” (or not)
• Two techniques
– r (correlation coefficient)
– 2 (Chi-squared)
• Apply to a familiar problem (barium
decay)
Copyright 2002
David M. Hassenzahl
Statistics
• Descriptive
• Comparison
– Z-scores, hypotheses
– Confidence levels
– Evaluating models
– Correlation and Chi-squared
Copyright 2002
David M. Hassenzahl
Confidence Levels
• Given 100 flips of a coin. Would you bet
$1000 that the next flip will yield heads if
–
–
–
–
50 heads?
90 heads?
99 heads?
999 heads out of the last 1000 flips?
• How about for $5? For 50% of your current
net worth?
Copyright 2002
David M. Hassenzahl
Statistical Significance
• Z=
– (sample occurrence – number in sample
times expected probability
– Divide by square root of (np(1-p)
•
•
•
•
Student’s t
One-sided versus two sided tests!
“p values”
Confidence intervals
Copyright 2002
David M. Hassenzahl
Type I and II errors
• Type I: reject the truth! (accuracy)
• Type II: accept an untruth! (precision)
• This is important… there’s often a
tradeoff here!
Copyright 2002
David M. Hassenzahl
Z-scores
Intuition
x  μ0
Z
s n
• Z score will be big if
– Numerator: if xbar >> OR  >> xbar
– s is very small
– n is very big
• Bigger Z-score: confidence that   xbar
• Small Z-score: confidence that   xbar
Copyright 2002
David M. Hassenzahl
From Z’s to r’s and 2
• r and 2 compare more than one
estimate
• Compare
– Set of model predictions to
– Set of data or observations
• If r is SMALL (little correlation) the
model doesn’t fit
• If 2 is SMALL then the model does fit
Copyright 2002
David M. Hassenzahl
“Goodness of fit”
• We say that r and 2 evaluate
“goodness of fit”
• Note that a good fit does not mean that
the model is right!
Copyright 2002
David M. Hassenzahl
Barium Decay
• Theory: barium is removed as a constant
function of concentration
• “Exponential decay”
• C(T) = C(0)ekT
– k = -0.007/min
– C(0) = 0.16 mgBa / liter blood
• (From SWRI page 56 – 63; hypothetical)
Copyright 2002
David M. Hassenzahl
Exponential Decay Model
0.16
Blood barium
concentration (mg/l)
0.12
0.08
0.04
0
0
60
120
180
240
300
360
420
time
Figure 2-9 from Should We Risk It?
Copyright 2002
David M. Hassenzahl
Sample blood at 1 hour intervals
Time (hours)
0
1
2
3
4
5
6
7
8
Measured
Concentration
0.16
0.13
0.087
0.055
0.040
0.022
0.009
0.002
Copyright 2002
0.001
David M. Hassenzahl
Measured and Expected
Time (hours)
0
1
2
3
4
5
6
7
8
Measured
Concentration
0.16
0.13
0.087
0.055
0.040
0.022
0.009
0.002
0.001
Predicted
Concentration
0.16
0.11
0.070
0.045
0.030
0.020
0.013
0.0095
2002
0.0056David Copyright
M. Hassenzahl
Graphical Comparison
0.16
Blood barium
concentration (mg/l)
0.12
0.08
0.04
0
0
60
120
180
240
300
360
420
time
After Figure 2-9 from Should We Risk It?
Copyright 2002
David M. Hassenzahl
How well does the model fit?
• Why do we care?
– Future predictions
– Is there a better model?
• Looks OK. Is that good enough?
• Try our two tools: r and 2
Copyright 2002
David M. Hassenzahl
r Conceptual
• Compares model predictions to the data
• Asks
– “What if there is no relationship (or
correlation) between model and data?”
– Is the model as close to the average value
of the x’s as it is to the actual x’s?
Copyright 2002
David M. Hassenzahl
r terms or components
• Predicted mean and standard deviation
• Observed mean and standard deviation
• “Covariance”
– Do they go up and down together?
– If independent, covariance = 0
• r = Covariance (predicted, observed)
(STDEV O)  (STDEV P)
Copyright 2002
David M. Hassenzahl
Means
• Observed xobar = ( xoi) /n
• xobar =
(0.16+0.13+0.087+0.055+0.040
+0.022+0.009+0.002+0.000)/9
= 0.056
• Observed xpbar = ( xpi) /n =
0.051
Copyright 2002
David M. Hassenzahl
Standard Deviations
so 
1

x

n 1
o
 xoi   0.056
sp 
1

x

n 1
p
 x pi   0.051
2
2
Copyright 2002
David M. Hassenzahl
Covariance
Covo, p   1

x

n 1
o
 xoi x p  x pi 
Copyright 2002
David M. Hassenzahl
Calculated r
• r = Covariance (predicted, observed)
(STDEV O)  (STDEV P)
cov p , o
r
 0.99
so  sp
Copyright 2002
David M. Hassenzahl
Intuition Behind r
• If there is no relationship between
observed and predicted, r = 0
• If r  0, positive correlation
• If r  0, negative correlation
Copyright 2002
David M. Hassenzahl
r Discussed
•
•
•
•
0.99 seems reasonably good
Is there a better fit
What about theory?
Limitations: even low correlations may
be okay…just a screening tool
Copyright 2002
David M. Hassenzahl
t test for r
• n-2 = 7 degrees of freedom
• Look it up in the Student’s-t table
• Accept model validity at 99%
confidence level if Student’s t is greater
than 2.998
t
r n2
1 r
2
Copyright 2002
David M. Hassenzahl
Chi-squared
n
χ 
2
i 1
x
 x pi 
2
oi
x pi
• This formula “normalizes” to the size of the
individual xoi
• If all xoi  xip, 2 = 0
• Look up value in table (page 398)
Copyright 2002
David M. Hassenzahl
Chi-squared
• 9 data points
• Suppose we are concerned with 99%
confidence level
• We would need a chi-squared of greater
than 21.7 to reject this line
• Calculating, we find that 2 = 0.06!
• Note that it still might be possible to find
a better line, even with the exponential
Copyright 2002
David M. Hassenzahl
Conclusion
• Both r and Chi-squared appear to
validate this model
• Suggests that our theoretical idea about
the model may be valid
• Doesn’t tell us we are right, just that we
may be acceptably wrong!
Copyright 2002
David M. Hassenzahl