Stat 231 List of Concepts and Formulas
"Course Review" Sheets
Prof. Stephen Vardeman
Iowa State University
Statistics and IMSE
September 6, 2011
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
1 / 58
Day 1-Introduction
Probability vs Statistics
Simple Descriptive Statistics
x
=
s2 =
1 n
xi
n i∑
=1
1
n
n
1 i∑
=1
(xi
x )2
Properties of x̄ and s
for y = ax + b, y = ax + b and sy = jaj sx
JMP
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
2 / 58
Day 2-Notions of "Chance" and Mathematical Theory
Sample Space (Universal Set) S
Events (Sets) A, B
Empty Event ∅
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
3 / 58
Day 3-Set Operations on Events
Words to Symbols and Symbols to Words
Set Operations on Events
= A\B
A or B = A [ B
not A = Ac (Ā)
A and B
Mutuality Exclusive (Disjoint) Events A, B
A and B = ∅
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
4 / 58
Day 4-Axioms of Probability and "the Addition Rule"
Basic Rules of Operation
1
2
3
0 P (A) 1
P (S) = 1 (and P (∅) = 0)
If A1 , A2 , . . . are disjoint events
P (A1 or A2 or . . .) = P (A1 ) + P (A2 ) +
A Small "Theorem"
P (not A) = 1
P (A)
The "Addition Rule" (Another Theorem)
P (A or B ) = P (A) + P (B )
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
P (A and B )
September 6, 2011
5 / 58
Day 5-Conditional Probability and Independence of Events
Conditional Probability of A Given B
P (AjB ) =
P (A and B )
P (B )
The "Multiplication Rule"
P (A and B ) = P (AjB ) P (B )
Events A, B are Independent Exactly When
P (AjB ) = P (A) i.e. when P (A and B ) = P (A) P (B )
(Multiple Events are Independent When Every Intersection of Any
Collection of Them (or Their Complements) Has Probability
Obtainable as a Product of Individual Probabilities)
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
6 / 58
Day 6-Counting
When Outcomes are Equally Likely
P (A) =
# (A)
# (S)
A Basic Principle: When a complex action can be broken into a series of k
component actions, the …rst of which can be done n1 ways, the second of
which can subsequently be done n2 ways, the third of which can
subsequently be done n3 ways, etc., the whole can be accomplished in
n1 n2
nk
di¤erent ways
Count of Possible Permutations
Pr ,n =
n!
(n
r )!
Count of Possible Combinations
n!
n
=
r
r ! (n r ) !
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
7 / 58
Day 7-Discrete Random Variables and Specifying Their
Distributions
Probability Mass Function
f (x ) = P [X = x ]
Cumulative Distribution Function
F (x ) = P [X
F (x ) =
x ] (general)
∑ f (z ) (discrete)
z x
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
8 / 58
Day 8-Expectation for Discrete Variables
Expected (or Mean) Value of h (X ) for a Discrete X
Eh (X ) =
∑ h (x ) f (x )
x
Mean of X (Mean/Center of the Distribution of X )
EX =
∑ xf (x )
x
( = µX )
Variance of X (a Measure of Spread for the Distribution of X )
Var X
=
=
∑(x EX )2 f (x ) ( = σ2X )
∑ x 2 f (x ) (EX )2
= EX 2
Vardeman (ISU Stat and IMSE)
(EX )2
Stat 231 Summary
September 6, 2011
9 / 58
Day 9-More Mean and Variance/Independent Identical
Success-Failure Trials
Chebyschev’s Inequality (general)
P [ µX
kσX < X < µX + kσX ]
1
1
k2
Other Useful Facts (general)
E (aX + b ) = aEX + b
Var (aX + b ) = a2 Var X
q
σaX +b =
Var (aX + b ) = jaj σX
A Convenient (and Sometimes Appropriate) Model is the "Bernoulli
Trials" Model:
1
2
P [success on trial i ] = p (…xed, the same for all i)
The events Ai = "success on trial i" are all independent
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
10 / 58
Day 10-Binomial and Geometric Distributions
Under the Bernoulli Trials Model:
X = the number of successes in n trials Has the Binomial(n, p )
Distribution
8
< n
p x (1 p )n x for x = 0, 1, . . . , n
x
f (x ) =
:
0
otherwise
With EX = np and Var X = np (1 p )
X = the trial on which the …rst success occurs
Geometric(p ) Distribution
f (x ) =
With 1
F (x ) = (1
Vardeman (ISU Stat and IMSE)
p (1
0
p )x
p )x , EX =
1
Has the
for x = 1, 2, . . .
otherwise
1
1 p
and Var X =
p
p2
Stat 231 Summary
September 6, 2011
11 / 58
Day 11-Geometric and Poisson Distributions
The Poisson(λ) Distribution is a Commonly Used Model for
X
= the number of occurrences of a relatively rare
phenomenon across a …xed interval of time or space
This Has
f (x ) =
8
< e
λ λx
: 0 x!
for x = 0, 1, 2, . . .
otherwise
With EX = λ and Var X = λ
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
12 / 58
Day 12-Continuous Random Variables, pdf’s and cdf’s
Probability Density Function, f (x )
P [a
X
b] =
0 with
Z b
f (x ) dx
a
(Continuous) Cumulative Distribution Function
F (x ) = P [X
x] =
Z x
∞
f (t ) dt
cdf to pdf
d
F (x ) = f (x )
dx
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
13 / 58
Day 13-Expectation for Continuous Variables/Normal
Distributions
Expected (or Mean) Value of h (X ) for a Continuous X
Eh (X ) =
Z ∞
∞
h (x ) f (x ) dx
Mean of X (Mean/Center of the Distribution of X )
EX =
Z ∞
∞
xf (x ) dx ( = µX )
Variance of X (a Measure of Spread for the Distribution of X )
Var X
=
=
Z ∞
Z
∞
∞
∞
= EX 2
(x
EX )2 f (x ) dx ( = σ2X )
x 2 f (x ) dx
(EX )2
(EX )2
All the Day 9 Facts Hold for Continuous Variables
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
14 / 58
Day 14-Normal Distributions
Normal µ, σ2 pdf
f (x ) = p
1
2πσ2
e
(x µ)2 /2σ2
for all x
Standard Normal (µ = 0, σ = 1) Version
1
f (z ) = p e
2π
z 2 /2
for all z
Standard Normal cdf (tabled)
Φ (z ) = F (z ) =
Z z
∞
1
p e
2π
t 2 /2
dt
Conversion to Standard Units
z=
Vardeman (ISU Stat and IMSE)
x
µ
σ
Stat 231 Summary
September 6, 2011
15 / 58
Day 15-Normal Approximation to Binomial/Exponential
and Weibull Distributions
For Large n and Moderate p, a Binomial(n, p ) Distribution is
Approximately Normal (With µ = np and σ2 = np (1 p ))
The Exponential(λ) Distribution Has
λe
0
f (x ) =
λx
for x > 0
otherwise
1
1
, Var X = 2 and F (x ) =
λ
λ
The Weibull(α, β) Distribution Has
With EX =
0
1
F (x ) =
With Median F
Vardeman (ISU Stat and IMSE)
1 (.5)
= βe
e
(x /β)α
(.3665/α)
Stat 231 Summary
0
1
e
λx
if x 0
if x > 0
if x < 0
if x 0
and Scale Parameter β
September 6, 2011
16 / 58
Day 16-Jointly Discrete Random Variables
Joint Probability Mass Function
f (x, y ) = P [X = x and Y = y ]
Marginal Probability Mass Functions
g (x ) =
∑ f (x, y ) and h(y ) = ∑ f (x, y )
y
x
Conditional Probability Mass Functions
g (x j y ) =
Vardeman (ISU Stat and IMSE)
f (x, y )
f (x, y )
and h (y j x ) =
h (y )
g (x )
Stat 231 Summary
September 6, 2011
17 / 58
Day 17-Jointly Discrete and Continuous Variables
Independence of Discrete Random Variables
f (x, y ) = g (x )h (y ) for all x, y
Joint Probability Density Function f (x, y )
P [(X , Y ) 2 R] =
Z Z
0
f (x, y ) dx dy
R
Marginal Probability Density Functions
g (x ) =
Vardeman (ISU Stat and IMSE)
Z ∞
∞
f (x, y ) dy and h (y ) =
Stat 231 Summary
Z ∞
∞
f (x, y ) dx
September 6, 2011
18 / 58
Day 18-Continuous Variables, Conditionals, and
Independence
Conditional Probability Densities
g (x j y ) =
f (x, y )
f (x, y )
and h (y j x ) =
h (y )
g (x )
Independence of Continuous Random Variables
f (x, y ) = g (x )h (y ) for all x, y
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
19 / 58
Day 19-Functions of Several Random Variables and
Expectation
For Jointly Distributed Variables X , Y , . . . , Z the Distribution of
U = g (X , Y , . . . , Z ) Can Sometimes Be Derived
JMP Simulation to Approximate the Distribution of
U = g (X , Y , . . . , Z ) (and EU) for Independent X , Y , . . . , Z
Expectation of U = g (X , Y )
Eg (X , Y ) =
Eg (X , Y ) =
Vardeman (ISU Stat and IMSE)
∑ g (x, y ) f (x, y )
x ,y
Z ∞
∞
Z ∞
∞
(discrete)
g (x, y ) f (x, y ) dx dy (continuous)
Stat 231 Summary
September 6, 2011
20 / 58
Day 20-Covariance, Correlation, and Laws of Expectation
Cov (X , Y ) = E (X
EX) (Y
= EXY
EX EY
EY )
( = E (X
( = EXY
µX ) (Y
µY ) )
µX µY )
Cov (X , Y )
Cov (X , Y )
ρ = Corr (X , Y ) = p
=
σX σY
VarX VarY
1 ρ 1 With ρ =
Linearly Related
1 Exactly When X and Y are Perfectly
X , Y Independent Implies ρ = 0
E(aX + b ) = aEX + b (from Day 9)
X , Y Independent Implies Es (X ) t (Y ) = Es (X )Et (Y )
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
21 / 58
Day 21-Laws of Expectation and Variance
Var X = EX 2
(EX )2 (From Day 8)
Var (aX + b ) = a2 Var X (From Day 9)
Var (aX + bY ) = a2 Var X + b 2 Var Y + 2abCov(X , Y )
X , Y Independent Implies Var (aX + bY ) = a2 Var X + b 2 Var Y
For Independent X , Y , . . . , Z , U = a0 + a1 X + a2 Y +
= a0 + a1 EX + a2 EY +
Var U = a12 Var X + a22 Var Y +
EU
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
+ an Z Has
+ an EZ
+ an2 Var Z
September 6, 2011
22 / 58
Day 22-Propagation of Error/Transition to Statistics
For Independent X , Y , . . . , Z Approximations for the Mean and
Variance for U = g (X , Y , . . . , Z ) Are
Eg (X , Y , . . . , Z )
Var g (X , Y , . . . , Z )
g ( µX , µY , . . . , µZ )
∂g
∂x
2
Var X +
∂g
∂z
+
2
Var Z
(Where the Partials Are Evaluated at µX , µY , . . . , µZ )
Random Sampling From a Large Population or a Physically Stable
Process is (at Least Approximately) Described by a Model That Says
Data X1 , X2 , . . . , Xn Are Independent Identically Distributed Random
Variables (With Marginal Probability Distribution the Population
Relative Frequency Distribution)
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
23 / 58
Day 23-Distributional Properties of the Sample Mean
If X1 , X2 , . . . , Xn Are Independent Identically Distributed (Each With
Mean µ and Variance σ2 ) the Random Variable
X =
1
( X1 + X2 +
n
+ Xn )
Has
EX
Var X
= µX = µ
σ2
= σ2X =
n
Further, X is Approximately Normal if
1
The Population Distribution is Itself Normal
2
The Sample Size, n, is Large (The Central Limit Theorem)
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
24 / 58
Day 24-Introduction to Con…dence Intervals
Following From
Z =
X µ
p is (at least approximately) Standard Normal
σ/ n
The Interval Formula
X
σ
σ
zp ,X +zp
n
n
Will Cover µ In a Fraction P ( z < Z < z ) of All Applications
The End Points
X
σ
zp
n
Are Thus (Typically Practically Unusable) Con…dence Limits for µ
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
25 / 58
Warning About Convention
Henceforth Drop the Convention That Random Variables Are Represented
By Capital Letters and Their Possible Values by Lower Case Letters.
Typically (But Not Always) Lower Case Will Be Used For Both, and
Context Will Have to Be Used to Distinguish.
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
26 / 58
Day 25-Large Sample Con…dence Intervals for Means and
Proportions
Large n Con…dence Limits for µ
x
Follow From
x µ
p
Z =
s/ n
s
zp
n
(at least approximately) Standard Normal
For Large n, Con…dence Limits for p
r
p
e (1
p
b z
p
e)
n
(Where p
e = (nb
p + 2) / (n + 4)) Follow From
Z = r
p
b
p
p (1
Vardeman (ISU Stat and IMSE)
p)
(at least approximately) Standard Normal
n
Stat 231 Summary
September 6, 2011
27 / 58
Day 26-Small Sample Con…dence Intervals for a (Normal)
Mean
Small n Con…dence Limits for µ (When Sampling From a Normal
Distribution)
s
x tp
n
(For t a Percentage Point of the tn
T =
Vardeman (ISU Stat and IMSE)
1
x µ
p
s/ n
Stat 231 Summary
Distribution) Follow From
tn
1
September 6, 2011
28 / 58
Day 27-Small Sample Con…dence Intervals for a (Normal)
Standard Deviation and Normal Prediction Limits
Con…dence Limits for σ (For a Normal Distribution)
s
s
n 1
n 1
s
and s
χ2upper
χ2lower
Follow From
X2 =
(n
1) s 2
χ2n 1
σ2
Prediction Limits for xnew (From a Normal Distribution)
r
1
x ts 1 +
n
(For t a Percentage Point of the tn 1 Distribution) Follow From
x xnew
T = r
tn 1
1
s 1+
n
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
29 / 58
Day 28-Normal Prediction and Tolerance Limits/Normal
Plotting
"Tolerance" Limits for a Large (User Chosen) Part of a Normal
Distribution
x
τ 2 s (two-sided)
x
τ 1 s or x + τ 1 s (one-sided)
Where τ 2 or τ 1 is Chosen For Given "Part of the Distribution" and
Con…dence Level
Normal Plots For an Ordered Data Set x1 x2
xn Made
Plotting n Points
i
.5
data quantile,
n
=
xi , Φ
1
Vardeman (ISU Stat and IMSE)
i
.5
n
i
.5
n
standard normal quantile
= (xi , zi )
Stat 231 Summary
September 6, 2011
30 / 58
Day 29-Hypothesis Testing Introduction 1
Devore 7-Step Format
Null and Alternative Hypotheses
Test Statistic
Type 1 and Type 2 Errors
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
31 / 58
Day 30-Hypothesis Testing Introduction 2
Test Criteria/Rejection Criteria and Corresponding Error Probabilities
α, β and Their Competing Demands
Hypothesis Testing/Criminal Trial Analogy
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
32 / 58
Day 31-One-Sample Testing for a Mean
Large n Testing of H0 :µ = # Uses
Z =
x #
p
s/ n
and a Standard Normal Reference Distribution
(Normal Distribution) Small n Testing of H0 :µ = # Uses
T =
and a tn
1 Reference
Vardeman (ISU Stat and IMSE)
x #
p
s/ n
Distribution
Stat 231 Summary
September 6, 2011
33 / 58
Day 32-One Sample Testing for a Proportion/"p-values"
Large n Testing of H0 :p = # Uses
Z = r
p
b
#
# (1
#)
n
and a Standard Normal Reference Distribution
In ANY Hypothesis Testing Context
p-value = "observed level of signi…cance"
= the probability (computed under H0 )
of seeing a value of the test statistic
"more extreme" than the one observed
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
34 / 58
Day 33-One Sample Testing for a (Normal) Standard
Deviation/(Large) Two-Sample Inference for Means
(Normal Distribution) Testing H0 :σ2 = # Uses
X2 =
(n
1) s 2
#
and a χ2n 1 Reference Distribution
Large n1 and n2 (Independent Samples) Con…dence Limits for
µ1 µ2 are
s
x1
x2
and a Test Statistic (for H0 :µ1
z
s12
s2
+ 2
n1
n2
µ2 = #) is
x1 x2 #
Z = s
s12
s2
+ 2
n1
n2
With Standard Normal Reference Distribution
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
35 / 58
Day 34-(Small) Two-Sample Inference for (Normal) Means
(Somewhat Approximate) Small n1 or n2 (Normal Distribution)
(Independent Samples) Con…dence Limits for µ1 µ2 are
s
s12
s2
x 1 x 2 bt
+ 2
n1
n2
(for bt a Percentage Point of the t Distribution With
d.f . = min (n1 1, n2 1)) and a Test Statistic (for
H0 :µ1 µ2 = #) is
x1 x2 #
T = s
s12
s2
+ 2
n1
n2
With t Reference Distribution With d.f . = min (n1
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
1, n2
1)
September 6, 2011
36 / 58
Day 35-Inference for a Mean Di¤erence/a Di¤erence in
Proportions
When Paired Values (x, y ) Can Sensibly be Reduced to Di¤erences
d =x
y
and n of These to d and sd , One-Sample Inference Formulas Apply to
Inference for µd .
Large n1 , n2 (Independent Samples) Con…dence Limits for p1 p2 are
s
pe1 (1 pe1 ) pe2 (1 pe2 )
+
pb1 pb2 z
n1
n2
(Where pe1 = (n1 pb1 + 2) / (n1 + 4) and pe2 = (n2 pb2 + 2) / (n2 + 4) .)
A Test Statistic for H0 :p1 p2 = 0 is
pb1 pb2
r
Z = q
1
1
p
b (1 p
b)
+
n1
n2
With a Standard Normal Reference Distribution.
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
37 / 58
Day 36-Inference for a Ratio of (Normal) Variances
Where s1 and s2 are Based on Independent Samples from Normal
Distributions With Respective Standard Deviations σ1 and σ2 ,
F =
s12 /σ21
s22 /σ22
Has the (Snedecor) F Distribution With n1 1 and n2
Freedom.
Hence, Con…dence Limits for σ1 /σ2 are
s
s
p1
p1
and
s2 Flower
s2 Fupper
1 Degrees of
and H0 :σ1 = σ2 is Tested Using
F =
s12
s22
and an F Reference Distribution With n1
Freedom.
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
1 and n2
1 Degrees of
September 6, 2011
38 / 58
Day 37-Least Squares Fitting of a Line
Based on n Data Pairs (x1 , y1 ) , . . . , (xn , yn )
The "Least Squares Line" Through the Scatterplot Has
slope b1 =
intercept
∑ni=1 (xi x )(yi
∑ni=1 (xi x )2
b0 = y
y)
=
(∑ni=1 xi ) (∑ni=1 yi )
n
n
(∑i =1 xi )2
n
2
∑i =1 xi
n
∑ni=1 xi yi
b1 x
The Sample Correlation Between x and y is
r
=
=
∑ni=1 (xi x )(yi y )
∑ni=1 (xi x )2 ∑ni=1 (yi y )2
(∑ni=1 xi ) (∑ni=1 yi )
∑ni=1 xi yi
n
v
!
!
u
2
2
n
n
u
x
y
(
)
(
)
∑
∑
i
i
i
=
1
i
=
1
n
n
t ∑ x2
∑i =1 yi2
i =1 i
n
n
p
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
39 / 58
Day 38-Coe¢ cient of Determination and the SLR Model
Based on n Data Pairs (x1 , y1 ) , . . . , (xn , yn ) and Least Squares Fitted
Values ybi = b0 + b1 xi
SSTot = (n
n
1) sy2 =
∑ (yi
y )2 ,
n
SSE =
i =1
and SSR
= SSTot
∑ (yi
i =1
SSE
ybi )2
Then
R2 =
SSR
= (sample correlation of y and yb)2 ( = r 2 in SLR only )
SSTot
The (Normal) Simple Linear Regression Model is
yi = β0 + β1 xi + ei
for independent N 0, σ2
random "errors" ei
(Responses are Independent Normal Variables with Means
µy jx = β0 + β1 x and Constant Standard Deviation, σ)
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
40 / 58
Day 39-Inference Under the SLR Model 1
Single-Number Estimates of SLR Model Parameters are
r
SSE c
b=s
σ
, β = b1 , and c
β 0 = b0
n 2 1
χ2n 2 , Con…dence Limits for σ are
s
s
n 2
n 2
s
and s
2
χUpper
χ2Lower
p
Since b1 is Normal
Mean β1 and StdDev σ/ ∑ni=1 (xi x )2 ,
p With
Write SEb1 = s/ ∑ni=1 (xi x )2 and Have Con…dence Limits for β1
Using (n
2) s 2 /σ2
b1
And Test H0 :β1 = # Using a tn
T =
Vardeman (ISU Stat and IMSE)
t SEb1
2
Reference Distribution for
b1 #
SEb1
Stat 231 Summary
September 6, 2011
41 / 58
Day 40-Inference Under the SLR Model 2
Since yb = b0 + b1 xnew Has Mean µy jxnew = β0 + β1 xnew and StdDev
s
r
1
(xnew x )2
(xnew x )2
1
+
σ
+ n
,
Write
SE
=
s
y
b
n
∑ni=1 (xi x )2
n ∑i =1 (xi x )2
Con…dence Limits for µy jxnew = β0 + β1 xnew are Then
yb
t SEyb
And H0 :µy jxnew = # May Be Tested Using a tn
Distribution for
yb #
T =
SEyb
2
Reference
Prediction Limits for ynew at x = xnew Are
s
1
(xnew x )2
yb t s 1 + + n
n ∑i =1 (xi x )2
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
42 / 58
Day 41-SLR and "ANOVA"
Breaking Down SSTot Into SSR and SSE Is a Kind of "ANalysis Of
VAriance" (of y ). Further, an F Test of H0 :β1 = 0 Equivalent to a
Two-Sided t Test Can be Based on F = MSR/MSE and an F1,n 2
Reference Distribution. Calculations Are Summarized in a Special
"ANOVA" Table.
Source
Regression
Error
Total
SS
SSR
SSE
SSTot
ANOVA Table (for SLR)
df
MS
1
MSR = SSR/1
n 2 MSE = SSE /(n 2)
n 1
F
F = MSR/MSE
The Facts that EMSE = σ2 and EMSR = σ2 + β21 ∑ni=1 (xi
Provide Motivation for Rejecting H0 For Large Observed F .
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
x )2
September 6, 2011
43 / 58
Day 42-Practical Considerations 1
The Possibility that Neither Interpolation Nor Extrapolation is
Completely Safe Must Be Considered When Using a Fitted Equation.
Rational Practice Requires That One Investigate the Plausibility of a
Regression Model Before Basing Inferences on It.
In "Single x" Contexts One Should Plot y versus x Looking for a Trend
Consistent With the Fitted Model and for Constant Spread Around
That Trend.
In General, "Residuals"
ei = yi ybi
Should Be "Patternless" and "Normal-looking."
Common Practice is to Normal-Plot and Plot Against All Predictors
(and ybi and Other Potential Predictors) "Standardized" Residuals
1
0
e
ei = i
SEei
Vardeman (ISU Stat and IMSE)
B
B
B= s
B
@
s 1
ei
1
n
Stat 231 Summary
x )2
∑ni=1 (xi x )2
(xi
C
C
in SLR C
C
A
September 6, 2011
44 / 58
Day 43-Practical Considerations 2
For c Di¤erent (Sets of) "x Conditions" in the Data Let
SScond j = (ncond j
2
1) scond
j
And De…ne (A "Pure Error" Sum of Squares)
c
SSPE =
∑ SScond j
j =1
With Degrees of Freedom n c = ∑cj=1 (ncond j
"Lack of Fit" Sum of Squares) Is
SSLoF = SSE
1) . Then (a
SSPE
With Degrees of Freedom
d.f .LoF = error d.f .
(n
c)
Then H0 :the …tted model is appropriate Can Be Tested Using
SSLoF /d.f .LoF
F =
SSPE / (n c )
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
45 / 58
Day 44-SLR Practical Considerations 3/MLR Model
Sometimes, Replacing y With Some Function of y (Like, e.g.,
y 0 = ln y ) and/or x’s With Some Function(s) Thereof Can Make The
Simple Technology of Regression Analysis Applicable to the Analysis
of a Data Set.
The Multiple Linear Regression Model is
yi = β0 + β1 x1i + β2 x2i +
+ βk xki + ei
Where the ei Are Independent Normal With Mean 0 and Standard
Deviation σ.
Least Squares (e.g. Implemented in JMP) Can Be Used to Fit
yb = b0 + b1 x1 + b2 x2 +
+ bk xk
(That is, Estimate the β’s). The Corresponding Estimate of σ Is
s
2
∑ (yi ybi )
s=
n (k + 1)
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
46 / 58
Day 45-MLR R-Squared/Overall (Model Utility) F Test
In MLR as in SLR,
SSTot = (n
1) sy2 =
n
∑ (yi
y )2 ,
n
SSE =
i =1
∑ (yi
i =1
= SSTot SSE
SSR
and R 2 =
SSTot
The Basic ANOVA Table for MLR Is
and SSR
Source
Regression
Error
Total
SS
SSR
SSE
SSTot
ANOVA Table (for MLR)
df
MS
k
MSR = SSR/k
n k 1 MSE = SSE /(n
n 1
Which Organizes an F Test of H0 :β1 = β2 =
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
ybi )2
F
F =
k
MSR
MSE
1)
= βk = 0
September 6, 2011
47 / 58
Day 46-MLR Partial F Tests/Fitted Coe¢ cients
In the (Full) MLR Model
y = β0 + β1 x1 + β2 x2 +
+ βk xk + e
For l < k, The Hypothesis H0 :βl +1 =
= βk = 0, Is That the Full
Model is Not Clearly Better Than the Reduced Model
y = β0 + β1 x1 + β2 x2 +
+ βl xl + e
This Can Be Tested Using
F =
(SSR (full) SSR (reduced)) / (k
SSE (full) / (n k 1)
l)
With an Fk l ,n k 1 Reference Distribution
The Fitted Coe¢ cient bl Is Normal With Mean βl And Standard
Deviation
σ (a complicated function of the values of the predictors xli )
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
48 / 58
Day 47-Con…dence and Prediction Limits in MLR
The Fitted Value yb = b0 + b1 x1 + b2 x2 +
+ bk xk Is Normal With
+ βk xk And Standard
Mean µy jx1 ,...,xk = β0 + β1 x1 + β2 x2 +
Deviation
σ (a complicated function of the values of the predictors xli )
Replacing σ with s In the Previous Two Standard Deviations
Produces Standard Errors SEbl and SEyb That Are Obtained From JMP
(NOT "By Hand")
Con…dence and Prediction Limits Are Then
s
s
n k 1
n k 1
and s
s
for σ
2
χupper
χ2lower
bl
t SEbl
yb
t SEyb for µy jx1 ,...,xk
q
s 2 + (SEyb )2 for ynew at (x1 , . . . , xk )
t
yb
Vardeman (ISU Stat and IMSE)
for βl
Stat 231 Summary
September 6, 2011
49 / 58
Day 48-Practical Considerations
Model Checking Involves Residual Plots (Residuals are ei = yi ybi
and Standardized Residuals are ei = ei /SEei for SEei =
σ (a complicated function of the values of the predictors xli )), Lack
of Fit Tests, and Examination of the "PreSS" Statistic. For yc
(i ) A
Fitted Value For the ith Case Obtained Not Using the Case in the
Fitting
n
PRESS =
∑ (yi
i =1
2
yc
(i ) )
n
∑ (yi
i =1
ybi )2 = SSE
(Ideally, PRESS Is Not Much Larger Than SSE )
Extrapolation is a Potentially Big Issue in MLR
Transformations Extend the Potential Applications of MLR
Variable/Model Selection in MLR Involves Balancing "Good Fit"
versus a Small Number of Predictors Variables
Formal Tools are Partial F Tests and Tests for Lack of Fit
Examination of R 2 , MSE , and Cp For "All Possible Regressions" Is A
More Flexible Informal Approach
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
50 / 58
Day 49-Practical Considerations-Model Selection
Considering Submodels (Reduced Models)
y = β0 + β1 x1 + β2 x2 +
+ βp xp + e
of a (Full) Model
y = β0 + β1 x1 +
+ βp xp + βp +1 xp +1 +
+ βk xk + e
Assumed to Produce Correct Values of The Means µyi ,
Cp = ( n
k
1)
SSEp
SSEk
+ 2 (p + 1)
n
(Under the Full Model) Estimates a Quantity that is
p+1+
a positive measure of how badly the
reduced model does at …tting the µyi
So, Simple (Small p) Models With Big R 2 , Small MSE , and
Cp p + 1 Are Desired
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
51 / 58
Day 50-Practical Considerations-Model
Selection/Multicollinearity
JMP Fit Model "Stepwise," Lack of Fit, and PRESS
The Complication of Multicollinearity Arises in MLR When One or
More of the Predictors is Nearly a Linear Combination of Others of
the Predictors (and Is Therefore Essentially Redundant in Practical
Terms). When This Occurs (Besides There Being Technical Problems
Associated With Solution of the Least Squares Fitting Problem):
While Good Prediction For Cases Like Those in the Data Set May Be
Possible, Extrapolation Is Extremely Dangerous, and
Assessment of Individual Importance of Particular Predictors Is Often
Impossible. (This Produces Big Standard Errors for Individual
Coe¢ cients, and Often Individual bl ’s That Make No Sense in the
Subject-Matter Application.)
Multicollinearity Can Be Prevented If One Gets to Choose
(x1i , x2i , . . . , xki ) Combinations (By Making Predictors Uncorrelated).
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
52 / 58
Day 51-Multicollinearity
With
e (j ) (yi ) = the ith y residual regressing
on all predictor variables except xj
e
(j )
and
(xji ) = the ith xj residual regressing
on all predictor variables except xj
JMP Plots Accompanying "E¤ect Tests" in Fit Model Are Plots Of
e (j ) (yi ) + y
versus e (j ) (xji ) + x j
So Small Spread In Horizontal Coordinates of Plotted Points Indicates
Multicollinearity.
When Predictors Are Uncorrelated, Regression Sums of Squares
"Add" and Fitted Coe¢ cients bl Are The Same For All Models
Including xl .
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
53 / 58
Day 52-Multicollinearity/The One Way Normal Model
Multicollinearity Means That One Only Has (x1i , x2i , . . . , xki )
Essentially In Some Lower-Dimensional (Than k) Subspace of
k-Dimensional Space and Thus Can Hope To Reliably Predict Only
There.
One-Way Analyses Are "r -Sample" Analyses (Not Unlike the
2-Sample Analyses of Devore Ch 9). They Are Based On A Model
For
yij = jth observation in the ith sample
Of The Form
yij = µi + eij
For The eij Independent Normal Random Variables With Mean 0 and
Standard Deviation σ. This Is "Samples From r Normal Populations
With Possibly Di¤erent Means But A Common Standard Deviation."
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
54 / 58
Day 53-Inference in the One Way Normal Model
A Single Number Estimate of σ Is
s
(n1 1) s12 + (n2 1) s22 +
spooled = sP =
( n1 1 ) + ( n2 1 ) +
+ (nr 1) sr2
+ ( nr 1 )
And Con…dence Limits For σ Are
s
s
n r
n r
sP
and sP
2
χupper
χ2lower
In The Context of Lack of Fit In Regression, (SSPE /d.f. PE) = sP2 .
For Population and Corresponding Sample Linear Combinations of r
Means
L = c1 µ +
+ cr µ and b
L = c1 y 1 +
+ cr y r
1
r
Con…dence Limits For L Are
Vardeman (ISU Stat and IMSE)
b
L
tsP
s
c12
+
n1
Stat 231 Summary
+
cr2
nr
September 6, 2011
55 / 58
Day 54-One Way ANOVA and Model Checking
In The One Way Context
SSE
=
∑ (yij
y i ) 2 = ( n1
1) s12 +
+ ( nr
1) sr2
and
i ,j
r
SSTr
= SSTot
SSE =
∑ ni ( y i
y )2
i =1
And The Hypothesis H0 :µ1 = µ2 =
F =
= µr May Be Tested Using
MSTr
SSTr / (r
=
MSE
SSE / (n
1)
r)
And An Fr 1,n r Reference Distribution.
One Way Residuals and Standardized Residuals Are (Respectively)
eij
eij = yij y i and eij = r
ni 1
sP
ni
And Are Used In Model Checking Exactly As In Regression.
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
56 / 58
Day 55-Statistics and Measurement 1
Measurand x
Measurement y
Measurement Error ε = y
x (So y = x + ε)
Standard Modeling Is That ε Is Normal With Mean β (Measurement
Bias) And Standard Deviation σ
Then For Independent Measurements y1 , y2 , . . . , yn of a Fixed x
sy
t p Estimates x + β (Measurand Plus Bias)
n
s
s
n 1
n 1
and sy
Are For σ (If Gauge And Operator
Limits sy
2
χUpper
χ2Lower
Are Fixed, This Is A Repeatability Std Dev ... If Each yi Is From A
Di¤erent Operator This Is an R&R Std Dev)
y
If x Varies Independent of ε, The Situation is More Complex and
Modeling of Multiple Measurement Depends on Exactly
pHow Data
Are Taken ... For a Single y , Ey = µx + β And σy = σ2x + σ2
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
57 / 58
Day 56-Statistics and Measurement 2
So if Each of y1 , y2 , . . . , yn Has a Di¤erent Measurand
sy
y t p Estimates µx + β (Average Measurand Plus Bias)
n
s
p
n 1
n 1
Limits sy
and sy
Are For σ2x + σ2 (A
2
2
χUpper
χLower
Combination of Measurand Variability and Measurement Variability)
s
Two Important Applications of This Are Where
Di¤erent x’s Represent The Truth About Di¤erent Items, So σx
Measures Process Variability
Di¤erent x’s Represent Di¤erent Operator-Speci…c Biases, So σx
Measures Reproducibility Variability
Where a Data Set Has r Measurands and m Measurements Per
Measurand, One Way ANOVA Can Help Separate σ2x And σ2
sr
P (And Associated Con…dence Limits) Estimate σ
1 MSTr
max 0, m
MSE ) (And Limits Provided By JMP If You
(
Know How To Ask) Estimate σx
Vardeman (ISU Stat and IMSE)
Stat 231 Summary
September 6, 2011
58 / 58