Statistics 350
Lecture 24
Today
• Last Day: Exam
• Today: Start Chapter 9
Model Building Process
•
Read 9.1-9.2…important ideas
•
Four Steps to Model Building:
1. Data collection and preparation:
• What sort of data is being collect…source of data? What sort of study
has been conducted?
• Response and explanatory variables?
• Much of the work Masters statisticians do involves data
preparation…look for proablems
Model Building Process
2. Reduction of number of explanatory variables:
• Researchers are wise to gather all they can, since any unmeasured
phenomena get lumped into random error.
•
Key things are:
• Need number of variables in final model to be large enough so that
your model provides an adequate approximation to reality, but small
enough to be practical for use
• Do not omit anything that the researcher considers vital just because
it is highly correlated with another variable
Model Building Process
3. Model refinement and selection:
• Once step 2 is done, then it is time to use techniques learned in Chapters
2-7 (transformations, interactions, plots, tests, …)
• Try to develop one or more plausible models that could potentially be
considered as final answers
Model Building Process
4. Model validation:
•
Not always possible, but it's best not to claim to have discovered a new
scientific principle before you try it out on an independent data set
Example
•
Investigators studied physical characteristics and ability in 13 football punters
•
Each volunteer punted a football ten times
•
The investigators recorded the average distance for the ten punts, in feet
•
In addition, the investigators recorded five measures of strength and flexibility for
each punter: right leg strength (pounds), left leg strength (pounds), right hamstring
muscle flexibility (degrees), left hamstring muscle flexibility (degrees), and overall leg
strength (foot-pounds)
•
From the study "The relationship between selected physical performance variables and
football punting ability" by the Department of Health, Physical Education and
Recreation at the Virginia Polytechnic Institute and State University, 1983
Example
• Variables:
•
•
•
•
•
•
Y: Distance traveled in feet
X1: Right leg strength in pounds
X2: Left leg strength in pounds
X3: Right leg flexibility in degrees
X4: Left leg flexibility in degrees
X5: Overall leg strength in pounds
Example
Example
Model Summary
Model
1
R
.902a
R Square
.814
Adjusted
R Square
.682
•
When we looked at this example in
lecture 22, we saw that the regression
equation was significant, but that the
individual t-tests identified none of the
variables as important
•
Why?
•
We then went through a process of
selecting variable to include in the
model…seemed a bit ad-hoc
•
Chapter 9 deals with the problem of
model selection
Std. Error of
the Estimate
14.64982
a. Predictors: (Constant), X5, X4, X2, X1, X3
ANOVAb
Model
1
Regres sion
Residual
Total
Sum of
Squares
6590.987
1502.321
8093.308
df
5
7
12
Mean Square
1318.197
214.617
F
6.142
Sig.
.017a
a. Predic tors: (Constant), X5, X4, X2, X1, X3
b. Dependent Variable: Y
a
Coeffi cients
Model
1
(Const ant)
X1
X2
X3
X4
X5
Unstandardized
Coeffic ients
B
St d. Error
-29.580
65.700
.279
.456
.070
.484
1.241
1.449
-.395
.745
.224
.131
a. Dependent Variable: Y
St andardiz ed
Coeffic ients
Beta
.245
.062
.373
-.131
.412
t
-.450
.611
.144
.857
-.531
1.714
Sig.
.666
.561
.890
.420
.612
.130
Criteria for Model Selection
• So how do we deal with Step 2 of our procedure?
• Would like tests that identify some variables as unimportant
• Suppose have P-1 potential explanatory variables and fir all possible sub-sets
of variables
• There are
such models
• So, in the football example, there are
possible subsets
Criteria for Model Selection
• Goal is to choose a “good” set from these variables that explain the data well
• So, may wish to choose a few possible models that appear “good”
• What is good?
• There exist criteria that assess the relative goodness of models
Criteria for Model Selection
• Criteria: R2p
•
•
•
•
Is the coefficient of determination (SSR/SSTO)
p represents
Good models have
What happens if you add more variables?
• How to use?
Criteria for Model Selection
• Criteria: R2a
Criteria for Model Selection
• Mallows Cp
Criteria for Model Selection
• AIC
Criteria for Model Selection
• PRESS: