Chapter 16 : Single-Factor Studies
Lecture 6
February 22, 2007
Psychology 791
Lecture 6
Psychology 791
Today’s Lecture
•
Last class we discussed research design.
•
Today we will talk about analysis of single factor studies.
•
Notice the buzz word "factor" - meaning an explanatory
variable to be studied.
•
We are going to concentrate on studies with one
explanatory variable.
•
Thinking about our beer example, our design would include
one factor, the amount of beer that would influence our test
score.
Overview
• Today’s Lecture
• Research Design
ANOVA Model
Cell Means Model
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Research Design
•
Overview
• Today’s Lecture
• Research Design
ANOVA Model
Our basic research design will include:
◦
One factor (either exploratory or experimental).
◦
A number of categorial levels of the factor (denoted as r
levels).
◦
Randomization of the treatment levels to each of the
subjects.
Cell Means Model
Model Specs
F-test
Wrapping Up
•
Note that we can use both exploratory and experimental
factors for this analysis.
◦
•
Lecture 6
The methods used for statistical analysis are the same.
One approach is to construct a linear model with r-1
indicator variables as predictors.
Psychology 791
ANOVA Model
Lecture 6
Psychology 791
Linear Model
•
Recall our linear model with r-1 predictors:
Overview
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
Yij = β0 + β1 Xij1 + β2 Xij2 + ... + βr−1 Xij,r−1 + ǫij
where:
Cell Means Model
Model Specs
Xij1 =
(
1 If observation is in treatment 1
0 Otherwise
Xij2 =
(
1 If observation is in treatment 2
0 Otherwise
F-test
Wrapping Up
..
.
Xij,r−1 =
Lecture 6
(
1
0
If observation is in treatment r − 1
Otherwise
Psychology 791
Linear Model
Yij = β0 + β1 Xij1 + β2 Xij2 + ... + βr−1 Xij,r−1 + ǫij
Overview
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
•
You will notice it looks exactly like the GLM that we looked
at last semester with regression.
•
Because the predictors are indicator variables, this model
is sometimes referred to an analysis of variance model.
•
We called this dummy coding.
Cell Means Model
Model Specs
F-test
Wrapping Up
Lecture 6
◦
It’s so easy, a caveman can do it.
Psychology 791
Today’s Example: Drug Therapy
•
A hospital research staff wished to determine the best
dosage level for a standard type of drug therapy to treat a
medical condition.
•
In order to compare the effectiveness of the three dosage
levels, 30 patients with the medical problem were recruited
to participate in a pilot study.
•
Each patient was randomly assigned to one of the three
drug dosage levels.
•
Randomization was performed in such a way that an equal
number of patients ended up being evaluated for each drug
dosage level, i.e., exactly 10 patients studied in each drug
dosage level group.
Overview
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
Cell Means Model
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Drug Therapy: Google’s Images
KUMC
Paul Richardson
Lecture 6
Psychology 791
Design Questions
•
What is the research design?
•
What is the factor?
•
How many levels of that factor?
•
How many subjects are in this study?
Overview
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
Cell Means Model
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Design Matrix
•
Everyone remember the idea of a design matrix?
•
What would the design matrix be for this study?
Overview
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
Cell Means Model
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
ANOVA model
•
Write out the ANOVA model for this study...
Overview
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
Cell Means Model
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Distinction between ANOVA and Regression
•
If you look at the ANOVA model, it looks a lot like the
regression model for categorical variables that we studied
last semester.
•
In fact, if you run a regression model with categorical
predictors or an ANOVA on this data set, you will obtain
identical results.
•
So, why the distinction?
•
To make a long story short - the setup of the model is
slightly different.
•
I will show you the slight distinction through the
assumptions.
Overview
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
Cell Means Model
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Refresher: Assumptions of Regression
•
In regression, the assumption of normality was placed on
the error terms:
ǫi ∼ N (0, σ 2 )
•
From this distributional assumption, if you might also recall,
we talked briefly about that the distribution at each point on
the line was normal (fig 16.1 pg 680).
Overview
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
Cell Means Model
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Move to ANOVA Assumptions
•
The assumptions really follow along the same lines, but
instead of a continuous line, we have a discrete number of
points.
•
At each point, then, we have a normal probability
distribution.
•
The ANOVA model assumes:
Overview
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
Cell Means Model
Model Specs
F-test
◦
Each conditional distribution is normal.
◦
Each conditional distribution has the same variance
(homoscedasticity).
◦
The responses for each factor level are random
selections from the corresponding conditional
distribution and are independent of the responses of any
other factor level.
Wrapping Up
Lecture 6
Psychology 791
Goal of Analysis
•
Thinking about it piecewise:
Overview
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
◦
We have differing factor levels.
◦
We have a distribution of responses at each factor level.
◦
We have r distributions - one for each factor level.
Cell Means Model
Model Specs
•
The ANOVA model - a model that compares the r
distributions
•
Because they are all normally distributed and they all have
equal variance, the only thing that can differ is their means.
•
So the ANOVA model compares the means for each factor
level.
F-test
Wrapping Up
Lecture 6
Psychology 791
ANOVA Analysis
•
Overview
Analysis of these probability distributions usually has two
steps:
ANOVA Model
• Today’s Example
• Questions...
• ANOVA v. Regression
• ANOVA Goal
• ANOVA Steps
Cell Means Model
Model Specs
◦
Determine whether or not the factor level means are the
same (Chapter 16, our F-test).
◦
If the factor level means differ, examine how they differ
and what it means (Chapter 17, referred to as post-hoc
test).
F-test
Wrapping Up
Lecture 6
•
First, let’s talk about testing "mean differences."
Psychology 791
Cell Means Model
Lecture 6
Psychology 791
Representation: Cell Means Model
•
There are mainly two types of design matrices used in
ANOVA, the first falling under the cell means model.
•
Notation for Model:
Overview
ANOVA Model
Cell Means Model
• The Model
• Important features of
Model
• Another Question...
• Equivalent Models
• Testing
◦
r - the number of levels of the factor in the study.
◦
i - any of the levels of the factor (i = 1,2,...,r).
◦
ni - number of cases in the ith factor.
• Interpretation
Model Specs
F-test
Wrapping Up
◦
nT - the total number of cases:nT =
r
X
ni
i=1
◦
Lecture 6
j is now the subject or observation number.
Psychology 791
Representation: Cell Means Model
•
Note a major distinction in notation: j refers to the subject
(or case or trial).
•
In Kutner et al. notation for ANOVA models, the last
subscript will always refer to the subject (or case or trial).
•
In this model, we have two subscripts, i and j, so j refers
to the subject while i refers to the factor level.
Overview
ANOVA Model
Cell Means Model
• The Model
• Important features of
Model
• Another Question...
• Equivalent Models
• Testing
• Interpretation
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Cell Means Model
•
Our ANOVA model can now be stated as follows:
Overview
Yij = µi + ǫij
ANOVA Model
Cell Means Model
where:
• The Model
• Important features of
Model
• Another Question...
• Equivalent Models
• Testing
• Interpretation
•
Yij is the response variable in the jth observation for the ith
factor level.
•
µi are the parameters of the model: factor level means.
•
ǫij are independent N (0, σ 2 ).
•
i = 1, . . . , r.
•
j = 1, . . . , ni .
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Important features of Model
•
The observed value Y in the jth observation for the ith
factor level is the sum of two components:
a. The constant (or factor level mean µi ).
b. Random error (ǫij ).
•
Because E(ǫij ) = 0, it follows that E(Yij ) = µi (all responses
at each level have the same expectation, the mean).
Overview
ANOVA Model
Cell Means Model
• The Model
• Important features of
Model
• Another Question...
• Equivalent Models
• Testing
• Interpretation
Model Specs
•
All observations have the same variance, σ 2 .
•
Since ǫij is normally distributed, so is each Yij .
•
Error terms are independent.
•
Yij are independent N (µi , σ 2 ).
•
The book refers to this model as ANOVA Model I.
F-test
Wrapping Up
Lecture 6
Psychology 791
Linear?
•
We have restated our ANOVA model in terms of our cell
means, does this still fall into line as a linear model?
•
Reminder of the form: Y = X β + ǫ
Overview
ANOVA Model
Cell Means Model
• The Model
• Important features of
Model
• Another Question...
• Equivalent Models
• Testing
• Interpretation
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Distinction between two models
•
Do you see the tricky distinction between the two models?
•
The difference is in the parameters.
•
Although the results will be the same, we define these two
things differently:
Overview
ANOVA Model
Cell Means Model
• The Model
• Important features of
Model
• Another Question...
• Equivalent Models
• Testing
• Interpretation
Model Specs
◦
The Design Matrix - X
◦
the Matrix of parameters - β
F-test
Wrapping Up
Lecture 6
•
Depending upon how we parameterize the model, the
meaning of the parameters will be different.
Psychology 791
Write out the ANOVA Model in Matrix Form
Overview
ANOVA Model
Cell Means Model
• The Model
• Important features of
Model
• Another Question...
• Equivalent Models
• Testing
• Interpretation
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Write out the Cell Means Model in Matrix Form
Overview
ANOVA Model
Cell Means Model
• The Model
• Important features of
Model
• Another Question...
• Equivalent Models
• Testing
• Interpretation
Model Specs
F-test
Wrapping Up
Lecture 6
Psychology 791
Interpretation of Factor Level Means
•
Observational Data:
Overview
◦
ANOVA Model
Cell Means Model
• The Model
• Important features of
Model
• Another Question...
• Equivalent Models
• Testing
•
Experimental Data:
◦
• Interpretation
Model Specs
The factor level mean represents the observed means at
each of the factor levels.
The factor level mean represents the mean response
that would be obtained if the treatment level was applied
to the entire population.
F-test
Wrapping Up
Lecture 6
•
The distinction between these is made in the meaning of
the factor level means, not the calculation.
Psychology 791
Model Specs
Lecture 6
Psychology 791
Estimation
•
Least squares estimation is used.
Overview
ANOVA Model
◦
Maximum likelihood will lead to same results.
Cell Means Model
Model Specs
• Estimation
• Residuals
• ANOVA
• Sum of Squares
• Degrees of Freedom
• Mean Squares
• Resulting ANOVA table
F-test
Wrapping Up
Lecture 6
Psychology 791
Residuals
•
Residuals for this model should look familiar to regression:
Overview
eij = Yij − Ŷij = Yij − Y i·
ANOVA Model
Cell Means Model
Model Specs
• Estimation
• Residuals
• ANOVA
• Sum of Squares
• Degrees of Freedom
• Mean Squares
• Resulting ANOVA table
F-test
•
Note the dots in the notation: anytime you see a dot, that
means it is over all the values of that particular subscript,
so the i· means it is the mean for i over all the values of j.
•
Note if we have two dots, then it would be the overall mean,
mean over all i and all j.
Wrapping Up
Lecture 6
Psychology 791
ANOVA
•
So, here is the good stuff, right, the Analysis of Variance.
•
We are going to partition the total variance.
•
Total variance is defined as: Yij − Y··
•
let’s break it apart:
Overview
ANOVA Model
Cell Means Model
Model Specs
• Estimation
• Residuals
• ANOVA
• Sum of Squares
• Degrees of Freedom
• Mean Squares
• Resulting ANOVA table
Yij − Y ·· = Y i· − Y ·· + Yij − Y i·
F-test
Wrapping Up
Lecture 6
•
First two is total deviation.
•
Second two is deviation of factor level mean.
•
Third is deviation around estimation factor mean (residual).
Psychology 791
Sum of Squares
•
Overview
ANOVA Model
Cell Means Model
Model Specs
• Estimation
• Residuals
• ANOVA
• Sum of Squares
• Degrees of Freedom
• Mean Squares
• Resulting ANOVA table
F-test
•
•
SSTR =
P
2
n
(Y
−
Y
)
i
i·
··
i
P P
2
Y
)
(Y
−
i·
ij
j
•
SSE =
•
Or: SSTO = SSTR + SSE
Wrapping Up
Lecture 6
If we take that formula, slap on some summations and add
a squared component, we end up with a partition of the
total variance, or our Sum of Squares (from our ANOVA
table)
P P
SSTO = i j (Yij − Y·· )2
i
Psychology 791
Degrees of Freedom
•
We also need to partition our Degrees of Freedom to two
components.
•
SSTO has nT − 1 df.
•
SSTR has r - 1 df.
•
SSE has nT − r df.
•
Or: dfSST O = dfSST R + df SSE.
•
Or: nT − 1 = r - 1 + nT − r.
Overview
ANOVA Model
Cell Means Model
Model Specs
• Estimation
• Residuals
• ANOVA
• Sum of Squares
• Degrees of Freedom
• Mean Squares
• Resulting ANOVA table
F-test
Wrapping Up
Lecture 6
Psychology 791
Mean Squares
•
Overview
To compute our Mean Squares, we take our Sum of
Squares and divide by the df.
ANOVA Model
Cell Means Model
Model Specs
• Estimation
• Residuals
• ANOVA
• Sum of Squares
• Degrees of Freedom
• Mean Squares
• Resulting ANOVA table
F-test
Wrapping Up
Lecture 6
Psychology 791
Resulting ANOVA table
•
See the outline on page 694
Overview
ANOVA Model
Cell Means Model
Model Specs
• Estimation
• Residuals
• ANOVA
• Sum of Squares
• Degrees of Freedom
• Mean Squares
• Resulting ANOVA table
F-test
Wrapping Up
Lecture 6
Psychology 791
F-test
Lecture 6
Psychology 791
F-test
•
We have the model.
•
We have the means.
•
We’ve partitioned the variance and df (ANOVA table).
•
How do we determine mean differences?
•
We perform an F test.
Overview
ANOVA Model
Cell Means Model
Model Specs
F-test
• F-test
• Hypothesis
• Test Statistic
Wrapping Up
Lecture 6
Psychology 791
Hypothesis
•
Our null hypothesis is that there are no mean differences,
trying to test for differences
•
H0 : µ1 = µ2 = ... = µr
•
Ha : not all µi are equal
•
We test that at least 2 means are different, but we have no
idea which ones they are.
•
Also note that the alternative is not that they all differ – not
all may differ, maybe one is different from the rest
Overview
ANOVA Model
Cell Means Model
Model Specs
F-test
• F-test
• Hypothesis
• Test Statistic
Wrapping Up
Lecture 6
Psychology 791
Test Statistic
•
This should look familiar.
•
Your F-test is a ratio of the MSTR and the MSE
•
F =
•
The F has r - 1, nT − r df.
•
Another note: The DF of the F statistic are the DF for the
top followed by the DF for the bottom.
•
Top is MSTR (DF = r - 1) : Bottom is MSE (DF = nT − r).
•
Reject if p < α (usually 0.05).
Overview
ANOVA Model
Cell Means Model
Model Specs
F-test
• F-test
• Hypothesis
• Test Statistic
Wrapping Up
Lecture 6
M ST R
M SE .
Psychology 791
Final Thought
•
Hopefully today you have
learned a bit about the
ANOVA model, its
formulations, its purpose
•
In lab you will learn how to
run a one-factor (one-way)
ANOVA model in SAS and
look at the output.
•
Thursday we will talk about the model in more detail
(moving on to determining factor level means and
comparing them).
•
The other ANOVA model will be discussed next time.
Overview
ANOVA Model
Cell Means Model
Model Specs
F-test
Wrapping Up
• Final Thought
• Next Class
Lecture 6
Psychology 791
Next Time
•
More Chapter 16.
•
Expected Mean Squares.
•
The factor effects model.
Overview
ANOVA Model
Cell Means Model
Model Specs
F-test
Wrapping Up
• Final Thought
• Next Class
Lecture 6
Psychology 791