Solving One-way ANOVA Problems as a General Linear Model
1 of 26
Homework problems are multiple answer rather than multiple choice. The format for multiple
answer questions is shown in the example below.
The directions for the problems instruct you to mark the check boxes for all of the statements
that are true. One or more answers must be marked for each problem. Full or partial credit is
computed for each question. To receive full credit, you must mark all of the correct answers
and not mark any of the incorrect answers. Partial credit is computed by summing the points
for each correct response and subtracting points for each incorrect answer. If the computation
for partial credit results in a negative number, zero credit is assigned.
Level of Measurement Requirement and Sample Size Requirement
In a one-way analysis of variance, the level of measurement for the independent variable can
be any level that defines groups (dichotomous, nominal, ordinal, or grouped interval) and the
dependent variable is required to be interval level. If the dependent variable is ordinal level,
we will follow the common convention of treating ordinal variables as interval level, but we
should consider noting the use of an ordinal variable as a limitation to our findings.
I have imposed a minimum sample size requirement of 5 cases per category of the independent
variable for these problems. This is a convention for these problems and is based on the
needed to have a reasonably stable mean for each cell when analyzing observational data.
If the sample size requirement (along with the level of measurement requirement) is satisfied,
the check box “The level of measurement requirement and the sample size requirement are
satisfied” should be marked. If the sample size requirement is not satisfied, the correct answer
to the problem is “Inappropriate application of the statistic.” All other answers should be
unmarked when the answer is “Inappropriate application of the statistic.”
2 of 26
The Assumption of Normality
Analysis of variance assumes that the dependent variable is normally distributed, but there is
general consensus that violations of this assumption do not seriously affect the probabilities
needed for statistical decision making, especially when the number of cases in each cell are
equal.
Our problems evaluate normality based on the criteria that the skewness and kurtosis of the
dependent variable fall within the range from -1.0 to +1.0. If the dependent variable satisfies
these criteria for skewness and kurtosis, the check box “The skewness and kurtosis of income
satisfy the assumption of normality” should be marked. If the criteria for normality are not
satisfied, the check box should remain unmarked and we should consider including a statement
about the violation of this assumption in the discussion of our results.
In these problems we will not test transformations or consider removing outliers to improve the
normality of the variable.
The Assumption of Homogeneity of Variance
Analysis of variance assumes that the variance of the dependent variable is homogeneous
across all of the cells formed by the factors (independent variable). We will use the
significance of Levene’s test for equality of variance as our criteria for satisfying the
assumption. SPSS computes the Levene test as part of the output for general linear models.
Levene’s test is a diagnostic statistic that tests the null hypothesis that the variance is
homogeneous or equal across all cells. The desired outcome, and support for satisfying the
assumption, is to fail to reject the null hypothesis.
If the significance for the Levene test is greater that the alpha for diagnostic statistics, we fail
to reject the null hypothesis and the check box “The assumption of homogeneity of variance is
supported by Levene's test for equality of variances” should be marked. If the criterion for
homogeneity of variance is not satisfied, the check box should remain unmarked.
Analysis of variance is robust to violations of the assumption of homogeneity of variances
provided the ratio of the largest group variance is not more than 3 time the smallest group
variance.
If we violate this assumption, but the ratio is less than or equal to 3.0, we should consider
including a statement about the violation of this assumption in the discussion of our results.
If we violate this assumption and the ratio of largest to smallest variance is greater than 3.0,
we should not use a one-way analysis of variance for the data for these variables and we mark
the check box, “Inappropriate application of the statistic.” The check boxes for level of
measurement and sample size, and the assumption of normality should remain marked if they
were previously satisfied, even if the problem is found to be an inappropriate application of a
statistic because of heterogeneity.
Interpretation of the Relationship
The statement of the relationship between the dependent and independent variable is a
statement that the different categories of the independent variable are linked to different
average scored on the dependent variable. The statement is correct if the relationship is
statistically significant in the table of “Tests of Between-Subjects Effects.” Since there is only
one independent variable in this analysis, the table entries for the “Corrected Model” and the
variable will be identical.
3 of 26
SPSS computes partial eta squared as a measure of effect. We characterize it as trivial, small,
moderate, or large, applying Cohen's criteria for effect size (less than .01 = trivial; .01 up to
0.06 = small; .06 up to .14 = moderate; .14 or greater = large). Effect size should only be
interpreted if the relationship is statistically significant.
Determination of the correctness of statements about specific relationships is a two stage
process. First, it is required that the relationship be statistically significant and the strength of
the relationship be correctly described. Second, it is required that the statement be a correct
comparison of the direction of the means, based on either a direct comparison of the group
means when the factor contains two categories, or a post-hoc test when the factor includes
three or more categories.
We will use the Bonferroni test for multiple comparisons for these and future problems rather
than the Tukey HSD or Games-Howell post test. The Bonferroni test is a set of t-tests of the
difference for all possible pairs with an adjusted alpha for each t-test so that the total alpha
for all comparisons does not exceed the desired level of significance.
For example, suppose that there are three categories for the independent variable and we set
alpha to 0.05. For three categories, there are three pairs to be compared in the t-test: group 1
vs. group 2, group 1 vs. group 3, and group 2 vs. group 3. To maintain the 0.05 error rate, we
should divide 0.05 by the 3 tests, and compared the p-value for each test to 0.05 / 3 = 0.017.
We would only report significant findings for the pairs where the p-value of the t-test less than
or equal to 0.017.
To save us the work of dividing alpha by the number of tests, SPSS adjusts the sigs or p-values
that it reports so that they can be compared directly with the alpha level of 0.05. Though I am
not sure how they compute the actual adjustment, I would roughly estimate it to be 3 times
the actual p-value for the test. If a t-test had an actual probability of 0.01 when three pairs
were compared, they would report a p-value of 0.03 (3 x 0.01). This multiplication can result in
unusual reported p-values (like p = 1.0, which implies absolute certainty). A p-value of 1.0
should be adjusted in our report to p > 0.99, just like we adjust reported p-values of 0.01 as p
< 0.01.
Inappropriate application of the statistic
We should not use one-way analysis of variance if we violate the level of measurement
requirement, the minimum sample size requirement, or violate the assumption of homogeneity
of variance with the ratio of largest to smallest group variance is larger than 3.0.
Solving Problems in SPSS
We will demonstrate the use of
SPSS to compute a one-way
analysis of variance with the
general linear model procedure
with this problem.
The introductory statement identifies
the variables for the analysis and the
significance levels to use. Note that
we use a more conservative alpha
(.01) for diagnostic statistics than we
do for the statistics that answer our
research questions.
Level of Measurement – 1
The first statement in the problem asks about
level of measurement and sample size. In a oneway analysis of variance, the level of
measurement for the independent variable can be
any level that defines groups (dichotomous,
nominal, ordinal, or grouped interval) and the
dependent variable is required to be interval level.
4 of 26
5 of 26
Level of Measurement - 2
To determine the level of measurement,
we examine the information about
variables in the SPSS data editor,
specifically the values and value labels.
"Subjective class identification" [class] is ordinal satisfying
the requirement for an independent variable. The
dependent variable "highest year of school completed"
[educ] is interval level satisfying the requirement for the
dependent variable.
Using Univariate General Linear Model for Descriptive Statistics - 1
Select General Linear
Model > Univariate from
the Analyze menu.
To check for compliance with sample
size requirements, we run the univariate
general linear model procedure. This
procedure will give us the correct
number of cases in each cell, taking into
account missing data for both of the
variables in the analysis.
Using Univariate General Linear Model for Descriptive Statistics - 2
First, move educ to
the Dependent
Variable text box.
Second, move class to
the Fixed Factor(s) list
box.
Third, click on the
Options button.
Using Univariate General Linear Model for Descriptive Statistics - 3
First, mark the check
boxes for Descriptive
statistics.
Second, since this is the
only output we need for now,
click on the Continue button.
6 of 26
Using Univariate General Linear Model for Descriptive Statistics – 4
Click on the OK
button to obtain
the output.
Descriptive Statistics from the Univariate General Linear Model - 1
The table of Descriptive
Statistics contains the number
of cases in each cell for the
combination of factors.
The smallest cell in the analysis
had 13 cases. The sample size
requirement of 5 or more cases
per cell is satisfied.
7 of 26
Marking the Statement for the Level of Measurement and Sample Size Requirement
Since we satisfied both the level
of measurement and the sample
size requirements for analysis of
covariance, we mark the first
checkbox for the problem.
The Assumption of Normality
The next statement in the problem focuses on
the assumption of normality, using the skewness
and kurtosis criteria that both statistical values
should be between -1.0 and +1.0.
8 of 26
Computing Skewness and Kurtosis to Test for Normality – 1
Skewness and kurtosis are
calculated in several procedures.
We will use Descriptive Statistics.
Select Descriptive Statistics >
Descriptives from the Analyze
menu.
Computing Skewness and Kurtosis to Test for Normality – 2
We add the variable whose
normality we are concerned about.
First, move the dependent
variable, educ, to the
Variable(s) list box.
Second, click on the
Options button to
specify the statistics
we want computed.
9 of 26
Computing Skewness and Kurtosis to Test for Normality – 3
Kurtosis and Skewness are
not selected by default, so
we mark their check boxes.
Second, click
on the Continue
button to close
the dialog box.
First, mark the check
boxes for Kurtosis and
Skewness.
Computing Skewness and Kurtosis to Test for Normality – 4
We have finished entering the
specifications we need for the
evaluation of normality.
Click on the OK
button to obtain
the output.
10 of 26
Evaluating the Assumption of Normality
"Highest year of school completed" [educ] did not
satisfy the criteria for a normal distribution. The
skewness of the distribution (-.137) was between
-1.0 and +1.0, but the kurtosis of the distribution
(1.246) fell outside the range from -1.0 to +1.0.
Though analysis of variance is robust to violations
of the assumption normality, we should consider
including the violation as a limitation of the
analysis.
Marking the Statement for the Assumption of Normality
Since the assumption of
normality is not satisfied, the
check box is not marked.
11 of 26
The Assumption of Homogeneity of Variance
The next statement in the problem
focuses on the assumption of
homogeneity of variance, which can be
computed with the Univariate General
Linear Model procedure.
Running the complete Univariate General Linear Model – 1
In the Dialog Recall
menu, select the
Univariate procedure.
12 of 26
13 of 26
Running the complete Univariate General Linear Model – 2
First, click on the
Options button to specify
additional output.
Running the complete Univariate General Linear Model - 3
First, mark the check
box for Homogeneity
tests to add the Levene
test to our output.
Second, mark the check box
for Estimates of effect size to
include partial eta squared in
the output.
14 of 26
Running the complete Univariate General Linear Model - 4
Move the variable class to the
Display Means for list box. This
will enable us to get estimated
means and the Bonferroni post
hoc test.
Running the complete Univariate General Linear Model - 5
First, mark the check
box Compare main
effects. This will
compute the post hoc
tests for the main
effects.
The Univariate procedure will
calculate all of the post hoc
tests that are available in the
one-way ANOVA procedure. I
am using this option because
we can continue this strategy
when we do more complex
analyses, like Factorial
ANOVA, ANCOVA, and
Repeated Measures ANOVA.
Second, select Bonferroni from
the Confidence interval adjustment
drop down men. This will hold the
error rate for our multiple
comparisons to the specified alpha
error rate.
15 of 26
Running the complete Univariate General Linear Model - 6
We have finished
selecting our
optional statistics.
Click on the Continue
button to close the
dialog box.
Running the complete Univariate General Linear Model - 7
We have finished entering
the specifications we need
for our analysis.
Having completed all of
the specifications, click on
the OK button to generate
the output.
Levene’s Test for Homogeneity of Variance - 1
The probability associated with Levene's test for
equality of variances (F(3, 264) = 2.85, p = .038)
is greater than the alpha for diagnostic tests
(0.01). The assumption of equal variances is
satisfied.
Had we not satisfied the assumption of
homogeneous variances, we would calculate the
ratio of largest squared variance to smallest
squared variance to determine whether or not we
could report the results from this analysis.
Marking the Statement for the Assumption of Homogeneity of Variance
Since the results of the Levene
test supported the assumption of
homogeneity of variance, we
mark the checkbox for the
statement.
16 of 26
The Relationship between the Dependent and Independent Variables
17 of 26
The next statement in the problem focuses
on the effect size or strength of the
relationship between the variables.
Significance and Strength of the Relationship – 1
Differences in average highest year of
school completed by subjective class
identification were statistically significant
(F(3, 264) = 4.966, p = .002, partial eta
squared = 0.05).
The null hypothesis that "the mean
highest year of school completed was
equal across all categories of subjective
class identification" is rejected.
Significance and Strength of the Relationship – 2
If the F-test for class had not been
statistically significant, we do not
interpret the effect size or the post
hoc tests.
Applying Cohen's criteria for effect size (less than .01 = trivial;
.01 up to 0.06 = small; .06 up to .14 = moderate; .14 or greater
= large), a partial eta square of .05 would be characterized as
explaining a small proportion of the variance. The statement that
"membership in categories defined by subjective class
identification accounts for a moderate amount of the differences in
average highest year of school completed" is not correct.
Marking the Statement for the Effect Size
Since the statement that "membership in
categories defined by subjective class
identification accounts for a moderate
amount of the differences in average
highest year of school completed" is not
correct, the check box is not marked.
Even though the statement about effect
size was not marked, the relationship was
statistically significant, so we continue with
the post hoc tests.
18 of 26
19 of 26
Interpreting the Post Hoc Tests - 1
The next three statements are possible
interpretation of the post hoc effects.
Each one should be verified independently
for significance in the Table of Pairwise
Comparisons. The means and standard
deviations are extracted from the Table of
Descriptive Statistics.
Interpreting the Post Hoc Tests - 2
The first statement was: "survey respondents who
said they belonged in the middle class (M=13.83,
SD=3.14) completed more years of school those
who said they belonged in the working class
(M=12.58, SD=2.50), and the difference was
statistically significant.”
The means and standard
deviations cited in the
statement are correct.
Interpreting the Post Hoc Tests - 3
The Bonferroni pairwise comparison
of the difference (1.25) for Middle
Class versus Working Class was
statistically significant (p = .005).
Interpreting the Post Hoc Tests – 4
For this statement, the means and standard
deviations are correct and the post hoc test
was statistically significant. The check box for
this statement is marked.
20 of 26
21 of 26
Interpreting the Post Hoc Tests – 5
The second statement was: Survey respondents
who said they belonged in the working class
(M=12.58, SD=2.50) completed more years of
school those who said they belonged in the upper
class (M=12.23, SD=2.95) and those who said they
belonged in the lower class (M=12.17, SD=2.85),
and all differences were statistically significant”
The means and standard
deviations cited in the
statement are correct.
Interpreting the Post Hoc Tests - 6
The Bonferroni pairwise comparison
of the difference (0.35) was not
statistically significant (p > 0.99).
NOTE: the probability in the table (1.00) is a
consequence of the adjustment in probabilities made by
SPSS to maintain the alpha error rate for the multiple
tests. Like adjusting p = 0.000 to p < .001, we should
adjust the statement to p > 0.99 so that our audience is
not misled into thinking that this is not an absolute
certainty.
22 of 26
Interpreting the Post Hoc Tests - 7
For this statement, the means and standard
deviations are correct, but the post hoc test
was not statistically significant. The check box
for this statement is not marked.
Interpreting the Post Hoc Tests - 8
The third statement was: " survey respondents who
said they belonged in the upper class (M=12.23,
SD=2.95) completed more years of school those
who said they belonged in the lower class
(M=12.17, SD=2.85), and the difference was
statistically significant.”
The means and standard
deviations cited in the
statement are correct.
23 of 26
Interpreting the Post Hoc Tests – 9
The Bonferroni pairwise comparison
of the difference (0.064) was not
statistically significant (p > 0.99).
Interpreting the Post Hoc Tests - 10
For this statement, the means and standard
deviations are correct, but the post hoc test
was not statistically significant. The check box
for this statement is not marked.
The problem
is complete.
24 of 26
The Problem Graded in BlackBoard
When this assignment was submitted,
BlackBoard indicated that all marked
answers were correct, and we received
full credit for the question.
Logic Diagram for One-Way Analysis of Variance Problems – 1
Level of measurement
and sample size ok?
No
25 of 26
Do not mark check box
Mark: Inappropriate
application of the statistic
Yes
Stop
Mark check box for
correct answer
Ordinal dv?
Yes
Assumption of normality
ok? (skewness and
kurtosis between +/-1)
No
Consider limitation in
discussion of findings
Do not mark check box
Consider limitation in
discussion of findings
Yes
Mark check box for
correct answer
Assumption of
homogeneity of variance
ok? (Levene Sig >
diagnostic alpha)
No
Do not mark check box
Yes
Ratio of largest group
variance to smallest
group variance ≤ 3
Mark check box for
correct answer
Yes
Mention violation in
discussion of findings
No
Mark: Inappropriate
application of the statistic
Stop
26 of 26
Logic Diagram for One-Way Analysis of Variance Problems – 2
Main effect statistically
significant?
(Sig < alpha)
No
Do not mark check boxes for
effect size or post hoc tests
Stop
Yes
Correct adjective used to
describe effect size,
applying Cohen’s scale?
(< .01 = trivial,
.01 up to .06=small,
.06 up to .14=moderate,
.14 or larger = large)
No
Do not mark check box for
effect size
No
Do not mark check box for
post hoc comparison
Yes
Post hoc comparisons
correctly interpreted?
Yes
Mark check box for
correct comparison