One-way ANOVAs Lab 6 1
One-Way ANOVA’s
We have already discussed the t-test. The t-test is used for comparing the means of two groups to
determine if there is a statistically significant difference. The t-test allows us to determine the probability
that we are making a Type I error (rejecting the null when it is true). If the probability is less than alpha
(generally set at 0.05) then we reject the null hypothesis (conclude that there is a difference) with at least
95% confidence that the difference between the two conditions is not due simply to chance.
T-tests are appropriate for making comparisons when
- your independent variable (IV) is discrete and your dependant variable (DV) is continuous.
- there is only one IV in your study (it is not a factorial Design)
- there are only two levels of the IV
One-way ANOVAs are appropriate when
- your independent variable is discrete and your dependant variable is continuous.
- there is only one IV in your study (it is not a factorial design)
- there are more than two levels of the IV.
One-way ANOVAs allow us to determine if there is a difference AMONG three or more conditions.
The procedure is done in two parts.
1) Determine if there is a great enough difference AMONG the three or more conditions of your study.
When making comparisons between three or more conditions, we are really looking to see if there is a
difference between at least 2 of the conditions. For Example, in a study conducted by a student in last
years BR class three Background Sound (BS) conditions were used. These were White noise, Ambient
Noise and Music. In each condition participants are given spatial problems to complete. The dependant
variable was the number of spatial problems they are able to solve correctly in the allotted time. We will
refer to these as Ravens’ Matrix Scores (RMS)
The student wanted to compare scores from each of the conditions to each other .
1) White Noise to Ambient noise.
2) White Noise to Music
3) Ambient noise to Music
He could do three t-tests to address each of these questions, however, for each t-test he conducts he is
adding an additional 5% probability of making a Type I error. Over these three comparisons, he would
have a .05 + .05 + .05 probability of making at least one Type I error. In other words, if he found
significant difference between two conditions, he would really be only 85% certain that this is real
difference and not simply due to chance. There alpha level for this set of comparisons is actually .15. This
is much too high.
One way ANOVAs allow us to determine the likelihood of finding a difference between at least two of the
conditions. The p value for a one way ANOVA refers to the overall probability that at least one of the
conditions will differ from at least one other condition. We want this probability .05 or less.
A one-way ANOVA is used to compare any number of group means and still maintain the probability of
making a Type I error at 5%. An ANOVA is called an omnibus test because it looks at the amount the
whole set of means differ from each other and determines if that pattern of differences is likely to have
occurred less than 5 % of time by chance alone. If the entire set does not have a p value of greater than .05,
than no one individual comparison can either.
One-way ANOVAs Lab 6 2
Scientific Hypothesis - there are differences between at least 2 of the groups. (One of these things is not
like at least one of the others).
Null Hypothesis – There are no differences among the groups.
The logic of this test is very simple (the mathematics is more complex – but the computer does it). Here is
the basic idea.
Review: When we discussed Measures of Dispersion last term, we talked about variance and standard
deviations. Both of these are measures of how spread out the scores are in a distribution. Recall the
variance is just the square of the standard deviation.
Sources of Variance
Each participant’s RMS can be seen as effected by two influences. First, all participants are
individuals and come into the study with characteristics that are unique to them. For example, some might
be in nasty moods while others may be poor test-takers. Variation due to differences between participants
characteristics is called error variance . Error variance is the variation we would expect to find among
any group of participants due simply to the fact that we have a group of people that differ from each other
in innumerable ways. These differences effect their measurement on the DV but should effect the
measurement of the DV about the same in each condition. The important thing to remember here is that
error variance is variation that occurs among subjects that is not due to differences in the condition they are
participating in.
The second possible source of variation among participants’ RMS is the influence of the different
BS conditions. Thus, each participant’s RMS can been seen to be due to two separate influences. Error
variance (characteristics of the individual) and variance due to the independent variable , BS. Assume that
there is no effect of BS. We would still not expect to obtain exactly the same mean in each condition.
There would be some variation between conditions just due to error variation. Since we are able to
estimate Error variance we have an estimate of how much the means of the conditions are expected to vary
from each other, due to chance alone.
When we did t-tests we were able to use the variance in each sample to estimate the variance we would
expect to find between samples, if we repeatedly sampled from the same population. With t- tests that
s
estimate was called the Standard Error of the Mean ( x ). One way ANOVAs SPSS uses the variances
from all the conditions to estimate of how much variation there is in the DV ratings that is due to factors
other than the IV. In the Background Sound study there are three conditions, so SPSS finds the averaged
variance within the three conditions which is called Within Group Variation. Within group variation,
thus, can be seen as an estimate of how much variation we would expect to find between scores if the IV
has no effect.
Next, we need to measure the amount of between groups variance. Conceptually, this is just the
average amount the means of each condition vary from the mean of the entire group of scores. The way
this is computed is pretty complex, but for this class we do not need to worry about it (SPSS will do it for
us). What you need to know is the logic of what SPSS is doing. The means obtained from each condition
differ from the Grand Mean due to both random variation and to variation due to the independent variable.
Between Group Variance = Error Variance + Variance Due to the Independent Variable
If there is no effect of the IV, then variance added due to the IV will be zero. The equation above
will become
Between Group Variance = Error Variance + 0.
Or
Between Group variance = Error variance.
One-way ANOVAs Lab 6 3
ANOVAs produce a statistic called an F ratio or F statistic. It is simply
F
Between _ Group _ Variance
Error _ Variance
If there is no effect of the IV, the numerator of this ratio should be very close to the denominator.
Therefore, if there is no effect of the IV we expect the F ratio to equal 1 (any number divided by itself is 1).
Both the numerator and the denominator of this ratio however are estimates, and we would expect there to
be some difference between them simply due to chance. Remember, chance is predictable. If there is no
effect due to the IV, than we will generally obtain an F ratio that is very close to 1. As the F ratio becomes
larger, it is less likely that the difference was obtained solely due to chance. SPSS provides the probability
of obtaining the F ratio you have, due to chance alone. This is the p value and it is provided in the SIG
Column of the SPSS print out. If p<.05, then we can conclude that the variation among our groups is
unlikely to be due to chance alone, and that the IV has an effect.
A significant one-way ANOVA (p < .05) tells us there is something other than chance influencing
the differences among the scores we obtained. If the p value is not significant (p > .05) we must accept the
null hypothesis, that there is no difference among our conditions. If we have controlled all influences other
than error variance, that influence must be the IV. What a one-way ANOVA does not tell us, however, is
which conditions are different from which others.
Step 2) The next step is to compare each condition to the other conditions individually. There are a large
number of tests that do this. They are called Post hoc Multiple Comparisons and essentially they are all ttests, with slight modifications. There are many reasons you might choose one of these test over the other
(you will learn these reasons in Graduate School). Here we will simply use one of the most common tests
for looking at differences between individual conditions.
For a within subject design the test is called a Least Squares Difference test, or an LSD. SPSS (if you
direct it to) will provide an LSD test comparing each condition to each other condition. In the
Background Sound study there were three LSD’s conducted. Each LSD has a p value associated with it.
You determine whether the difference between any two conditions is significant, by comparing the p value
obtained to our alpha level (.05). For a between subjects design you simply run follow up t-tests (just like
in lab 5).
Q. Why do we need to run a one way ANOVA first? The one way ANOVA limits the chance of
making a type one error to 5% for all the comparisons we are making. In the Background Sound Study the
ANOVA controls for the fact that the researchers are making six comparisons. If the one way ANOVA is
significant, then we can be 95% certain that the difference AMONG the entire set of conditions is not
simply due to chance. Therefore, we can also be 95% confident that the difference between any two of the
means is not simply due to chance.
Q. What if my one way ANOVA is not significant? Can I still do multiple comparisons? NO! If
you do, you may well find significant differences, but they are not really significant. They are most likely
due to the fact that you are allowing a 5% chance of making a Type I error with each comparison.
Remember when comparing three conditions to each other, you really have a 15% chance you will find at
least one significant difference among at least two of the condition means due solely to chance. If the one
way ANOVA is not significant, you do not interpret the multiple comparisons. You simply conclude that
your IV did not have a significant effect on your DV -- that there was no significant difference found
between your conditions.
Between Group and Within Groups Designs.
Just like with t-tests, the type of one way ANOVA you run for a Within subjects design is
different than for a Between Groups Design. When we run a Within Subjects design, the same subjects
serve in all conditions of the study. Each groups is equally affected by individual difference variables such
as age, IQ, sex, income, family background, pervious experience etc. Any difference between the
One-way ANOVAs Lab 6 4
conditions can not be due individual differences (THEY ARE THE SAME PEOPLE IN EACH
CONDITION). For a Between Subjects design you are comparing different groups of people to each other.
Error variance for a Within Subject design therefore is much lower than for a Between Subjects design.
Therefore, the same difference between means might be significant if found using a within subjects design
but not significant using a between subjects design. This makes the Within Subjects design much more
powerful. (i.e., you have a much greater chance of finding a significant difference between conditions,
when a real difference actually exists).
Lab 6
For this Lab you will find two sets of data. The first is called Lab6a and the second is called
Lab6b. The data contained in LAB6a will be used to run a Between Subjects one way ANOVA. The data
in Lab6b will be used to run a Within Subjects ANOVA. SPSS refers to a within subjects ANOVA as a
repeated measures design. This is because you are taking repeated measures from the same individual
under different conditions.
Between Subjects One way ANOVA.
The data for this analysis come from a study conducted by Corston and Colman (1996). Corston
and Colman wanted look at the effects of audiences on females’ performance of a motor task. Since they
were only interested in generalizing these results to women, only woman were included as participants in
this study. The motor task these women performed involved using a computer mouse to keep the cursor on
a moving target presented on a computer screen. The women who participated in the study were randomly
assigned to one of three conditions in which they performed the task either alone (with no audience), in the
presence of a female audience, or in the presence of a male audience. The percentage of time spent on
target in the three audience conditions were compared. Higher On-Target Percentages (OTPs) indicate
better performance on the motor task and thus better motor skills.
The data entered into LAB6a contains the following variables. The first column contains the
subject ID number. It is important when running a between subjects design to be sure the each row
contains scores from different subjects. The second column indicates the audience condition the subject
served in . SPSS refers to the IV as the Factor in the analysis. Remember, the word factor is just another
name for IV. I have defined this variable for you . I named it AUDIENCE and then define the three levels
(1 = alone; 2= female audience; 3 = male audience). The third column contains the percentage “on target
scores“. Use the define variable menu to name this variable SCORES.
Procedure
1. From the menu at the top of the screen click on: Analyze, then click on Compare means then on One
way ANOVA.
2. Click on your dependant variable and move it into the box marked Dependant list by clicking on the
arrow button.
3. Click on your independent variable and move it into the box labeled factor.
4. Click the Options button and click on Descriptives (this will instruct SPSS to provide descriptive
statistics for each condition) and Homogeneity of Variance (this will instruct SPSS to include a Levene
test of homogeneity of variance for your conditions). Then click on Continue.
5. Click on the button marked POST HOC. Click on LSD. This will instruct SPSS to conduct post hoc
multiple comparisons for you using LSD tests. There are several multiple comparison tests available
from SPSS. Be sure that you have selected only LSD or your output will provide a lot of stuff that you
will not know what to do with.
6. Click on Continue and then on OK.
Output
Your output fie will contain four tables. The first provides the sample sizes (N), Means, Standard
Deviations, standard error, Minimum and Maximums for each condition and for the entire data set (total).
You should make sure that your minimum and maximum scores make sense. For this data set, the
dependant variable is measured in percentages, therefore, the minimum can not be less than zero and the
One-way ANOVAs Lab 6 5
maximum can not be more than 100. If they are, this indicates that an error has been made in entering the
data. You should also make sure that there are means only for the conditions you expect. If, in error. I had
entered a 5 in the Audience column, SPSS would treat this as a separate condition, and it would severely
mess up your results.
The second table provides the results of the Levene Test of homogeneity of variance. Like the ttest, in order for this test to be valid the variances between conditions should not statistically differ from
each other. If there is a significant difference between the variances of the conditions the p value for this
test will be less than .05. If you have equal numbers of subjects in each of your conditions, violation of the
assumption of homogeneity of variance is not a problem and you can ignore it. In analyzing your study
data, if the Levene test is significant and you do not have equal numbers of subjects in each condition, you
should come see me and I will tell you how to deal with it. Don’t panic, it is an easy problem to deal with.
The third table provides the output for the ANOVA. The three columns that are important to you
are the degrees of freedoms (df), the F statistic and the p value (SIG column). If the p value is less than .05
your One Way ANOVA is significant. This means that you can safely interpret the multiple comparisons
provided in the next table. SPSS also provides Sum of Scores for Between Groups, Within Groups and
Total. These Sums of Squares are used for determining the effect size for the One way ANOVA.
Eta _ Squared 
Sum _ of _ Squares _ between _ Groups
Total _ Sum _ of _ Squares _
Interpreting Eta Squared for a One Way ANOVA is the same as interpreting it for a t-test. We use Cohen’s
classification of .01 as a small effect, .06 as a medium effect, and .14 as a large effect.
The fourth table provides LSD multiple comparison results comparing each condition to each
other condition. Like many of SPSS tables, each comparison is given twice. For example, LSDs are
provided for comparing Alone to Female, and for comparing Female to Alone. These are the same
analysis. The first column of the table indicates which conditions are being compared to which other. One
variable is labeled I and the other J. In the second column, mean differences are given (I - J). This is just
the differences in the means (you can find the means in the first table). Notice the value listed in
comparing Alone to Male is the same as the value listed for comparing Male to Alone except one value is
negative and the other is positive. You should be able to figure out why this is the case. The third column
gives standard errors and the fourth gives p values (SIG column)
When you have finished this analysis, you should write a summary of the Analysis in APA format. I have
provided an example below. Feel free to copy the structure of the sentences.
A one way between groups analysis of variance was conducted to determine the impact
of Age on levels of Optimism. There was a statistically significant difference at the
p<.05 level among Optimism scores for the three Age groups [F(2,432) = 4.60, p = .01].
Despite reaching a statistically significant level, the actual difference in mean scores
among groups was quite small. The effect size, calculated using eta squared, was .02.
Post hoc comparisons using Least Squares Difference analysis indicated that the mean
score for Group 1 (M = 21.36, SD = 4.55) was statistically different from Group 3 (M
=22.96, SD = 4.49). Group 2 did not differ significantly from either Group 1 or Group 3.
The two numbers in parenthesis following the letter F are the degrees of freedom between groups
and the degrees of freedom within groups. The first is related to the number of groups in the analysis (it
should be one less than the number of groups), and the second is related to the number of subjects (it should
be the number of subjects minus the number of groups). These numbers can be obtained from the df
column of the ANOVA Table. They are always presented as Between Groups, comma, Within groups.
The order is important in letting the reader know which is which.
One-way ANOVAs Lab 6 6
One Way Repeated Measures (Within Groups) ANOVA
Data provided in file Lab6b. The data for this analysis involve a study in which a group of students were
invited to participate in an intervention designed to increase their confidence in their ability to do statistics.
Their confidence levels (CLs) were measured before the Intervention (time 1), immediately after the
intervention (time 2) and again three months following the Intervention (time 3). Notice that there are three
CLs per subject and each score for a given subject is entered in the same row (record).
Procedure for one way repeated measures ANOVA
1. From the menu at the top of the screen click on: Analyze, then click on General Linear Model, then
on Repeated Measures.
2. In the Within Subjects Factor Name box type in a name that represents your independent variable
(e.g., Time). This is not an actual variable name, it is just a name you give your IV.
3. In the Number of Levels box type the number of levels (e.g., 3) involved in the analysis.
4. Click Add
5. Click on the Define button on the right-hand side.
6. Select the three variables that represent your repeated measures variable (e.g., your three confidence
scores). Click on the arrow button to move them into the Within Subjects Variables box.
7. Click on the Options box at the button right of the screen.
8. Tick the Descriptive Statistics and Estimates of effect size boxes in the area labeled Display.
9. Click on Continue and then OK.
Output
The output for this analysis provides three tables. The first provides the means, Standard deviations and
Sample size for each condition. The second table is labeled Multivariate Tests. It provides four different
types of test (all of them are types of repeated measures ANOVAs). Multivariate analysis refer to analysis
that are conducted when you have multiple related DVs. We will not be using these type of analysis in this
course, but SPSS is set up to do them. Since you have only one DV all of the values are the same for all
four of the analysis provided in this table. The values that should be of interest to you are the F statistic,
the hypothesis df, the Error df, the p value (in the SIG column) and eta squared. If this analysis is
significant (p< .05) this means that there is a difference between the means of at least two of your
conditions. We do not know which means differ significantly from each other however. SPSS does not
provide Multiple comparison analysis for repeated measures designs. In order to tell which means differ
from each other the simplest thing to do is to compare each condition mean to each other mean using paired
samples (within subjects) t-tests (see lab 5).
e.g. Time 1 to Time 2
Time 1 to Time 3
Time2 to Time 3
You can review how to do these from LAB5. While SPSS does not conduct Multiple comparisons for
repeated measures analysis, it does compute eta squared automatically for you. The third table in the
printout labeled Mauchley’s Test of Sphericity can be ignored. It does not apply to univariate analysis.
Sample APA format Write up.
A one way repeated measures ANOVA was conducted to determine if there was a
difference among IQ scores obtained for individuals using the WAIS, WISC or Raven’s
Progressive Matrices Tests. There was a statistically significant difference at the p<.05
level in optimism scores for the three IQ tests [F(2,432) = 4.60, p = .01]. Despite
reaching a statistically significant level, the actual difference in mean scores among
groups was quite small. The effect size, calculated using eta squared, was .02. Post hoc
comparisons using within group t-tests indicated that subjects mean scores on the WAIS
(M = 121.36, SD = 14.55) were statistically higher than the scores they obtained on the
One-way ANOVAs Lab 6 7
WISC (M =102.96, SD = 14.49). Subjects scores for the Raven’s Progressive Matrices
did not differ significantly from either their scores on the WAIS or on the WISC.
With a repeated Measures ANOVA, the first number given for df comes from the column labeled
hypothesis df. The second number comes from the column labeled error df.
What you need to submit for LAB 6:
1. Your SPSS output for all analysis.(including the within groups t-tests)
- You can delete tables that are not relevant to your analysis.
- You can submit one copy for your group (up to 2 persons per group).
2. Two APA format summary paragraphs (one for each analysis).