Repeated measures ANOVA
LabSyntax, 02/23/06
T. Florian Jaeger
Intro

Basis idea:
We’re interested in populations
 The sample of subjects & items we see is (so we
hope/assume) representative for the population, but
not identical
 I.e. we assume that we observe the effect of our
manipulation on a randomly selected sample of the
population of interest (e.g. native speakers of
English; question-embedding verbs)


What this means:
The statistical analysis should reflect our
assumptions
 Subjects and Items are random effects/factors

[2]
What does it mean to be a random
effect?

Not all levels of the effect are observed in the
experiment
 Sometimes
we have such specific research
questions that we DO observe all levels (in terms
of items), consider e.g. irregular verbs with a
specific stress pattern.

[3]
For corpus work: the # of levels for a random effect
usually increases as the sample size increases
The standard (psycholinguistic)
analysis

ANOVA (Analysis of variance)
Designed for balanced design (unreliable for
unbalanced designs)
 Check other assumptions (normally distributed error,
sphericity/homogeneity of variance), e.g. Howell
1995


Separate subject (F1) and item (F2) analyses:
F1: for each subject average/aggregate within each
condition over all items
 F2: for each item average/aggregate within each
condition over all subjects
 Advantage: even for missing cells, averaging gives
you a mean value for each condition for each
subject/item

[4]
SPSS
Example courtesy of John Hale
Repeated Measures in SPSS

Open file,
import data
1st subject, Subject ID=0
20 items
Design: 2 (LIKE) x 2 (WAY)
[6]
UCP1.DIFF.CONJ
He wants to go to the movies and mini-golfing.
UCP1.SAME.CONJ
He wants to go to the movies and to the mini-golf
course.
UCP1.DIFF.REPAIR
He wants to go to the movies, I mean, mini-golfing.
UCP1.SAME.REPAIR
He wants to go to the movies, I mean, to the mini-golf
course.
Repeated Measures in SPSS, e.g. F1


Open file,
import data
Aggregate
(e.g. over
items)
Data  Aggregate
[7]
Repeated Measures in SPSS, e.g. F1


Open file,
import data
Aggregate
(e.g. over
items)
Data  Aggregate


Open
aggregate
file
Restructure
Data  Restructure
[8]
Repeated Measures in SPSS, e.g. F1


Open file,
import data
Aggregate
(e.g. over
items)
Data  Aggregate


Open
aggregate
file
Restructure
Data  Restructure
[9]
Then go “Next”  “Next”  “Next” 
“Finish”
Repeated Measures in SPSS, e.g. F1


Open file,
import data
Aggregate
(e.g. over
items)
Data  Aggregate


Open
aggregate file
Restructure
Data  Restructure

Check!
Look at output and
data

[10]
SAVE!!
1 subject per line
All 2 x 2 = 4 conditions in
Columns in that line
Repeated Measures in SPSS,
ANALYSIS

Start
repeated
measures
Analysis  GLM 
repeated measures

[11]
Define
factors and
measure
Repeated Measures in SPSS,
ANALYSIS

Start
repeated
measures
Analysis  GLM 
repeated measures


Define
factors and
measure
Define levels
Make sure order is
right!
[12]
Repeated Measures in SPSS,
ANALYSIS

Start
repeated
measures
Analysis  GLM 
repeated measures H


Define
factors and
measure
Define levels
LIST could go here, as could AGE, GENDER, e
Frequency, Length could be entered as covariat
Make sure order is
right!


[13]
Define some
plots
Optionally
post-hoc
tests
Add e.g. interaction plot
Repeated Measures in SPSS,
Results
Sphericity is not an issue with binary factors
But with more levels:
check whether sphericity holds
(Mauchly’s test)
If not, use Greenhouse-Geisser
[14]
Repeated Measures in SPSS,
Results (plots)


Be aware that plots use marginal means (not the
same as means)
To read plots, look at Within-subject Factors
Within-Subjects Factors
Measure: rating
like
1
2
way
1
2
1
2
Dependent
Variable
diff.conj
diff.repair
s ame.conj
s ame.repair
DIFF.REPAIR
He wants to go to the
movies, I mean, mini-golfing.
[15]
DIFF.CONJ
He wants to go to the
movies and minigolfing.
SAME.CONJ
He wants to go to the
movies and to the minigolf course.
SAME.REPAIR
He wants to go to the
movies, I mean, to the minigolf course.
Repeated Measures in SPSS, do
results hold?

Item analysis
Tests of Within-Subjects Effects
Measure: rating
Source
like
Error(like)
way
Error(way)
like * way
Spherici ty As sum ed
Greenhouse-Gei ss er
Huynh-Fel dt
Lower-bound
Spherici ty As sum ed
Greenhouse-Gei ss er
Huynh-Fel dt
Lower-bound
Spherici ty As sum ed
Greenhouse-Gei ss er
Huynh-Fel dt
Lower-bound
Spherici ty As sum ed
Greenhouse-Gei ss er
Huynh-Fel dt
Lower-bound
Spherici ty As sum ed
Greenhouse-Gei ss er
Huynh-Fel dt
Lower-bound
Spherici ty As sum ed
Greenhouse-Gei ss er
Huynh-Fel dt
Lower-bound
Type III Sum
of Squares
1.712
1.712
1.712
1.712
1.726
1.726
1.726
1.726
7.552
7.552
7.552
7.552
4.210
4.210
4.210
4.210
.166
.166
.166
.166
4.847
4.847
4.847
4.847
df
1
1.000
1.000
1.000
19
19.000
19.000
19.000
1
1.000
1.000
1.000
19
19.000
19.000
19.000
1
1.000
1.000
1.000
19
19.000
19.000
19.000
Mean Square
1.712
1.712
1.712
1.712
.091
.091
.091
.091
7.552
7.552
7.552
7.552
.222
.222
.222
.222
.166
.166
.166
.166
.255
.255
.255
.255
F
18.847
18.847
18.847
18.847
Sig.
.000
.000
.000
.000
34.086
.000
34.086
34.086
.000
.000
34.086
Interaction not confirmed
by .000
F2 analysis
But less power (look at DF of error, 19 instead of 86)
Probably small effect size
.651
.651
.651
.651
.430
.430
.430
.430
Exaggerated MORAL:
Wasted subjects! Lots of power in terms
of subjects does not help with F2 analysis!
Error(like*way)
[16]

[17]
The previous example was taken from a study by
John Hale (in progress) not to be cited or circulated
without his permission.