Repeated Measures Designs:
Incomplete and Complete
Definition of Repeated Measures Design
• Researcher’s point of view: the same set of
subjects serves at all levels of the IV. Each
subject is measured at each level of the IV.
• Subject’s point of view: each subject
experiences all levels of the IV.
• The ultimate “Matched Groups” Design.
• Example: The Stroop effect with pictureword stimuli
Suppose we had all participants do first
“Incongruent label” (INCL) and then “No
label” (NL) and we found faster times to name
the silhouettes with no label (as predicted).
What other “plausible explanations” might
exist?
What if we did first NL and then INCL and
found slower times for INCL?
How did we “fix” this issue in our study?
• Repeated measures designs are very powerful.
Perfectly matched subjects at each level of IV.
• Because subjects are measured repeatedly,
there are potential issues of co-varying timerelated secondary variables (threats to
internal validity), commonly called “order
effects” or “carry-over effects”
• The most common order effects are: practice,
boredom, and fatigue.
• You cannot eliminate these order effects but
you can “balance” them across levels of your
IV (counterbalancing)
• This can be done in several different ways and
how you do this is tied to the type (name) of
repeated measures design you use
(Incomplete versus Complete)
Incomplete Repeated Measures Design
• The same set of subjects is measured at all
levels of the IV but each subject experiences
and is measured at each level of the IV only
once (only one trial at each level of the IV).
• Example: Lab Stroop task. Each person was
exposed and measured once with NL and once
with INCL.
• DV=time to complete one sheet. Each subject
contributed two scores, one for NL and one for
INCL.
• How did we deal with time-related secondary
variables such as boredom, practice, fatigue?
Counterbalancing for Incomplete
Repeated Measures Designs
• Controlling time-related variables that are
potential threats to internal validity in
incomplete repeated measures designs is easy
for a two-level IV.
• There are only two possible orders for your IV:
AB and BA
• ½ subjects do one order and ½ do the other
• Does not eliminate order effects but does
“balance” them over the two levels of the IV.
• With a 3-level IV there are six possible orders
(3! Or 3 factorial=3 X 2 X 1=6)
• ABC
• ACB
• BAC
• BCA
• CAB
• CBA
• 1/6 of subjects for each order!
• Four level IV= 4! Orders=4X3X2X1=24
orders! Divide subjects into 1/24th!
• For an IV with >3 levels, can use a “Latin
Square”
• Latin Square: an arrangement of symbols in
rows and columns such that each symbol
occurs only once in each row and each column
Latin Square Example for a 4-level IV
Each letter occurs once and only once in each
column and each row. ¼ of subjects assigned
to each “order” (row)
ACBD
DACB
BDAC
CBDA
NOT a Latin Square!!!!
ACBD
DBAC
CDBA
BADC
How to use a Latin Square
• Three level IV: Levels= A, B, C
• Equal # of subjects for each order (row).
• ACB (1/3 of subjects)
BAC (1/3 of subjects)
CBA (1/3 of subjects)
• Each letter occurs only once in each row=each
subject experiences each condition only once.
• Each letter occurs only once in each column=
balances order effects across levels of IV.
Example of an Incomplete Repeated Measures
Design:
Effect of exercise on mood (Hansen, Stevens,
& Coast, 2001 page 248 pb, page 236
hardback)
Four levels of “exercise”, 0 (30 minutes of quiet
resting), 10, 20, 30 min exercise on stationary
bike. All participants did all levels one time
each at one week intervals (over a 4 week
period).
Rotation Method of generating a Latin
Square
• Quick and “dirty”, not the best method
• Better method= “Diagram Balanced Latin
Square”
• Use a random arrangement of symbols
for your conditions for your first row.
A= 0 min, B=10 min, C=20 min, D=30 min
First Row:
BCAD
• For next row, put B at far right and slide rest of
symbols over one to the left
CADB
• Now put C at far right and slide rest of symbols
one to the left
ADBC
• For last row, put A at far right and slide rest of
symbols one to the left
DBCA
• A= 0 min, B=10 min, C=20 min, D=30 min
BCAD (1/4 subjects)
CADB (1/4 subjects)
ADBC (1/4 subjects)
DBCA (1/4 subjects)
Complete Repeated Measures Design
Definition
• The same set of subjects is measured at
all levels of the IV but each subject
experiences each level of the IV more
than once.
Counterbalancing Complete Repeated
Measures Designs: Block Randomization
• In block randomization each block involves
one occurrence of each level of the IV. The
order of the levels in each block is randomly
arranged. (hence, “block randomization”)
Generic example of a Block Randomization
(4 level IV)
• Four level IV: level A, level B, level C, level D.
• Suppose we want 6 trials at each level
• One block= a random order of the 4 levels
(A,B,C,D), for example “BDAC” is one block.
• We would need 6 blocks:
BDAC DACB BADC CADB ACBD DBCA
Each subject would be exposed to all 6 blocks.
Each subject would experience each level
(A,B,C,D) six times.
Research example of block randomization
• Sackheim, Gur and Saucy (1978) (Page 232233 of text)
• Does one side of our face express emotion
more intensely than the other?
• Set of photos of people expressing emotions
• Cut photos in half down middle and created:
– Composite photos of two left sides (L)
– Composite photos of two right sides (R)
– Original photo (O)
– One IV (type of photo) with three levels: O, L, R
Three versions of “disgust”:L,O,R
• One IV (type of photo) with three levels: O, L,
R
• Had different people pose expressing several
different emotions (disgust, fear, joy etc)
• Each participant viewed 54 photos, 18 O, 18L,
and 18 R
• Participants rated each photo on a 7-point scale
indicating the intensity of the emotion
expressed.
• Photos ordered by “block randomization”
• Each block contains one O, one L, and one R.
• There were 18 blocks altogether with O,L, R
randomly ordered in each block.
Which do you think showed most intense emotion?
Left side(a), Original (b), or Right side (c)?
• Findings: Most people judged left-side
composite (L) as showing more intense
emotion. We may display stronger emotion
with the left side of our face. (controlled by
right hemisphere of brain).