Relating Statistical & Methodological Variance Sources
“Statistical F”
SStotal = SSeffect + SSerror
SStotal  variation among all DV values
SSeffect  variation between conditions
SSerror  variation within conditions
“Methodological F”
SStotal = SSiv + SSsubcon + SSproccon + SSinddif + SSwcsubinf + SSwcprocinf
Numerator Variance Source

iv  IV
Denominator Variance Sources
 inddif  individual differences
subcon  subject variable confound(s)
wcsubinf  within-condition subject variable influences
proccon procedure variable confound(s)
wcprocinf  within-condition procedural variable influences
First Study:
We wanted to study how 3rd graders respond to practice with math facts (1-digit addition
problems). Our research hypothesis was that the amount of practice right before the test was
given would influence test performance. We used a sample of 90 3 rd graders from a single
elementary school. Each student was randomly assigned to complete 10 vs. 30 problems right
before the addition facts test sheet was handed out. Students had 90 seconds to complete the test.
For each replication below, identify and label the additional Design F source(s) of variation
and tell the expected effect on the Statistical F
“Common sense” among educational researchers:
 Self-paced task completion usually leads to increased heterogeneity
 Increasing task difficulty usually leads to increased heterogeneity
Replication #1
We used a sample of 90 3rd graders from a single elementary school. Each student was randomly
assigned to complete 10 vs. 50 problems right before the addition facts test sheet was handed out.
Students had 90 seconds to complete the test.
What’s different?
Which variance source is this ?
Will it influence numerator or denominator?
Bigger or smaller?
What will be influence on F?
Replication #2
We used a sample of 90 3rd & 4th graders from a single elementary school. 3rd graders completed
10 and 4th graders completed 30 problems right before the addition facts test sheet was handed
out. Students had 90 seconds to complete the test.
What’s different?
Which variance source is this ?
Will it influence numerator or denominator?
influence on F
Bigger or smaller?
What will be
Replication #3
Unlike the earlier study, our purpose was to study the practice effects for 3rd & 4th graders. Each
student was randomly assigned to complete 10 vs. 30 problems right before the addition facts test
sheet was handed out. Students had 90 seconds to complete the test.
What’s different?
Which variance source is this ?
Will it influence numerator or denominator?
Bigger or smaller?
What will be influence on F
Replication #4
We used a sample of 90 2nd 3rd & 4th graders from a single elementary school. Each student was
randomly assigned to complete 10 vs. 30 problems right before the addition facts test sheet was
handed out. Students had 90 seconds to complete the test.
What’s different?
Which variance source is this ?
Will it influence numerator or denominator?
Bigger or smaller?
What will be influence on F
Replication #5
We used a sample of 90 3rd graders from a single elementary school. Each student was randomly
assigned to complete 10 vs. 30 problems right before the subtraction facts test sheet was handed
out. Students had 90 seconds to complete the test.
What’s different?
Which variance source is this ?
Will it influence numerator or denominator?
Bigger or smaller?
What will be influence on F
Replication #6
We used a sample of 90 3rd & 4th graders from a single elementary school. 3rd graders completed
30 and 4th graders completed 10 problems right before the addition facts test sheet was handed
out. Students had 90 seconds to complete the test.
What’s different?
Which variance source is this ?
Will it influence numerator or denominator?
Bigger or smaller?
What will be influence on F
Replication #7
We used a sample of 90 3rd graders from a single elementary school. Each student was randomly
assigned to complete 10 vs. 30 problems right before the addition facts test sheet was handed out.
Students were given as much time as they wanted to complete the test.
What’s different?
Which variance source is this ?
Will it influence numerator or denominator?
Bigger or smaller?
What will be influence on F
Replication #8
We used a sample of 90 3rd graders from a single elementary school. Each student was randomly
assigned to complete 10 vs. 30 problems right before the addition facts test sheet was handed out.
Students had 90 seconds to complete the test if they had 30 practices and 130 seconds to
complete the test if they had only 10 practices.
What’s different?
Which variance source is this ?
Will it influence numerator or denominator?
Bigger or smaller?
What will be influence on F?