WattsSEDU7006-8-4
0
Experimental Designs II
Stephen W. Watts
Northcentral University
WattsSEDU7006-8-4
1
Experimental Designs II
Jackson (2012) Chapter Exercises
#2. A 4 x 6 factorial design has two independent variables; the first with four levels and
the second with six. A 4 x 6 factorial design has 24 conditions.
#4. A cell mean represents the average score of participants in a condition, where a
specific value of each independent variable interacts. Main effect means represent the average
score of participants for a single independent variable, where no interaction with other
independent variables are considered.
#6. In a complete factorial design comparisons are made between each level of all
independent variables with each level of every other independent variable. An incomplete
factorial design is considered incomplete because comparisons are not made between each level
of all independent variables with each level of every other independent variable; some
comparisons are not made (Jackson, 2012).
#8. The number associated with the way of the ANOVA identifies the number of
independent variables. A two-way ANOVA is an analysis of variance with two independent
variables. A three-way ANOVA is an analysis of variance comparing three independent
variables.
#10.
SOURCE
Factor A
Factor B
AxB
Error
Total
df
1
2
2
30
35
SS
60
40
90
200
390
MS
60 F(1, 30) =
20 F(2, 30) =
45 F(2, 30) =
6.667
SOURCE
Factor A
df
2
SS
40
MS
20 F(2, 72) =
F
9 Fcv =
3 Fcv =
6.75 Fcv =
F
3 Fcv =
0.05
4.17
3.32
3.32
0.05
3.15
0.01
7.56
5.39
5.39
0.01
4.98
WattsSEDU7006-8-4
2
Factor B
AxB
Error
Total
3
6
72
83
SOURCE
Factor A
Factor B
AxB
Error
Total
df
1
1
1
36
39
60
150
150
400
SS
10
60
20
60
150
20 F(3, 72) =
25 F(6, 72) =
2.083
3 Fcv =
3.75 Fcv =
2.76
2.25
4.13
3.12
MS
10 F(1, 36) =
60 F(1, 36) =
20 F(1, 36) =
1.667
F
1.50 Fcv =
9.00 Fcv =
3.00 Fcv =
0.05
4.17
4.17
4.17
0.01
7.56
7.56
7.56
#10a. For the first study, Factor A, F(1,30) = 9.00, p < .01; Factor B, F(2,30) = 3.00, not
significant; Interaction between A x B, F(2,30) = 6.75, p < .01.
For the second study, Factor A, F(2,72) = 3.00 not significant; Factor B, F(3, 72) = 3.00,
p < .05; Interaction between A x B, F(6.72) = 3.75, p < .01.
For the third study, Factor A, F(1, 36) = 1.50 not significant; Factor B, F(1,36) = 9.00, p
< .01; Interaction between A x B, F(1,36) = 3.00 not significant.
#10b. The first study is a 2 x 3 factorial design and has six conditions. The second study
is a 3 x 4 factorial design and has 12 conditions. The third study is a 2 x 2 factorial design and
has four conditions.
#10c. In the first study there were 36 participants in the six conditions. In the second
study there were 84 participants in the 12 conditions. In the third study there were 40
participants in the four conditions.
#10d. In the first study, factor A was the only significant main effect accounting for
approximately 15% of the variance, while there was also a significant interaction effect between
factor A and B accounting for approximately 23% of the variance, both at the p < .01 level. In
the second study, the only significant main effect was factor B, accounting for approximately
15% of the variance at the p < .05 level, while the interaction effect between factors A and B
WattsSEDU7006-8-4
3
accounting for approximately 37.5% of the variance, is significant at the p < .01 level. In the
third study there is no significant interaction effect between factors A and B and the only
significant main effect was factor B, accounting for approximately 40% of the variance at the p <
.01 level.
#12a.
SOURCE
Factor A - gender
Factor B - time
AxB
Error
Total
df
1
2
2
24
29
SS
0.027
140.600
0.073
28.000
168.700
MS
0.027 F(1, 24) =
70.3 F(2, 24) =
0.0365 F(2, 24) =
1.167
F
0.023 Fcv =
60.257 Fcv =
0.031 Fcv =
0.05
4.26
3.40
3.40
#12b. Gender, F(1,24) = .02, not significant; Practice, F(2,24) = 60.26, p < .01; Gender x
Time, F(2,24) = .03, not significant.
#12c. Gender and the interaction of gender and practice time do not affect reaction time,
but the main effect of practice time significantly affects reaction time.
#12d. The effect size (η2) is 0.833, meaning that approximately 83% of the variance
regarding reaction time is attributable to the amount of practice time.
14
12
10
Women's
Reaction Time
(sec)
8
6
Men's Reaction
Time (sec)
4
2
0
#12e.
2 hours
Assignment Question Answers
4 hours
6 hours
0.01
7.82
5.61
5.61
WattsSEDU7006-8-4
4
Explain the difference between multiple independent variables and multiple levels of
independent variables. Which is better? Multiple independent variables allow the researcher
to conduct more complicated experiments; manipulating multiple variables and determining the
effect of each independently as well as the interaction between each variable. Multiple levels of
independent variables allow the researcher to assess each independent variable more completely
with multiple treatments or conditions. These operations are not mutually exclusive. An
experiment with multiple independent variables could have one or more independent variables
with multiple levels. Both multiple independent variables and multiple levels of independent
variables allow the researcher to get more information from a single experiment while keeping
the number of participants low. Whether one is better than the other depends on what the
researcher is attempting to discover from the study. If the researcher is seeking to better
understand the behavior of a single factor then a multiple level independent variable is better. If
the researcher believes that a behavior is caused by several factors, or the interaction between
those factors, then a multiple independent variable design is better.
What is blocking and how does it reduce “noise”? What is a disadvantage of
blocking? Blocking is a technique whereby subjects are divided into “relatively homogeneous
subgroups” (Trochim & Donnelly, 2008, p. 198). Noise consists of the extraneous factors that
can make it difficult to determine effects in the real world. Blocking reduces noise and
strengthens the treatment effect by minimizing the variability between subjects in each block.
The disadvantage of blocking is that if, after the division, the subgroups are not homogeneous;
variability increases, decreasing the ability of the researcher to determine effect.
What is a factor? How can the use of factors benefit a design? A factor is a variable.
By using more than one independent variable in a design, researchers are able to determine the
WattsSEDU7006-8-4
5
effect of each independent variable, known as a main effect, as well as the effects of the
interactions between the variables, or interaction effects. Using a factorial design enables the
investigator to conduct research that mirrors the real world more accurately than what?
Explain main effects and interaction effects. Main effects depict the consequences of a
single independent variable in isolation from other independent variables in a factorial design.
Interaction effects depict the results of the combining of two or more independent variables. In
these interactions, the researcher may determine what outcomes are attributable to each variable
at various levels of other variables.
How does a covariate reduce noise? A covariate is a variable that is adjusted in a study
so as to remove its affect on the independent variable. This adjustment is done statistically. By
removing the affect of the covariate, the effect of the independent variable remains, but the
variability due to the covariate is removed, resulting “in a more efficient and more powerful
estimate of the treatment effect” (Trochim & Donnelly, 2008, p. 201).
Describe and explain three trade-offs present in experiments. The one constant of
experiments is the use of randomization to assign subjects to condition groups (Wiley, 2009).
All other decisions with regards to design involve trade-offs; selecting one design has both
advantages and disadvantages. Once a design is determined, the appropriate statistical analysis
is chosen. This analysis is conducted following the collection of data. The largest trade-off of
an experiment regards internal versus external validity. An experiment is about control. The
purpose of an experiment is to keep all conditions equivalent except for the manipulation of the
independent variable. This does not necessarily work well in the real-world. If the researcher
succeeds in this control, the internal validity of the experiment is very high and cause and effect
WattsSEDU7006-8-4
6
can be?? determined, while external validity may be very low since the results may not be
generalizable outside of the experimental conditions.
The number of participants available for a study poses a trade-off to the research
designer. A simple design will generally require fewer participants than will a more complex
design, but complex designs are able to determine interactions and effects that are more
consistent with the real-world, increasing external validity while maintaining internal validity.
Wiley (2009) acknowledged three trade-offs regarding sample size; (a) larger sample sizes allow
for discovering small effects but increase time and expense, (b) larger sample sizes, in a highly
human interactive experiment, may increase more variability than the effects discovered, and (c)
larger sized samples are less likely to show?? large differences between groups, but are unlikely
to eliminate all differences between groups???.
Wiley (2009) identified a potential trade-off in experiments using blocking. Blocking’s
advantages are its ability to test secondary hypotheses, obtain “repeated measures of each
experimental unit or subject” (p. 447), and reduce noise in the results. Blocking’s disadvantages
are that it results in a “lower sample size . . . for testing the primary hypothesis” (p. 447) and
diminishes the ability to generalize results. These are descriptions of some of the trade offs we
encounter when we conduct experiments, but I would have liked to see some explanation or
examples here regarding those trade offs. For example, what do you think about some of these
issues? Do you think we should try to achieve internal or external validity if we have to make a
choice between the two? Do you think the advantages of Blocking outweigh the disadvantages?
Why do you think some of these trade offs occur? How important are they? You could have used
some peer-reviewed articles here to support or illustrate your explanation.
WattsSEDU7006-8-4
7
References
Jackson, S. L. (2012). Research methods and statistics: A critical thinking approach (4th ed.).
Belmont, CA: Wadsworth Cengage Learning.
Trochim, W. M. K., & Donnelly, J. P. (2008). The research methods knowledge base (3rd ed.).
Mason, OH: Cengage Learning.
Wiley, R. H. (2009). Trade-offs in the design of experiments. Journal of Comparative
Psychology, 123(4), 447-449. doi:10.1037/a0016094