Factorial Designs
Topic: The analysis and interpretation of designs employing two factors.
Backing up . . .
In designs with only one factor, what do we do about other factors?
Other factors are 1) held constant, 2) randomized, 3) matched, etc. 4) Ignored 5) Included in research
But what if you want to include a second factor in the research.
Question
How should we combine the levels of the second factor with those of the first?
Example
Suppose two Types of Training are being compared – Lecture vs. CAI are being compared.
The situation involves teaching new employees the basic facts they need to know working in an
organization. The training period lasts for one week.
Our interest is in Type of Training, but it might be that the specific job in which an employee would
affect how much they learned. So, Job is an extraneous variable.
So we decided to include Job in the research.
Job has four levels – Clerical, Receptionist, Maintenance, and Managerial.
So this research involves two factors:
Factor 1: Type of Training with two levels – Lecture and CAI.
Factor 2: Job with four levels – Clerical, Receptionist, Maintenance, and Managerial
Suppose the dependent variable is a score on a test of amount learned during a training session.
The most efficient way to conduct research involving two different factors is a design called a Factorial
Design. It’s also called a completely crossed design.
In a factorial(or completely crossed) design, data are gathered at all combinations of levels of both
factors.
This design is best conceptualized using a two way table, with each dimension of the table representing
one of the factors . . .
Clerical
Receptionist Maintenance Managerial
Lecture
Data
Data
Data
Data
CAI
Data
Data
Data
Data
Cler mean
Rec mean
Maint mean
Manag mean
Factorial Designs - 1
Lecture
mean
CAI
mean
2/8/2016
Any researcher would certainly have two common questions concerning the research:
1. Is there any overall difference in performance of those taught using Lecture vs. performance of those
taught using CAI?
This question compares performance of participants in the first row of the two way table with that of
participants in the second row of the table. The difference between row means is called the Main
Effect of the row factor in the above design.
Clerical
Receptionist
Maintenance Managerial
Lecture
Data
Data
Data
Data
CAI
Data
Data
Data
Data
Cler mean
Rec mean
Maint mean
Manag mean
Lecture
mean
CAI
mean
versus
2. Are there any overall differences in performance of Clerical workers, Receptionists, Maintenance
workers, and Managers?
This question compares performance in of participants in the columns of the table. The difference
between column means is called Main Effect of the column factor in the above design.
Clerical
Receptionist
Maintenance Managerial
Lecture
Data
Data
Data
Data
CAI
Data
Data
Data
Data
Cler mean
Rec mean
Maint mean
Manag mean
versus
versus
Lecture
mean
CAI
mean
versus
These two questions are about what are called the Main Effects of the factors.
The effect of each factor by itself is called a Main Effect.
Factorial Designs - 2
2/8/2016
A Third Question
There is a 3rd question, called the interaction question, one that is a little less obvious, but important
nonetheless .
This question is emergent – it exists only because we’ve included two factors in our research and
included them in a factorial arrangement.
It can be asked in two equivalent ways. Both ways are about differences associated with one factor
across levels of the other factor.
3) Version 1: Does the difference between Lecture and CAI change across levels of the Job factor?
3) Version 2: Do the differences between Clericals, Receptionists, Maintenance and Managers
change across levels of the Type of Training factor?
This is a question about what is called the Interaction of the Row and Column factors.
An interaction exists when the row differences change across levels of the column factor or
equivalently, when the column differences change across levels of the row factor
Clerical
Lecture
CAI
Receptionist
Maintenance Managerial
L-C Mean
L-R Mean
L-M Mean
L-B Mean
versus
versus
versus
C-C Mean
C-R Mean
C-M Mean
C-B Mean
Cler mean
Rec mean
Maint mean
Clerical
Receptionist
Maintenance Managerial
Lecture
L-C Mean
L-B Mean
CAI
C-C Mean
L-R Mean
L-M Mean
versus
C-R Mean
C-M Mean
Cler mean
Rec mean
Manag mean
Lecture
mean
CAI
mean
Manag mean
Or
Maint mean
C-B Mean
Factorial Designs - 3
Lecture
mean
CAI
mean
2/8/2016
Formal statistical tests of the effects
1) Test of Row Main Effect – the Row Main Effect is the difference between overall performance in
Row 1 vs. overall performance in Row 2.
Each row is viewed as a group.
The mean of all scores in each row is computed.
The Row main effect is tested by assessing the significance of differences between the marginal
means of each row.
2) Test of Column Main Effect – the Column Main effect is the difference between overall
performance in Col 1 vs Col 2 vs. Col 3 vs. Col 4.
Each column is viewed as a group.
The mean of all scores in each column is computed.
The Column Main Effect is tested by assessing the significance of differences between the marginal
means of each column.
3) Test of interaction effect.
The differences between means within each column are compared with differences between means
within every other column.
If the differences within each column change from one column to the next, the Interaction is
significant.
Or
The differences between means within each row are compared with differences between means within
every other row.
Equivalently, if the differences within each row change from one row to the next, the Interaction is
significant.
Factorial Designs - 4
2/8/2016
Using graphs to visualize main effects and interactions.
The recommended graph
Plot Cell means vs. levels of the Column factor
Connect means of cells within the same row with a line.
Example using artificial data with no interaction
Hypothetical data with 2 scores per cell
Clerical
Lecture
Receptionist
40,50
M=45
30,40
M=35
40
CAI
Marginal
50, 60
M=55
40,50
M=45
50
Maintenance Managerial
60,70
M=65
50,60
M=55
60
70,80
M=75
60,70
M=65
70
Marginal
60
50
55
The plot of Cell Means
80
Lecture
70
CAI
60
50
Mean DV
ROW
40
1.00
30
2.00
1.00
Cler
2.00
Rec
3.00
Maint
4.00
Man
COL
Graphical Representation of Row Main Effect: The average difference in height of the two lines.
Note that in the example, the continuous line is above the dashed line, so there is (if significant) a Row
Main Effect.
Factorial Designs - 5
2/8/2016
Graphical Representation of Column Main Effect: The difference in average heights of points at
each column level.
80
Lecture
70
CAI
60
Mean DV
50
ROW
Average of
all scores in
Column 1
40
1.00
30
2.00
1.00
2.00
3.00
4.00
COL
The column main effect is assessed by comparing the heights of the filled ellipses added to the figure
above. There are clear differences in the heights of the ellipses, sugg esting (if significant) that there is a
Column Main effect.
Graphical Representation of The Interaction Effect
80
Lecture
70
CAI
60
Mean DV
50
Difference in
Row means for
Column 1
40
ROW
1.00
30
2.00
1.00
2.00
3.00
4.00
COL
The Interaction Effect is tested by comparing the differences between rows – represented by the lengths
of the arrows above – at each column.
The arrows all look like they’re about the same length, suggesting that the row differences are the same
from column to column. This means that there is no interaction. Note that the lack of an interaction
means that the lines for the different rows will be parallel.
Factorial Designs - 6
2/8/2016
Example using artificial data with an interaction
Clerical
Receptionist Maintenance Managerial
Lecture
40,50
M=45
30,40
M=35
40
CAI
Marginal
50, 60
M=55
40,50
M=45
50
50,60
M=55
50,60
M=55
55
Marginal
50
40,50
M=45
60,70
M=65
55
50
50
Graph illustrating interaction example
80
CAI
70
60
Lecture
50
Mean DV
ROW
40
1.00
30
2.00
1.00
2.00
3.00
4.00
COL
Graphical Representation of Row main effect: Compare “average” heights of lines.
We can plainly see that the continuous line is above the dashed line for 3 columns but below the dashed
line for the 4th column (Managers). Many times when there is an interaction, the issue of whether there
is a main effect may be in question, as it may be here.
Graphical Representation of Column main effect: Compare “average” heights of points at each
column
It seems that mean performance goes up as we move from Job 1 to Job 2 to Job 3, but then it levels off
between Job3 and Job 4. But we can see that the differences between the columns are not the same for
the dashed line as they are for the continuous line. Again, this is an instance in which there might be a
question concerning whether or not there is a main effect of the column (Job) factor.
Graphical Representation of Interaction effect: Compare differences between heights of the line at
each column.
The differences between heights of the lines are not the same from column to column. So if confirmed
by the appropriate statistical test, an interaction may be present.
Factorial Designs - 7
2/8/2016
Graphs of Types of Outcomes of Factorial Designs
Based on Aron & Aron, p. 374, Table 13-7.
1.
R1
R2
Margin
al
Means
C1
C2
C3
10
20
15
10
20
15
10
20
15
Row
Main Effect
Yes
Column
Main Effect
No
Marginal
Means
10
20
15
Interaction
No
-----------------------------------------------------------------------------------------------------------------------------2.
R1
R2
Margin
al
Means
C1
C2
C3
10
10
10
20
20
20
30
30
30
Row
Main Effect
No
Column
Main Effect
Yes
Marginal
Means
20
20
20
Interaction
No
-----------------------------------------------------------------------------------------------------------------------------3.
R1
R2
Margin
al
Means
C1
C2
C3
10
20
15
20
30
25
30
40
35
Row
Main Effect
Yes
Column
Main Effect
Yes
Marginal
Means
20
30
25
Interaction
No
Factorial Designs - 8
2/8/2016
4.
R1
R2
Margin
al
Means
C1
C2
C3
10
10
10
20
20
20
30
60
45
Row
Main Effect
Yes??
Column
Main Effect
Yes
Marginal
Means
20
30
25
Interaction
Yes
The performance in R2 increases more from C1 to C2 than does performance in R1.
The difference between R1 and R2 is changes as we go from C1 to C3.
-----------------------------------------------------------------------------------------------------------------------------5.
R1
R2
Margin
al
Means
C1
C2
C3
10
30
20
20
20
20
30
10
20
Row
Main Effect
No
Column
Main Effect
No
Marginal
Means
20
20
20
Interaction
Yes
This is a classic crossed interaction. Neither the Row nor the Column Main Effect is important here.
The interaction is the key feature.
----------------------------------------------------------------------------------------------------------------------------6.
R1
R2
Margin
al
Means
C1
C2
C3
10
20
15
20
40
30
30
60
45
Row
Main Effect
Yes
Column
Main Effect
Yes
Marginal
Means
20
40
30
Interaction
Yes
This is a situation that I would interpret as representing both Main Effects and an interaction.
----------------------------------------------------------------------------------------------------------------------------Factorial Designs - 9
2/8/2016
Two Way Factorial ANOVA
Worked Out Example Based on Minium et al. p. 359
The data
The data are from a hypothetical Verbal Learning Experiment in which participants with Low Anxiety
levels and High Anxiety levels are given a verbal learning task. Some are given instructions to induce
little if any pressure. Some are given instructions to induce moderate pressure to perform well.
Others are given instructions to induce strong pressure to perform well.
The data presumably illustrate the classic inverted U relationship of learning to
drive/anxiety/motivation.
id
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
verblearn
anxiety pressure
40
64
46
56
46
46
39
38
44
69
61
54
55
40
43
47
57
51
40
55
50
48
60
63
83
63
53
60
73
69
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
41
34
37
48
57
47
55
33
42
38
48
58
42
40
49
49
56
41
35
57
56
35
43
39
29
32
54
43
49
49
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
Conceptualization: As a 2 (Anxiety) x 3 (Pressure) Factorial
Low Anxiety: 1
High Anxiety: 2
Low pressure: 1
X
X
Moderate Pressure: 2
X
X
High Pressure: 3
X
X
The interests are:
1. Is there a Main Effect of Anxiety. On average do high anxious persons perform better or worse than
low anxious?
2. Is there a Main Effect of Pressure. On average, do persons under different amounts of pressure
perform this task differently?
3. Is there an Interaction of Anxiety and Pressure: Do performance differences between anxiety levels
change at different levels of pressure? Equivalently do the effects of different levels of pressure differ
for people with high anxiety vs. low anxiety?
Factorial Designs - 10
2/8/2016
The analysis
Analyze -> General Linear Model -> Univariate
Specifying Plots
Make the factor that you’re conceptualizing as the Column Factor the Horizontal Axis.
Make the Row Factor the one represented by Separate Lines.
Column factor
Row factor
Factorial Designs - 11
2/8/2016
Specifying Post Hocs for any main effect with more than 2 levels –the column main effect in this
example.
Pressure has 3 levels, so we can specify post hocs for it.
Anxiety has only two levels, so if we find a difference, we know which means are different, and no post
hocs need be specified for it.
The Output
UNIANOVA
verblearn BY anxiety pressure
/METHOD = SSTYPE(3)
/INTERCEPT = INCLUDE
/POSTHOC = pressure ( BTUKEY )
/PLOT = PROFILE( pressure*anxiety )
/PRINT = DESCRIPTIVE ETASQ OPOWER HOMOGENEITY
/CRITERIA = ALPHA(.05)
/DESIGN = anxiety pressure anxiety*pressure .
Univariate Analysis of Variance
G:\MdbT\P510 511\P511L13-Factorial\FactorEGBasedOnMinP359.sav
Be tw ee n-Subj ects Fac tors
N
an xiety
pre ssure
1
30
2
30
1
20
2
20
3
20
Factorial Designs - 12
2/8/2016
De scriptiv e Statis tics
De pend ent V ariab le: ve rblea rn
an xiety
1
pre ssure
1
2
To tal
Me an
48 .80
Std . Deviatio n
10 .685
N
10
2
50 .30
7.4 09
10
3
62 .20
10 .758
10
To tal
53 .77
11 .206
30
1
43 .20
8.3 51
10
2
47 .50
7.9 06
10
3
42 .90
9.1 83
10
To tal
44 .53
8.4 72
30
1
46 .00
9.7 66
20
2
48 .90
7.5 94
20
3
52 .55
13 .885
20
To tal
49 .15
10 .894
60
Low A
High A
df1
df2
5
Sig .
.89 4
54
High
Pressure
62.2
53.77
42.9
44.53
52.55
49.15
Effect sizes are presented either in terms of
f or eta-squared (2).
Small Medium
Large
f
.1
.25
.4
2

.01
.059
.138
De pend ent V ariab le: ve rble arn
.32 8
Moderate
Pressure
50.3
47.5
48.90
The means as a two way table.
Le v ene 's Te st of Equa lity of Error Va rianc es a
F
Low
Pressure
48.8
43.2
46.0
Te sts th e nul l hyp othesis tha t the error varia nce of the
de pend ent variab le is e qual acro ss gro ups.
a. De sign: Intercept+ anxie ty+p ressu re+an xiety * pre ssure
Tes ts of Betw een-Subj ects Effec ts
De pend ent V ariab le: ve rblea rn
So urce
Co rrecte d Mo del
Type III Sum
of Squa res
24 89.35 0 b
F
5.9 58
Sig .
.00 0
Pa rtial E ta
Sq uared
.35 6
No ncen t.
Pa rame ter
29 .791
a
5
Me an S quare
49 7.870
14 4943 .350
1
14 4943 .350
17 34.57 9
.00 0
.97 0
17 34.57 9
1.0 00
an xiety
12 78.81 7
1
12 78.81 7
15 .304
.00 0
.22 1
15 .304
.97 0
pre ssure
43 0.900
2
21 5.450
2.5 78
.08 5
.08 7
5.1 57
.49 3
an xiety *
pre ssure
77 9.633
2
38 9.817
4.6 65
.01 4
.14 7
9.3 30
.76 2
Error
45 12.30 0
54
83 .561
To tal
15 1945 .000
60
70 01.65 0
59
Int ercep t
Co rrecte d To tal
df
Ob serve d Po wer
.99 0
a. Co mput ed using a lpha = .05
b. R S quared = .356 (Adju sted R Sq uared = .2 96)
The Main effect of Anxiety was significant, and the estimate of effect size, eta-squared was .221, huge.
The Main effect of Pressure was not significant, although eta-squared equals .087.
Due to small sample size, test was not powerful enough to detect the fairly large difference.
The Interaction was significant, with eta-squared equal to .147.
Whenever you have a significant interaction, you should be very cautious in interpreting and reporting
main effects. The significant interaction may indicate that there is an effect of a variable, but that it is
not a MAIN effect – it is a specific effect, specific to one or more levels of the other factor.
Factorial Designs - 13
2/8/2016
Post Hoc Tests
pressure
Homogeneous Subsets
v e rblea rn
Tu key B
a,b
As might have been expected from the
nonsignficiant Main Effect of Pressure,
there are no significant overall
differences between any of the means at
each pressure level.
Su bset
pre ssure
1
N
20
1
46. 00
2
20
48. 90
3
20
52. 55
Me ans fo r gro ups in hom ogen eous subse ts are disp layed .
Ba sed o n Typ e III S um o f Squ ares
Th e erro r term is M ean S quare(Erro r) = 8 3.56 1.
a. Use s Harmoni c Me an Sa mple Size = 20 .000.
b. Alp ha = .05.
Profile Plots
Test question illustrated here:
Is there an effect of pressure?
Yes, but it’s not a MAIN effect.
Low
For Low Anxiety
persons, more
pressure leads to
higher
performance
High
Pressure
marginal
means not
significant
ly
different.
For High Anxiety
persons, more pressure
leads to higher
performance up to a
point, then to decreased
performance.
There IS an effect of pressure in these data, but it is not a MAIN effect. Instead, it would be best
characterized as an “anxiety specific” effect. For low anxiety participants, increasing pressure lead to
increasing performance.
But for high anxiety participants, increasing pressure lead to increasing performance only up to a point.
After that, further increases lead to a decrease in performance.
Factorial Designs - 14
2/8/2016
Analysis of Change Between Pre- and Post-test Performance
Start here on 11/24/15
A combination between-group and within-groups design
Two buildings of an organization were given a pretest measuring productivity. Then employees in
Building A were assigned to work in teams while those in Building B performed essentially the same
work as before. After 6 months, posttests of productivity were obtained for each. Thus, each person
was measured twice using the same test. The interest was in determining whether persons in Building
A increased productivity more than those in Building B. This is a Pretest-posttest with nonequivalent
groups design – Lecture 8, p. 12.)
The Data Editor . . .
id bldg
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
pre post
52
50
62
63
35
82
39
39
60
59
66
35
64
37
59
49
39
44
53
46
18
75
64
58
29
56
67
60
42
24
61
17
34
58
69
62
46
45
56
36
46
61
45
73
62
54
35
46
51
48
49
54
56
26
72
46
38
64
53
67
26
56
37
49
48
34
35
50
35
21
74
54
64
29
46
71
50
41
21
69
8
35
54
68
69
37
45
57
43
43
60
35
77
61
51
35
41
58
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
50
37
50
32
79
49
48
36
34
30
64
54
52
58
23
56
43
51
35
46
43
29
63
45
49
62
49
51
31
71
72
50
31
19
66
42
65
47
35
40
36
56
48
47
69
44
58
20
35
35
57
54
26
54
27
73
46
42
38
42
23
62
50
56
50
29
53
45
53
37
39
44
33
63
41
38
50
53
51
20
76
78
53
21
25
71
42
56
39
29
30
39
45
42
47
58
35
59
20
25
31
61
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Factorial Designs - 15
42
35
42
61
49
55
54
44
39
66
43
48
48
44
43
62
36
44
53
47
59
52
50
55
39
39
56
50
74
48
58
53
45
55
58
50
54
49
57
47
30
41
58
51
54
50
64
66
33
69
45
44
39
62
75
51
67
72
51
59
70
63
53
54
51
56
77
51
47
65
56
76
71
59
70
56
48
65
68
94
67
73
69
58
72
63
61
62
65
67
64
36
46
63
70
69
65
79
77
35
72
53
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
40
54
55
45
50
41
42
59
56
63
49
41
51
62
56
85
54
44
51
39
31
57
45
40
34
45
34
52
33
46
46
40
47
67
60
44
59
61
62
47
57
56
57
54
58
48
52
51
48
57
66
65
67
70
56
62
72
64
71
57
56
71
65
78
90
61
55
63
57
37
71
56
43
48
66
42
61
50
63
51
49
66
76
77
54
71
81
78
56
60
77
77
70
62
53
71
56
52
2/8/2016
The expected result – from Lecture 9 . . .
6. A salvage design: The Pretest-Posttest with Nonequivalent Groups Design
It is important to note that the pretest and posttest must be the same instrument.
One of the most frequently employed designs in the social sciences.
Some outcomes lead to defensible arguments for treatment differences.
Others do not.
The ideal outcome:
T
Mean
Performance
C
Pre
Post
If this pattern of results occurs – no difference on the pretest, difference favoring the treatment group on
the posttest, most researchers would argue that it is evidence for the existence of a treatment effect.
Using our new knowledge of factorial designs, we can recognize this as such a design . . .
Treatment Condition – T vs. C – is the Row factor.
Time – Pre vs. Post – is the Column factor.
In most applications of this design, the hoped-for result is a significant interaction with small, even
nonsignificant row and column main effects.
Factorial Designs - 16
2/8/2016
Analyze -> General Linear Model -> Repeated Measures
Make up a name
for the repeated
measures factor
and enter it here.
Enter the number
of levels of the
repeated measures
factor here.
Click Add, then
Click Define
Factorial Designs - 17
2/8/2016
The main dialog box
Click on Plots
Click on
Options
Factorial Designs - 18
2/8/2016
Specifying Plots
Put the Time factor (prepost in this case) on the horizontal axis.
Specify the Between-subjects factor (bldg) to be represented by separate lines.
Bldg A
Bldg B
Pre
50.32
48.75
Post
62.33
46.34
Two way table of means.
Factorial Designs - 19
2/8/2016
The multivariate tests require the fewest restrictive assumptions. In this case, all tests lead to the same
conclusions.
Factorial Designs - 20
2/8/2016
Results . . .
1. There is an overall difference between Pre and Post-test means.
But that may just be due to the huge increase in Building A.
2. There is an overall difference between Building A and Building B means.
But that may be an artifact of the huge increase in Building A.
3. There is an interaction of Building and Prepost. Thus, the difference between Pre- and Post-test
means depends on which building is considered.
This is the key finding here – Building A performance increased, Building B’s did not.
Use the plots to interpret the interaction in greater detail.
This shows, confirmed by the significant interaction shown on the previous page that Performance in
Building A increased from Pre to Post while performance in Build B decreased.
The conclusion is that assigning people to work in teams may have lead to increases in individual
performance.
Factorial Designs - 21
2/8/2016