Lecture 18: Factorial – Analysis
Recall Experiment
Informal Analysis
Analysis
Analysis of the distribution
of the response.
Sample means and plots.
Informal analysis.
Formal analysis.
Analysis of residuals.
1
2
Distribution of Recall
Distributions
Recall
Summary Statistics
6
Mean
24
Std Dev 7.0093106
N
24
Variance 49.130435
Overall sample mean: 24
Overall sample variance: 49.13
Distribution is slightly skewed
left with quite a bit of
variability.
5
4
3
2
1
5
10
15
20
25
30
35
40
3
4
Analysis
Model
Can we attribute some of
that total variability to
differences between factor
level means and/or to
interaction effects?
=
5
+
+
6
1
Lecture 18: Factorial – Analysis
Treatment Sample Means
Estimated Treatment Effects
Isolation Time
Isolation Time
20 minutes 40 minutes 60 minutes
Reinforcement
None
Reinforcement
Verbal
22.0
30.5
13.5
21.0
25.5
31.5
20 minutes
Reinforcement
None
24.0
Reinforcement
Verbal
40 minutes
60 minutes
22.0 – 24.0 30.5 – 24.0 13.5 – 24.0
= –2.0
= +6.5
= –10.5
21.0 – 24.0 25.5 – 24.0 31.5 – 24.0
= –3.0
= +1.5
= +7.5
7
8
Analysis of Variance
Source
Treatments
Error
C. Total
df
Model
Sum of Squares
5
18
23
1130.0
=
+
+
+
+
9
Factor Means
10
Estimated Factor Effects
Isolation Time
20 minutes 40 minutes 60 minutes
Reinforcement
None
Reinforcement
Verbal
22.0
30.5
13.5
22.0
21.0
25.5
31.5
26.0
21.5
28.0
22.5
24.0
11
Reinforcement
None
22.0
22.0 – 24.0 = –2.0
Reinforcement
Verbal
Overall
26.0
26.0 – 24.0 = +2.0
24.0
12
2
Lecture 18: Factorial – Analysis
Estimated Factor Effects
Isolation Time
20 minutes
40 minutes
60 minutes
21.5
28.0
22.5
21.5 – 24.0 28.0 – 24.0 22.5 – 24.0
= –2.5
= 4.0
= –1.5
Overall
24.0
Analysis of Variance
Source
Reinforcement
Isolation Time
Reinforce*IsoTime
Error
C. Total
df Sum of Squares
1
2
2
18
23
1130.0
13
14
15
16
17
18
Comment
The treatments (5 df) are
split up into the
Reinforcement (1 df),
Isolation Time (2 df) and
the interaction (2 df).
Comment
4th graders with verbal
reinforcement have a higher
average recall score (26)
than those who received no
reinforcement (22).
3
Lecture 18: Factorial – Analysis
Comment
4th graders who were isolated
for 40 minutes had the highest
average recall score (28).
Isolation times of 20 and 60
minutes had similar average
recall scores (around 22).
19
20
Comment
Comment
4th graders with verbal
reinforcement showed
higher average recall scores
the longer they were
isolated.
4th graders with no
reinforcement showed a higher
average recall score after 40
minutes compared to 20
minutes but then at 60 minutes
the average recall was very
low.
21
22
Formal Analysis
JMP Data Table
There are apparent effects due
to reinforcement and isolation
time as well as the interaction
between these two factors.
Are these apparent effects
statistically significant.
23
Character
Nominal
Reinforce
None
None
None
Character
Nominal
IsoTime
20
20
20
Verbal
60
Numeric
Continuous
Recall
18
24
26
32
24
4
Lecture 18: Factorial – Analysis
JMP
JMP
Analyze – Fit Model
Personality: Standard Least
Squares
Emphasis: Minimal Report
Y: Recall
Construct Model Effects
Reinforce
IsoTime
Reinforce*IsoTime
25
26
Summary of Fit
Response Recall
Summary of Fit
0.79292
RSquare
0.735398
RSquare Adj
3.605551
Root Mean Square Error
24
Mean of Response
24
Observations (or Sum Wgts)
Rsquare: 79.3% of the
variation in recall can be
explained by Reinforcement,
Isolation Time and their
interaction.
Analysis of Variance
Source
Model
Error
C. Total
DF
5
18
23
Sum of
Squares Mean Square F Ratio
179.200 13.7846
896.0000
13.000 Prob > F
234.0000
1130.0000
<.0001*
27
Summary of Fit
28
Analysis of Variance
This is the same as we have
seen before except that the
Model source is equivalent to
the source of variation due to
differences amongst
treatments.
RMSE = 3.360555
This is the estimate of the
error standard deviation, σ.
29
30
5
Lecture 18: Factorial – Analysis
Analysis of Variance
Test of Treatment Effects
Source df SS
MS
F
Prob > F
Model
5 896.0 179.2 13.785 < .0001
Error
18 234.0 13.0
C. Total 23 1130.0
H0:all treatment effects are 0
H0:some treatment effects are
not 0
H0:all treatment means are equal
H0:some treatment means are
not equal
31
Statistical Evidence
32
Conclusion
F = 13.785
P-value < 0.0001
The P-value is so small
(<0.05) that we should
reject the null hypothesis.
Some of the treatments
(combinations of
reinforcement and isolation
time) have different effects
(means).
33
Next Step
34
Note
We could look at multiple
comparisons (15) of the
treatment sample means.
Instead we will look at what
each factor and interaction
contributes to the differences
in treatment means.
Because there are a number of
different F statistics and P-values
in the JMP output, when
providing statistical evidence for
a test of hypothesis you must
report both the value of the F
and the P-value and use the Pvalue to reach a decision.
35
36
6