Factorial MANOVA
MANOVA is used when you want to integrate multiple dependent variables into a factorial analysis. The
procedure constructs a new “canonical variate” (a weighted linear combination of the DVs) for each effect that
maximally discriminates among the conditions of that effect, then tests whether the discrimination (mean difference) is
significant.
I like to start the analysis with separate factorial ANOVAs for each DV, to get an idea of the differential patterns of
effects for the different DVs. Also, it is always good to know the bivariate effects, so you can determine whether or not
the multivariate results have any “surprises.” Sometimes, if there are no interesting multivariate effects, it may be
preferable to present the individual factorial ANOVAs, rather than the factorial MANOVA. Considerations such as
your audience and the importance and representativeness of your individual DVs would all be part of this decision.
SPSS/GLM gives these results in a very nice and easy-to-use form.
The next stage is to get the multivariate tests. SPSS/GLM also does this very nicely. However, when it comes to
interpreting the multivariate effects and performing multivariate follow-ups, SPSS/GLM isn’t very helpful. The good
news is that SPSS/MANOVA gives us some additional information, and we can write a bit of code to get a very useful
approximation of the rest!
One challenge to these analyses is that there is a lot of output, and some of the tables are pretty large. So, this
runs on a bit… Also, this example works with a very small sample; as with other multivariate analyses, larger sample
sizes that give both statistical power and ample stability should be used whenever possible!
The Study:
The study sought to explore how different combinations of reinforcement (praising correct responses, criticizing
incorrect responses, or no feedback) and task difficulty (easy or hard) on initial trials would influence three outcome
variables, number of problems attempted, number of problems correct, and problem-completion speed on subsequent
trials. Participants completed 20 “training trials” which were designed to produce a pattern of 10 correct and 10
incorrect answers. Whether the task was presented as “easy” or “hard” and the type of reinforcement given were
randomly assigned. Participants were then given an opportunity to complete as may trials as they wanted, and were
paid $5 for each trial on which they got the problem correct. Verbal feedback was not given on these trials.
Using GLM/multivariate to get univariate analyses and multivariate tests:
While it is possible to use GLM/univariate to get separate factorial ANOVAs for each DV, using GLM/multivariate
organizes the output in a way that makes it easier to compare the effect patterns across DVS for each main effect and
the interaction. I’m going to show you the output in a slightly different order than it comes out of SPSS. We’ll look at
the univariate stats for each the DVS for each effect first and then look at the multivariate tests.
Univariate tests of individual DVs
SPSS code:
GLM numtry numcor speed BY task rein
/METHOD=SSTYPE(3)
/EMMEANS=TABLES(task)
/EMMEANS=TABLES(rein) compare (rein)
/EMMEANS=TABLES(task*rein) compare (rein)
/EMMEANS=TABLES(task*rein) compare (task).
 identifies DVs and IVs
 gets marginal means (k=2- follow-ups not necessary)
 gets marginal means and follow-up analyses
 gets SE of Rein for each Task difficulty (with follow-ups)
 gets SE of Task for each Rein type (with follow-ups)
Please note: SPSS includes confidence intervals that correspond to each of the means and means comparison
analyses. In order to same some space, I’ve not included those in the output below.
Univariate significance tests:
Number of items attempted (numtry):
 ME for Rein
 No ME for Task
 No Interaction
Number of items correct (numcor):
 ME for Rein
 ME for Task
 Interaction
Problem completion speed (speed):
 ME for Rein
 ME for Task
 Interaction
Looking at the Task main effect for each DV:
numtry marginal means look very similar.
numcor & speed both have substantial mean differences, and both
with simple > complex
So, thinking multivariately, we would anticipate that numcor and
speed to be part of the task ME canonical variate, and participants in
the simple condition to have higher scores on that variate.
Looking at the Rein main effect for each DV:
While there is a main effect for all 3 DVS, they
do not all have the same pattern.
numtry and speed both have the pattern
numcor has the pattern
 silence = praise > criticism
 praise = criticism > silence
So, thinking multivariately, we would anticipate a diffuse structure for this main effect, with two canonical variates each
with different contributing variables and different marginal mean difference patterns.
Looking at the interaction for each DV:
I’ve decided to use the SE of task for each type reinforcement, but equivalent results are obtained if you use the
SE of rein for each task. Ultimately you’ll need to look at both the check on the descriptiveness of each main effect, but to
save a tree or two I’ll just present the one set of SEs. Similarly, I’ve left out the simple effect F-tests that SPSS produces
for the interaction along with the pairwise comparisons.
There is no interaction for numtry. Numcor and speed have somewhat different simple effects patterns.
For numcor  simple > complex for each type of reinforcement
For speed  simple > complex only for criticism
So, thinking multivariately, we would anticipate a diffuse structure for this interaction, with two canonical variates one
involving numcor and the other involving speed. We expect that these variates will show different SE patterns from each
other.
On to the multivariate analyses…
Multivariate Significance Tests
MANOVA assumes that the
variance of each DV is the same
across effect conditions and that
the covariance among the DVs is
the same across those conditions
as well. Box’s test examines both
of these assumptions. The desired
outcome is to retain H0”:, support
the homogeneity assumptions, as
we found here.
While Wilk’s test is the most commonly used, Pillai’s is usually preferred for
smaller.sample sizes, unequal sample sizes and if there are violations of the
homogeneity assumption.
Notice that while the Rein main effect and the interaction both could have diffuse
structures, GLM only provides a single test of whether that effect is “significant”.
There is much we still don’t know from these GLM results…

What is the composition of the canonical variates that are the basis of the multivariate tests just above? We
can make some guess from the univariate results about the more and less important variables in each. But
we also know better than to assume a direct relationship between bivariate and multivariate results – several
kinds of collinearity effects could be operating. Said differently, we don’t have a description of these
multivariate effects, we only know that they are “significant”.

Both the Rein main effect and for the interaction involve multiple conditions, and, so, could have concentrated
or diffuse structures. We don’t know whether one canonical variate is sufficient to maximally differentiate
among the conditions, or if additional variates would contribute to our understanding of the pattern of
condition differences for these effects?

We know that the Rein main effect and the interaction are “significant.” But we don’t know the pattern of the
mean differences of the canonical variate(s) for those effects. Said differently, we don’t have a description of
these multivariate effects, we only know that they are “significant”.
Fortunately, there are ways to get more information out of SPSS…
Further Analyses of the Canonical Variates
Although not available via the SPSS GUI, the older MANOVA procedure can be accessed using syntax code to
give use several additional pieces of information to help us further understand our results. Many textbooks and web sites
suggest using discriminant function to obtain these same results. However, few statistical packages perform a factorial ldf.
SPSS Code:
 identifies the DVs & IVs
manova numtry numcor speed BY task (1,2) rein (1,3)
/ print = signif (multiv, eigen, dimenr)
 gets several multivariate tests & statistics
/ discrim stan cor.
 gets the weights to interpret and construct the
canonical variates
To save a tree or two, I’ve left of the univariate tests that are part of this output. They are exactly the same as obtained
from GLM. I’ve also left off the SE F-tests, because they are redundant with the pairwise follow-ups.
Examining the Interaction of Rein & Task:
EFFECT .. task BY rein
Multivariate Tests of Significance (S = 2, M = 0, N = 10 )
Test Name
Value
Approx. F
Hypoth. DF
Pillais
.79777
5.08745
Hotellings 3.70942
12.98296
Wilks
.21057
8.64768
Roys
.78718
Note.. F statistic for WILKS' Lambda is exact.
6.00
6.00
6.00
Error DF
Sig. of F
46.00
42.00
44.00
.000
.000
.000
These NHST results are exactly the same as from GLM.
Eigenvalues and Canonical Correlations
Root No.
1
2
Eigenvalue
3.69871
.01071
Pct.
99.71123
.28877
Cum. Pct.
99.71123
100.00000
Canon Cor.
.88723
.10295
Below are the significance tests of each canonical variate. The interaction has a concentrated structure.
Above are the effect sizes for each canonical variate. Rc2 is the variance accounted for canonical variate.
Dimension Reduction Analysis
Roots
Wilks L.
F
1 TO 2
.21057
8.64768
2 TO 2
.98940
.12319
Hypoth. DF
6.00
2.00
Error DF
44.00
23.00
Sig. of F
.000
.885
EFFECT .. task BY rein (Cont.)
Standardized discriminant function coefficients
Function No.
Variable
1
These are the Beta weights. Use them to weight Z-score versions
numtry
-.15606
of the DVs to construct the canonical variate.
numcor
.73197
speed
.71367
Correlations between DEPENDENT and canonical variables
Canonical Variable
Variable
1
These are the Structure weights. They are used to identify the variables that contribute to
numtry
.02545
the canonical variate. Numcor & speed contribute about equally to the interaction variate,
numcor
.65955
while numtry does not contribute.
speed
.73031
This is slightly different from what we expected after examining the bivariate effects.
We know that the canonical variate that describes the Rein x Task interaction is made up of numcor and speed
(weighted in the same direction). What we don’t know is the pattern of that interaction for this canonical variate. To get
that we will need to construct the canonical variate, and then use it in a factorial ANOVA.
First we need to obtain z-score versions of each DV, then calculate a canonical variate score using the Beta weights
shown above.
DESCRIPTIVES VARIABLES=numtry numcor speed
/SAVE.
 defines the DVs
 asks for the construction of Z-score versions of each
compute CV_int = (Znumtry * -.15606) + ( Znumcor * .73197) + (Zspeed * .71367).
GLM CV_int by rein task
/ EMMEANS = tables (rein*task) compare (rein).
 only want to examine the interaction effect
The interaction F-test will not match the F-values from the multivariate analyses, because they apply the df
differently, the error term is calculated somewhat differently, and this model only includes the significant first canonical
variate. Usually we will not report this interaction F-value The purpose of the analysis is to obtain the cell means for the
canonical variate and the follow-ups, to allow us to describe the pattern of the simple effects making up this significant
multivariate interaction.
Using this canonical variate, that is a combination of numcor and speed, we find no SE of Task for Praise, while
for both Criticism and Silence, Easy tasks were performed significantly better than complex tasks. We would probably also
note that the task effect is much larger for Criticism than for Silence (we could use mean difference or effect size
differences to make this case).
Examining and “Checking” the Main Effect of Rein
As with the interaction, we know there is a significant multivariate main effect for Rein, but we do not know which
DVs contribute to the canonical variate, nor do we knows the pattern of marginal mean differences for this effect. Also,
because we have a significant interaction, we need to determine whether the multivariate main effect for Rein is
descriptive for both Simple and Complex tasks.
EFFECT .. rein
Multivariate Tests of Significance (S = 2, M = 0, N = 10 )
Test Name
Value
Approx. F
Hypoth. DF
Pillais
1.20210
11.55041
Hotellings
9.25211
32.38238
Wilks
.07091
20.20539
Roys
.89812
Note.. F statistic for WILKS' Lambda is exact.
6.00
6.00
6.00
Error DF
46.00
42.00
44.00
Sig. of F
.000
.000
.000
These NHST results are exactly the same as from GLM.
Eigenvalues and Canonical Correlations
Root No.
Eigenvalue
Pct.
Cum. Pct.
Canon Cor.
1
2
8.81537
.43674
95.27957
4.72043
95.27957
100.00000
.94769
.55134
Below are the significance tests of each canonical variate. The Rein ME has a diffuse structure.
Above are the effect sizes for each canonical variate. Rc2 is the variance accounted for canonical variate.
Dimension Reduction Analysis
Roots
1 TO 2
2 TO 2
Wilks L.
F
.07091
.69602
Hypoth. DF
20.20539
5.02250
6.00
2.00
Error DF
44.00
23.00
Sig. of F
.000
.015
Standardized discriminant function coefficients
Function No.
Variable
numtry
numcor
speed
1
2
.93466
-.57407
.65244
-.09440
.91311
.47285
These are the Beta weights. Use them to weight Zscore versions of the DVs to construct the
canonical variate.
Correlations between DEPENDENT and canonical variables
Canonical Variable
Variable
numtry
numcor
speed
1
.59336
-.20073
.50607
2
.20055
.86880
.47715
The structure weights tell us that the first canonical variate is a
combination of speed and numtry, weighted in the same direction.
The second canonical variate is a combination of numcor and
speed, also weighted in the same direction.
Again, this isn’t quite what we expected from the bivariate analyses!
In order to explore the pattern and descriptiveness of this multivariate main effect, we have to compute the
canonical variates, then employ them in ANOVAs with follow-ups.
compute CV_rein1 = (Znumtry * .93455) + (Znumcor * -.57407) + (Zspeed * .65244).  compute 1st canonical variate
compute CV_rein2 = (Znumtry * -.09440) + (Znumcor * .91311) + (Zspeed * .47285).  compute 2nd canonical variate
.
GLM CV_rein1 by rein task
 getting results for 1st canonical variate
/ EMMEANS = table (rein) compare (rein)
 follow-ups to describe pattern of Rein main effect
/ EMMEANS = tables (task*rein) compare (rein)
 follow-ups to check descriptiveness of Rein main effect
/ print descriptives.
GLM CV_rein2 by rein task
/ EMMEANS = table (rein) compare (rein)
/ EMMEANS = tables (task*rein) compare (rein)
/ print descriptives.
 getting results for 1st canonical variate
 follow-ups to describe pattern of Rein main effect
 follow-ups to check descriptiveness of Rein main effect
Describing the Main effect of Rein
Showing just the parts of the output that directly apply…
For the 1st canonical variate…
For the 2nd canonical variate …
For the main effect of Rein, the first canonical variate is combination of numtry and speed, with Silence producing
the best performance, Praise having intermediate performance and Criticism leading to the poorest performance.
The second canonical variate, which is a combination of numcor and speed, had the best performance from
Praise, intermediate performance from Silence, and the poorest performance from Criticism.
There were two “surprises” in these effects – highlighting the value of multivariate analyses:


Silence yielding the best performance on the first variate
Speed being multi-vocal to both variates
Checking on the Descriptiveness of the Rein Main effect
This is a little different than we’ve done before! In the past, whenever we’ve had an interaction we’ve had to
check if the pattern of the interaction makes one or both of the main effects misleading. Now, however, it is important
to remember that the canonical variate that is the basis for the interaction and the canonical variate that is the basis
for each main effect might not be the same! In other words, in a factorial MANOVA, it is possible to have a
misleading main effect, even if there is not a significant interaction (because the main effect and interaction
might not involve the same canonical variate)!
For CV_rein1 (numtry & speed)
For CV_rein2 (numcor & speed)
Main effect

Silence > Praise > Criticism
Main effect 
Praise = Silence > Criticism
SE Simple

Silence > Praise > Criticism
SE Simple

Criticism = Praise = Silence*
SE Complex 
Silence > Praise > Criticism
SE Complex 
So, the main effect of the 1st canonical variate is
descriptive.
Praise > Silence > Criticism
So, the main effect of the 2nd canonical variate is
misleading – no pairwise comparison among the Rein
conditions is the same for simple and complex tasks.
*Criticism > Silence
Examining and “Checking” the Main Effect of Task
As with the main effect of Rein, we know there is a significant multivariate main effect for Task, but we do not
know which DVs contribute to the canonical variate, nor do we knows the pattern of marginal mean differences for this
effect. Also, because we have a significant interaction, we need to determine whether the multivariate main effect for
Rein is descriptive for each of Praise, Criticism and Silence.
EFFECT .. task
Multivariate Tests of Significance (S = 1, M = 1/2, N = 10 )
Test Name
Value
Exact F
Hypoth. DF
Pillais
.85654
43.78373
Hotellings
5.97051
43.78373
Wilks
.14346
43.78373
Roys
.85654
Note.. F statistics are exact.
3.00
3.00
3.00
Error DF
22.00
22.00
22.00
Sig. of F
.000
.000
.000
Below is the effect sizes for the single canonical variate. Rc2 is the variance accounted for canonical variate.
Eigenvalues and Canonical Correlations
Root No.
Eigenvalue
Pct.
Cum. Pct.
Canon Cor.
1
5.97051
100.00000
100.00000
.92549
Standardized discriminant function coefficients
Function No.
Variable
numtry
numcor
speed
1
-.05052
.98232
.35629
These are the Beta weights. Use them to weight Zscore versions of the DVs to construct the
canonical variate.
Correlations between DEPENDENT and canonical variables
Canonical Variable
Variable
numtry
numcor
speed
1
-.00005
.85494
.40153
The structure weights tell us that the canonical variate is a
combination of numcor & speed, weighted in the same direction
with a stronger contribution of numcor.
Exactly what we expected from the bivariate analyses!
In order to explore the pattern and descriptiveness of this multivariate main effect, we have to compute the canonical
variate, then employ it in an ANOVA with follow-ups.
compute CV_task = (Znumtry * -.05052) + (Znumcor * .85494) + (Zspeed * .35629).  compute canonical variate
.
GLM CV_task by rein task
 getting results for the canonical variate
/ EMMEANS = tables (task)
 getting marginal means for the canonical variate
/ EMMEANS = tables (rein*task) compare (task)
 follow-ups to check descriptiveness of Task main effect
/ print descriptives.
Describing the Main effect of Task
Showing just the parts of the output that directly apply
All we need is the direction of the main effect of Task, we know the effect is
significant.
Not surprisingly, on a canonical variate comprised of accuracy & speed, those
performing a simple task did better on average than those doing a complex
task.
Checking on the Descriptiveness of the Rein Main effect
As with the main effect of Rein, it is important to remember that the canonical variate that is the basis for the
interaction and the canonical variate that is the basis for each main effect might not be the same! In other words, in a
factorial MANOVA, it is possible to have a misleading main effect, even if there is not a significant interaction
(because the main effect and interaction might not involve the same canonical variate)!
Main effect of Task

Simple > Complex
SE for Praise

Simple > Complex
SE for Criticism

Simple > Complex
SE for Silence

Simple > Complex
So, the main effect main effect for Task is
descriptive. People do better on Simple
tasks than Complex tasks, and this is the
case for each type of Reinforcement.