MANOVA
Multivariate Analysis of Variance
Adv. Experimental
Methods & Statistics
PSYC 4310 / COGS 6310
Michael J. Kalsher
Department of
Cognitive Science
PSYC 4310
Advanced Experimental Methods and Statistics
Statistical Analysis of Data
© 2012, Michael Kalsher
1 of 42
38
MANOVA:
What is it?
Used to determine main and interaction effects of categorical variables
on multiple DVs.
ANOVA tests the differences in means of a single DV for two or more
categories of IVs.
MANOVA tests whether the vectors of means for the two or more
groups are sampled from the same sampling distribution.
In other words, MANOVA gives a measure of the overall likelihood of picking two or more random vectors of means out
of the same hat.
Purposes for MANOVA
• To compare groups formed by categorical IVs on group differences in a
set of interval-level DVs.
• To use lack of difference for a set of DVs as a criterion for reducing a
set of IVs to a smaller, more easily modeled number of variables.
• To identify the IVs which differentiate a set of DVs the most.
Statistical Analysis of Data
2 of 42
38
Why Use MANOVA?
Advantages:
• Improves chances of discovering changes as a result of
different variables and their interactions.
• Protects against inflated Type I error due to multiple tests of
correlated dependent variables.
• Can detect whether groups differ along a combination of
variables (i.e., a variate), whereas ANOVA can detect only if
groups differ along a single variable.
Disadvantages:
• More complicated and less powerful than ANOVA.
• Analysis often ambiguous in interpretation of the effects of IVs
on any single DV.
Statistical Analysis of Data
3 of 42
38
MANOVA:
Parts of the Analysis
Main Analysis
Four commonly used ways of assessing the overall significance
of a MANOVA: Pillai’s trace (V); Hotelling’s trace, Wilks’s
lambda (), and Roy’s largest root.
•
Debate over which method is best in terms of power and sample size
considerations.
•
Rules of thumb: If group differences are concentrated on the first variate,
Roy’s statistic most powerful followed by Hotelling’s trace, Wilks’s lambda and
Pillai’s trace. When groups differ along more than one variate, the power
ordering is reversed.
Approaches to Follow-up Analysis
•
Perform separate ANOVAs for each DV (controlling for Type I error).
•
Transform linear combinations of DVs to z scores, add them together, then
evaluate the combined scores using ANOVA.
•
Use discriminant function analysis (DFA) (yields one or more uncorrelated
linear combinations of DVs that maximize differences among the groups)
Statistical Analysis of Data
4 of 42
38
MANOVA test statistics:
A Comparison
Statistical Analysis of Data
5 of 42
38
Pillai-Bartlett trace (V) (Pillai’s trace)
Given by the equation below in which  represents the
eigenvalues for each of the discriminant variates, and s
represents the number of variates. Pillai’s trace is the sum
of the proportion of explained variance on the discriminant
functions and is similar to the ratio of SSM/SST
s
V=

i=1
Statistical Analysis of Data
i
1 + i
6 of 42
38
Hotelling’s trace
The Hotelling-Lawley trace is the sum of the
eigenvalues for each variate and is computed by the
equation below. This test statistic is the sum of
SSM/SSR for each of the variates and so it compares
directly to the F ratio in ANOVA.
s
T=

i
i=1
Statistical Analysis of Data
7 of 42
38
Wilks’s lambda ()
Wilks’ lambda is the product of the unexplained variance
on each of the variates. The  symbol is similar to the
summation symbol () except that it means multiply
rather than add up. Wilks’s lambda represents the ratio of
error variance to total variance (SSR/SST) for each variate.
=
Statistical Analysis of Data
s
1
i=1
1 + i

8 of 42
38
Roy’s largest root
Roy’s largest root is the eigenvalue for the first variate.
In a sense, it is the same as the Hotelling-Lawley trace,
except for the first variate only. This statistic represents
the proportion of explained variance to unexplained
variance (SSM/SSR) for the first discriminant function.
This value is conceptually the same as the F-ratio in
univariate ANOVA and represents the maximum
possible between-group difference given the data
collected.
Largest root = largest
Statistical Analysis of Data
9 of 42
38
MANOVA: Assumptions
Independence:
Observations should be statistically independent.
Random Sampling:
Data should be score level and randomly sampled
from the population of interest.
Multivariate Normality:
In ANOVA, we assume that our DV is normally
distributed within each group. In MANOVA, we assume that the DVs (collectively)
have multivariate normality within groups (cannot be tested directly by SPSS).
Homogeneity of Covariance Matrices:
In ANOVA, it is assumed
that the variances in each group are roughly equal (homogeneity of variance). In
MANOVA, we assume this is true for each DV, but also that the correlation
between any two DVs is the same in all groups. As a preliminary test, Levene’s
test should not be significant for any of the DVs. Since Levene’s test doesn’t take
account of the covariances, Box’s M test should be used to test whether the
population variance-covariance matrices of the different groups in the analysis are
equal
Note: The F test from Box’s M statistics should be interpreted cautiously in that a significant result may
be due to violation of the multivariate normality assumption for the Box’s M test, and a non-significant
result may be due to a lack of power.
Statistical Analysis of Data
10 of 42
38
Effect Size Statistics for MANOVA
The multivariate GLM procedure computes a
multivariate effect size index. The multivariate effect
size associated with Wilks’s lambda () is the
multivariate eta square.
Multivariate 2 = 1 - 
1
s
Here, s is equal to the number of levels of the factor minus 1
or the number of DVs, whichever is smaller.
• This statistic should be interpreted similar to a univariate eta square and ranges
in value from 0 to 1.
• A 0 represents no relationship between the factor and the DV, while a 1 indicates
the strongest possible relationship.
• Unclear what should be considered a small, medium, and large effect size for
this statistic.
Statistical Analysis of Data
11 of 42
38
Controversies and MANOVA
Ability of MANOVA to detect true effects
– Ramsey (1982): As correlation between DVs increases, power
of MANOVA decreases.
– Tabachnick & Fidell (2001): MANOVA works best with highly
negatively correlated DVs and acceptably well with moderately
correlated DVs in either direction; MANOVA wasteful when DVs
are uncorrelated.
– Cole et al. (1994): Power of MANOVA depends on combination
of the correlation between DVs and effect size.
• Expecting large effect: MANOVA most powerful if the measures are
somewhat different and if the group differences are in the same direction
for each measure.
• If 2 DVs differ in terms of group differences (one large, one small), then
power increased if DVs are highly correlated.
Statistical Analysis of Data
12 of 42
38
The Matrix: Revisited
The MANOVA test statistic is derived by comparing the ratio of a matrix
representing the systematic variance of all DVs to a matrix representing
the unsystematic variance of all DVs
Off-diagonal
components
5
3
6
10
3
12
4
6
6
4
2
7
10
6
7
6
Square Matrix
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1
Identity Matrix
Statistical Analysis of Data
2
6
4
8
Row Vector
Single person’s score
on four different
variables
Diagonal
components
8
6
Column Vector
10
Four participants’ score
on one variable
15
13 of 42
38
Partitioning the Variance
SST = Sums of Squares Total
SSm = Sums of Squares Model (Systematic Variance)
SST
SSR = Sums of Squares Error
(Unexplained Variance)
MANOVA Test Statistic
Sum of squares and cross-products matrices
SSR
SSM
Systematic
Variance
H
Hypothesis sum of squares
and cross-product matrix or
hypothesis SSCP
Unsystematic
Variance
E
Error sum of squares and
cross-products matrix or error
SSCP
Total Variance
T
Total sum of squares and
cross-products matrix or total
SSCP
Cross-product deviations represent a total value for the combined error between two variables, so in
some sense, they represent an unstandardized estimate of the total correlation between two variables.
Statistical Analysis of Data
14 of 42
38
MANOVA: Performing MANOVA Using SPSS
Statistical Analysis of Data
15 of 42
38
Move “Social Dimension
Summed Scale” and
“Pragmatic Dimension
Summed Scale” to the
Dependent Variables box.
Move “Gender (q047_r01)”
to the Fixed Factors box.
Then, click “Options”.
Statistical Analysis of Data
16 of 42
38
Move “Gender (q047_r01” to
the Display Means box.
Click “Descriptive statistics”,
“Estimates of effect size”, and
“Homogeneity tests” in the
Display box.
Click “Continue”
Statistical Analysis of Data
17 of 42
38
Statistical Analysis of Data
18 of 42
38
SPSS Output
Statistical Analysis of Data
19 of 42
38
SPSS Output: Main Analysis
Statistical Analysis of Data
20 of 42
38
SPSS Output: Univariate ANOVAs
Statistical Analysis of Data
21 of 42
38
Results Section
A one-way MANOVA on the two dependent variables, the summed scales for the Social
and Pragmatic dimensions, was significant for gender, Wilks’s  = .995, F(2,1340) =
3.289, p<.05. Table 1 contains the means and the standard deviations on the dependent
variables for males and females.
Analyses of variance (ANOVAs) on each dependent variable were conducted as follow-up
tests to the MANOVA. The ANOVA on the Social Dimension summed scale scores was
significant, F(1,1341) = 6.29, p<.05, 2 =.005. Males had significantly higher scores
(M=16.66, SE=.31) than females (M=15.59, SE=.297). The ANOVA on the Pragmatic
Dimension summed scale scores was not significant, F(1,1341) = .072, p>.05.
Statistical Analysis of Data
22 of 42
38
Practice Problem:
Effects of study strategies on learning
A researcher investigates the effectiveness of different study strategies
on learning. Thirty undergraduates are randomly assigned to one of three
study conditions. All participants receive the same set of study questions,
but each group receives different instructions about how to study.
The write group is instructed to write responses to each question, the
think group is instructed to think about answers to the questions, and the
talk group is instructed to develop a talk that they could deliver centering
on the answers to the questions.
At the completion of the study session, all students take a quiz consisting
of two types of questions: recall and application.
Statistical Analysis of Data
23 of 42
38
Data Set:
Subject
Group
Effects of study strategies on learning
Application
Recall
Subject
Group
Application Recall
1
1
3
1
16
2
5
7
2
1
4
4
17
2
5
4
3
1
4
3
18
2
5
5
4
1
4
5
19
2
8
7
5
1
2
3
20
2
6
5
6
1
3
2
21
3
4
3
7
1
4
3
22
3
6
6
8
1
3
3
23
3
4
4
9
1
3
5
24
3
4
3
10
1
3
3
25
3
5
6
11
2
6
7
26
3
5
5
12
2
7
4
27
3
5
5
13
2
5
6
28
3
2
4
14
2
6
3
29
3
3
3
15
2
5
2
30
3
4
5
Statistical Analysis of Data
24 of 42
38
Statistical Analysis of Data
25 of 42
38
Move “recall” and
“applicat” to the Dependent
Variables box.
Move “Study Strategies” to
the Fixed Factors box.
Then, click “Options”.
Statistical Analysis of Data
26 of 42
38
Move “group” to the Display
Means box.
Click “Descriptive
statistics”, “Estimates of
effect size”, and
“Homogeneity tests” in the
Display box.
Click “Continue”
Statistical Analysis of Data
27 of 42
38
Select “Post Hoc” from
the Multivariate Screen
Move “group” to the
Post Hoc Test for box.
Select “Bonferroni”
Select “Games Howell”
Click “Continue”
Click “OK”
Statistical Analysis of Data
28 of 42
38
SPSS Output for MANOVA
Statistical Analysis of Data
29 of 42
38
SPSS Output for MANOVA
Statistical Analysis of Data
30 of 42
38
SPSS Output:
Univariate ANOVAs
Statistical Analysis of Data
31 of 42
38
SPSS Output for Post-Hoc Tests
Statistical Analysis of Data
32 of 42
38
Results Section
A one-way Multivariate analysis of variance (MANOVA) was conducted
to determine the effect of the three types of study strategies (thinking,
writing, and talking) on the two dependent variables, the recall and the
application test scores. Significant differences were found among the
three study strategies on the dependent measures, Wilks’s  = .42,
F(4,52) = 7.03, p<.01. The multivariate 2 based on Wilks’s  was .35.
Table 1 contains the means and the standard deviations on the
dependent variables for the three groups.
Statistical Analysis of Data
33 of 42
38
Results Section - continued
Univariate ANOVAs on each dependent variable were conducted as
follow-up tests to the MANOVA. The ANOVA on the recall scores was
significant, F(2,27) = 17.11, p<.01, 2 =.56, as was the ANOVA on the
application scores, F(2,27) = 4.20, p = .026, 2 =.24.
Post-hoc analyses to the univariate ANOVA for the recall and application
scores consisted of conducting pair-wise comparisons to find which study
strategy affected performance most strongly. With respect to the recall
scores, the writing group produced significantly superior performance on
the recall questions in comparison with either of the other two groups
(ps<05). The thinking and talking groups were not significantly different
from each other (p>.05). With respect to the application scores, the
writing group produced significantly better performance on the
application questions than the thinking group (p<.05), but no other
comparisons were significant (ps>.05).
Statistical Analysis of Data
34 of 42
38
One More Step … Following up
MANOVA with Discriminant Analysis
The practice of conducting ANOVAs as follow-up tests
to significant MANOVA has been criticized because
univariate ANOVAs do not take into account the
multivariate nature of MANOVA
An alternative is to conduct follow-up analyses using
discriminant function analysis (DFA). DFA can be used
after MANOVA to see how the dependent variables
discriminate the groups.
DFA yields one or more uncorrelated linear
combinations of dependent variables (variates) that
maximize differences among the groups.
Statistical Analysis of Data
35 of 42
38
Discriminant Analysis:
Statistical Analysis of Data
Step by Step
36 of 42
38
Discriminant Analysis:
Step by Step
Predictors
Statistical Analysis of Data
37 of 42
38
Discriminant Analysis:
Already in the
MANOVA output
Produces the bs
for each variate
Statistical Analysis of Data
Step by Step
Useful for gaining insight
into the relationships
between DVs for each group
Finally, click on “Continue”, then
“Classify” on the main dialog box
for discriminant analysis
38 of 42
38
Discriminant Analysis:
Step by Step
Plots the variate
scores for each
participant
groups
according to the
study strategy
they were given
Provides an overall
gauge of how well the
discriminant variates
classify the actual
participants
Statistical Analysis of Data
39 of 42
38
Discriminant Analysis:
Step by Step
Click on “Save”
These scores can be useful because the
variates that the analysis identifies may
represent important underlying constructs
Statistical Analysis of Data
40 of 42
38
Discriminant Analysis Output
Shows that only one of the variates is significant. Thus, the group differences
shown by the MANOVA can be explained in terms of one underlying dimension
Statistical Analysis of Data
41 of 42
38
Discriminant Analysis Output
These coefficients tell us the
relative contribution of each
variable to the variates.
These values are comparable to
factor loadings. They represent
the relative contribution of each
DV to group separation.
Statistical Analysis of Data
42 of 42
38
Results
The MANOVA was followed up with discriminant analysis,
which revealed two discriminant functions. The first explained
96.7% of the variance, canonical R2 = .56, whereas the second
explained only 3.3% of the variance, canonical R2 = .04. In
combination, these discriminant functions significantly
differentiated the treatment groups, Λ = 0.42, 2(4) = 22.90, p
< .001, but removing the first function indicated that the second
function did not significantly differentiate the treatment groups,
Λ = 0.96, 2(1) = 1.12, p > .05. The correlations between
outcomes and the discriminant functions revealed that total
correct on recall questions loaded almost exclusively on the
first function (r = .99); total correct on application questions
loaded more highly on the second function (r = .89) than on the
first function (r = .47).
Statistical Analysis of Data
43 of 42
38