Inferential Statistics III:
ANCOVA
Michael J. Kalsher
Department of
Cognitive Science
MGMT 6970
PSYCHOMETRICS
© 2014 Kalsher
1 of 43
Analysis of Covariance
• When and Why do we use ANCOVA?
• Partitioning Variance
• Doing ANCOVA using SPSS
• Interpretation
– Main Effects
– Covariates
2 of 43
What is ANCOVA?
ANOVA can be extended to include one or more
continuous variables, termed covariates, that are
not part of the main experimental manipulation, but
have an influence on the DV.
ANCOVA tests whether manipulated factors (IVs)
have a significant effect on the DV after removing-or partialing out--the variance accounted for by the
covariates.
3 of 43
Accounting for Variance:
With and Without a Covariate
SSM
SSR
Systematic Variance
=
SSR without Covariate
SSM
SSR
Unsystematic Variance
SSR with Covariate
SSM
SSR
SSCov
Variance explained by the covariate
reduces the overall amount of
unexplained (error) variance.
4 of 43
Purpose of Covariates in ANOVA
Reduce error variance
–
The F-ratio in ANOVA compares the amount of variability explained by the
experimental manipulation (MSM), against the variability that it cannot explain
(MSR). What we hope is that the covariate explains some of the variance that was
previously unexplained. This has the effect of reducing the unexplained variance
(i.e., MSR becomes smaller) and so our F-ratio gets bigger. In real terms this
means we obtain a more sensitive measure of the experimental effect.
Eliminate Confounds
–
In any experiment, there may be unmeasured variables that confound the results
(i.e., a variable that varies systematically with the experimental manipulation). If
any variables are known to influence the dependent variable being measured, then
ANCOVA can help to remove the bias of these variables. Once a possible
confounding variable has been identified, it can be measured and entered into the
analysis as a covariate.
ANCOVA is like an ANOVA on the values of the DV, after removing the
influence of the covariate, rather than on the original values.
5 of 43
Example 1: Using ANCOVA to evaluate the
relative effectiveness of competing math textbooks
• An educational psychologist is interested in evaluating the relative
effectiveness of two textbooks for improving math skills.
• A literature review reveals that general IQ is related to math skills.
• She uses this information to make the test of the association between
math skills and textbook type more sensitive (i.e., more powerful).
• Procedure
– In each one of the two textbook groups, we compute the correlation
coefficient between IQ and math skills.
– We then estimate the amount of variation in math skills that is accounted
for by IQ, and the amount of residual variation (the variance in math
skills not explained by IQ).
– We use the residual variance in an ANOVA as an estimate of the true
unexplained variance (error) after controlling for IQ.
6 of 43
Example 2:
Evaluating the effectiveness of the
types of treatment for treating a fatal disease
• A researcher compares the effectiveness of three treatments ("Placebo",
"Drug 1", and "Drug 2”) for treating a fatal disease in terms of average survival
time.
• ANOVA could be used to analyze these data. But, suppose that
supplementary information is available to him (i.e., each patients’ age).
• Age in this case is a "covariate" - it is not related to treatment, but it
might be related to survival time.
• He uses the information to make the test of the association between
survival time and treatment type more sensitive (more powerful).
– In each of the treatment groups, he computes the correlation coefficient
between age and survival time.
– He estimates the amount of variation in survival time accounted for by age,
and the amount of residual variation that is not explained by age.
– He uses the residual variance in an ANOVA as an estimate of the true error
after controlling for age.
7 of 43
Choosing Covariates
In general, it is preferable to have a small
number of covariates.
Covariates should be correlated with the DV,
but NOT with each other.
– When covariates are significantly correlated:
•
•
•
they contribute very little to error reduction
they can cause computational difficulties such as
multicollinearity.
one or the other should be removed since they are
statistically redundant.
8 of 43
Assumptions of ANCOVA
ANCOVA has the same basic assumptions of all of
the parametric tests, but also has two additional
requirements:
1.independence of the covariate and treatment effect.
2.homogeneity of regression slopes.
9 of 43
Assumptions of ANCOVA:
Independence of the covariate and treatment effect
We assume the covariate is correlated with the outcome
variable (DV), so that as scores on the covariate change,
scores on the DV change by a similar amount.
Important to ensure that:
• the covariate explains previously unexplained variance in
the DV—not variance explained by the treatment.
• the effect that the covariate has on the outcome should
be the same for all of the experimental groups.
– Randomizing participants to experimental groups or matching
experimental groups on the covariate can help.
– Important to test this before running the ANCOVA.
10 of 43
Basic ANOVA showing that the total
variance in the DV can be partitioned
into treatment variance (SSM) and error
or unexplained variance (SSR).
Ideal situation for ANCOVA in which
the covariate shares variance with the
currently “unexplained” variance, but
is independent from the treatment
effect (no overlapping variance).
Situation in which ANCOVA should
NOT be used since the effect of the
covariate overlaps with the
experimental effect. Thus, the
experimental effect is confounded
with the effect of the covariate.
11 of 43
Assumptions of ANCOVA:
Homogeneity of Regression Slopes
What ANCOVA actually does is to create a single regression
equation, ignoring groups, to predict dependent variable (Y)
scores using the covariate(s) (X1, X2, X3). Then, in essence, an
ANOVA, not ignoring groups, is performed using the residualized
scores of each person computed by subtracting each person’s
predicted score based on the single equation, from the
participants’ actual scores.
These computations are legitimate if the regression equations
that predict the Y scores, computed separately in the ANOVA
groups, have parallel slopes. This is the homogeneity of
regression assumption, which requires that the “b” weight
applied to the covariates are reasonably equal across each
group.
12 of 43
Assumptions of ANCOVA:
Homogeneity of Regression Slopes
The slope of the line predicting the DV from the CV should
be relatively the same for each level of the IV.
– In other words the regression coefficient (bi) relating a particular
CV to the DV should be the same for each group.
– In still other words, this means no IV x DV interaction.
13 of 43
Sample ANCOVA Problem:
Viagra and Libido
Previously, we considered an example looking at the effects of
Viagra on libido. Viagra was shown to influence libido, but other
factors could also play a role in explaining a person’s libido,
including their health, mood, and their partner’s libido.
If one or more of these variables is measured, it is possible to control
for the influence they have on the dependent variable by including
them in the analysis.
In the context of hierarchical regression, we would enter one or more
of these variables into the model in the first block, followed by entry
of the “dummy variables” representing the experimental manipulation
in the second block. As such, we “partial out” the effect of the
covariates (e.g., their partner’s libido) to see what effect the
independent variable has after the effect of the covariate.
14 of 43
ViagraCovariate.sav
Dose
Participant’
s Libido
Partner’s
Libido
Placebo
3.22 (1.79)
3.44 (2.07)
Low Dose
4.88 (1.46)
3.12 (1.73)
High Dose
4.85 (2.12)
2.00 (1.63)
15 of 43
How Does ANCOVA Work?
• Imagine we had just two groups:
– Placebo
– Low Dose
• This paradigm can be expressed as a
regression equation using a dummy coding
variable:
Yi  b0  b1Xi
Libidoi  b0  b1Dos ei
16 of 43
Dummy Coding
– Placebo = 0, Low Dose = 1
– When Dose = Placebo, Predicted Libido = mean of
placebo group:
X Placebo  b1  0   b0
X Placebo  b0
– When Dose = Low Dose, Predicted Libido =
Difference between the means of the Placebo and
Low Dose groups:
X LowDose  b1  1  b0
X LowDose  b1  X Placebo
X LowDose  X Placebo  b1
17 of 43
ANCOVA and the GLM
• We can run a regression with Libido as the
outcome and the Dose (Placebo or Low) as the
predictor, Note:
– Intercept is the mean of Placebo group
– b for the Dummy Variable is the difference between
the means of the placebo and low dose group (4.883.22 = 1.66)
Coefficientsa
Model
1
(Constant)
Dummy Variable 1
(Placebo vs. Low)
Unstandardized
Coefficients
B
Std. Error
3.222
.547
1.653
.798
Standardized
Coefficients
Beta
.472
t
5.888
Sig .
.000
2.072
.056
a. Dependent Variable: Libido
18 of 43
ANCOVA as Regression
• The covariate can be added to the regression model
of the ANOVA.
• To evaluate the effect of the experimental
manipulation controlling for the covariate we enter
the covariate into the model first (think back to
hierarchical regression), then next the dummy coded
IV (dose of Viagra).
• Yi = b0 + b3Covariatei + b2Highi + b1Lowi + εi
• Yi = b0 + b3Partner’s Libidoi + b2Highi + b1Lowi + εi
19 of 43
SPSS Output:
ANCOVA as Regression
20 of 43
SPSS Output:
ANCOVA as Regression
21 of 43
ANCOVA Initial Considerations:
Testing Independence of the IV and Covariate
22 of 43
Initial Considerations:
One-Way ANOVA using Partner’s Libido as the DV
The non-significant F-test (p>.05)
tells us that the treatment groups
do not differ significantly on the
covariate (Partner’s libido).
23 of 43
ANCOVA: Main Analysis
24 of 43
ANCOVA: Contrasts
Note that the “Post-Hoc”
option isn’t available for
analyses using covariates.
Be sure to “click” on
the “Change” button!
25 of 43
ANCOVA: Options
You can perform “Post-Hoc”
tests through “Options”.
1.Move the IV, “Dose”, into the
“Display Means for” box.
2.“Click” on the “Compare
main effects” box.
3.The “Confidence interval
adjustment” option becomes
available.
4.The resulting output will be a
table of estimated marginal
means (the means adjusted for
the effect of the covariate).
26 of 43
SPSS Output:
Descriptive Statistics and Homogeneity test
27 of 43
Testing for Equal Variance:
Different Approaches
• Levene’s Test
– If Levene’s test is significant, a more stringent alpha can be
used (.01) or the variable can be dropped from the analysis.
– This is a very conservative test and is not necessarily the
best way to judge whether variances are unequal enough to
cause problems.
• Alternative Procedure: Highest and lowest variances
– Obtain standard deviation values for each of the groups;
square these values to obtain the variances.
– Take the largest value and divide it by the smallest value. If
the resulting value is less than 2, then we shouldn’t worry
too much. If it is larger than 2, then we do (see next slide).
28 of 43
The ratio of the highest variance
(4.49) to lowest variance (2.13) =
2.11. Look up the critical value for
10 subjects, and 3 variances.
Critical value = 5
29 of 43
SPSS Output:
Main Analysis
ANCOVA
ANOVA
30 of 43
The Main Effect
10
8
6
5.15
4.71
2.93
4
2
0
Placebo
Low
High
F(2, 26) = 4.14, p < .05
31 of 43
SPSS Output:
Parameter Estimates
Dummy Coded Variables
Dose = 1: Represents the difference between the Placebo and High Dose groups.
Dose = 2: Represents the difference between the Low Dose and High Dose groups.
Dose = 3: The High Dose reference group.
32 of 43
SPSS Output:
Planned Contrasts
Low dose vs.
Placebo
High dose vs.
Placebo
Adjusted
Means
33 of 43
SPSS Output:
Post-Hoc Tests
Note: Difference between Placebo and Low Dose groups is no longer significant (p>.05).
34 of 43
Assumption of Homogeneity of Regression Slopes
We assume the relationship between the DV and covariate is the same in each treatment group.
35 of 43
Re-run the ANCOVA:
Custom Model
The purpose: To test the Covariate by Outcome Interaction.
If the interaction is significant, the homogeneity of regression slopes has been
violated.
36 of 43
Step 1
Step 2
Step 3
37 of 43
SPSS Output:
Testing the Dose*Partner_Libido Interaction
38 of 43
Calculating Effect Size for ANCOVA
Eta squared (2 ) is essentially r2. Derived by dividing
SSM by SST. Assesses the proportion of total variance
explained by a variable. 2 = SS
Effect
SSTotal
Partial eta squared (partial 2 ) assesses the proportion
of variance that a variable explains that is not explained
by other variables in the analysis.
Partial 2 =
SSEffect
SSEffect + SSResidual
39 of 43
Calculating Effect Size for ANCOVA
Partial 2 Dose =
SSDose
SSDose + SSResidual
Partial 2 Partner Libido =
SSPartnerLibido
=
25.19
25.19 + 79.05
=
=
15.08
=
15.08 + 79.05
.24
.16
SSPartnerLibido + SSResidual
These values show that Dose explained a bigger proportion of the variance not
attributable to other variables than Partner_Libido
40 of 43
ANCOVA:
Reporting the Results
The covariate, partner’s libido, was significantly related
to the participant’s libido, F(1,26) = 4.96, p<.05, r = .40.
There was also a significant effect of Viagra on levels of
libido after controlling for the effect of partner’s libido,
F(2,26) = 4.14, p<.05, partial 2 = .24.
Planned contrasts revealed that having a high dose of
Viagra significantly increased libido compared to having
a placebo, t(26) = -2.77, p<.05, r=.48, but not compared
to having a low dose, t(26) = -0.54, p>.05, r = .11.
(see pp. 416-417 for specifics of results reporting).
41 of 43
Sample Problem #1:
Hangover Cures
A marketing manager for a well-known beverage manufacturer (AlkaSeltzer) believes his product is more effective than other recommended
hangover “cures.” To test his product against competitor cures he invites
15 people to join him at a bar and proceeds to get them drunk.
The next morning, he gives 5 of them strong coffee to drink (which is
assumed to have no beneficial effect), 5 of them Alka Seltzer, and 5 of
them Mimosas (champagne and orange juice—”hair of the dog that bit
them”). This variable is called “drink”. He measures how well they feel
two hours later on a Likert-type scale with the following anchors: 0 = “I
feel terrible” to 10 “I feel terrific”. This variable is called “well”.
The marketing manager realizes that it is important to control for how
drunk the person got the night before, and so he measures this on
another Likert-type scale with the following anchors: 0 = “Completely
sober” to 10 = “Completely intoxicated”. This variable is called “drunk”.
42 of 43
Hangover Cures Dataset
Strong Coffee
Alka Seltzer
Mimosas
Well
Drunk
Well
Drunk
Well
Drunk
5
5
5
6
5
2
5
3
4
6
6
3
6
2
6
4
6
2
6
1
8
2
6
3
3
7
6
3
6
2
Your task:
Conduct the appropriate analyses to see whether the drinks differ in
their ability to treat hangovers when controlling for how much was drunk
the night before. (Hint: Run the ANOVA first, then the ANCOVA).
Write up your results. Is Alka Seltzer better than the other cures? What
will you conclude from the study?
43 of 43