Overview of Techniques
Case 1 Independent Variable is Groups, or Conditions
Dependent Variable is continuous ( X )
One sample: Z-test or t-test
Two samples: T-test (independent or paired)
Three samples: One-way ANOVA F-test
Factorial design: Two-Way ANOVA F-test
Overview of Techniques
Case 2 Independent Variable is continuous (x)
Dependent Variable is continuous (y)
One DV, one predictor: correlation, simple linear regression
One DV, multiple predictors: partial correlation, multiple
correlation, multiple regression
What if we have two predictor variables?
We want to predict depression. We have measured stress and loneliness.
We can ask several questions:
1) which is the stronger predictor?
2) how well do they predict depression together?
3) what is the effect of loneliness on depression, controlling for stress?
What if we have
twobetter
predictor
variables?
is the
predictor?
How wellWhich
do they
predict
depression
together?
Regressing depression on stress
Predictor
Stress
Unstandardized
Coefficient
R2 = .0625
Standard Standardized
error
Coefficient
.50
.16
Regressing depression on loneliness
Predictor
Loneliness
Unstandardized
Coefficient
.80
t
sig
.25
3.125
<.05
R2 = .04
Standard Standardized
error
Coefficient
.28
.20
t
sig
2.86
<.05
Multiple Correlation
How well do they predict depression together?
depression
R2
loneliness
How well do they predict depression together?
depression
R2
stress
How well do they predict depression together?
depression
(c)
(a)
(b)
loneliness
Multiple R2: (a) + (b) + (c)
stress
Pearson’s R2 for loneliness: (a) + (b)
Pearson’s R2 for stress: (c) + (b)
Partial Correlation
What is the effect of loneliness controlling for stress?
depression
(c)
(a)
(b)
loneliness
stress
Partial R2 for loneliness: (a)
Pearson’s R2 for loneliness: (a) + (b)
Partial R2 for stress: (b)
Pearson’s R2 for stress: (c) + (b)
Multiple Regression
Types of effects
depression
(a)
loneliness
(c)
(b)
stress
Slope coefficients in simple
regression capture total effects
Total effect of stress: (b) + (c)
Unique effect of stress: (c)
Slope coefficients in multiple
regression capture unique effects
Shared effect of stress and loneliness: (b)
Reasons for Multiple Regression
depression
(a)
loneliness
(c)
(b)
stress
1) It allows you to directly compare the effect sizes for different predictor
variables
2) Adding additional predictors that are related to your Y variable (we call them
covariates) allows you to explain more of the residual variance. This makes MS
error smaller and increases your power.
2) If you are worried that your key predictor is confounded with other variables,
you can “partial them out” or “control for them” in your multiple regression by
including them in the analysis.
Two separate regressions
Regressing depression on stress
Predictor
Stress
Unstandardized
Coefficient
R2 = .0625
Standard Standardized
error
Coefficient
.50
.16
Regressing depression on loneliness
Predictor
Loneliness
Unstandardized
Coefficient
.80
t
sig
.25
3.125
<.05
R2 = .04
Standard Standardized
error
Coefficient
.28
.20
t
sig
2.86
<.05
A multiple regression
Multiple R2 = .0625
Regressing depression on loneliness and stress
Predictor
Unstandardized
Partial
Coefficient
Standard Standardized
error
Partial
Coefficient
Intercept
1.8
1.1
-
1.64
.08
Stress
.34
.11
.17
3.09
<.05
Loneliness
.10
.05
.05
2.00
.06
Y '  a  b1 X1  b2 X 2
Y '  1.8  .34Stress  .10 Lonely
t
sig
df = n – p - 1
Multiple Regression ANOVA
Source
SS
Model
 (Y 'Y )
Y 'Y
Error
Y Y'
Total
Y Y
s2
df
2
 (Y  Y ' )
 (Y  Y )
p
2
n  p 1
2
n 1
The F-test is for the whole model,
doesn’t tell you about individual predictors
s
2
model
2
smodel
F 2
sresid
2
sresid
sY2
2
s
multiple R 2  model
sY2
Categorical Predictors
in Multiple Regression
A dichotomous 0/1 predictor
Regressing depression on gender (0=female, 1=male)
Gender
depression
0
8
1
4
1
10
0
15
1
8
0
14
A dichotomous 0/1 predictor
Regressing depression on gender (0=female, 1=male)
Predictor
Unstandardized
Partial
Coefficient
Standard Standardized
error
Partial
Coefficient
t
sig
Intercept
12.33
1.99
-
6.2
<.01
Gender
-5.00
2.81
.66
-1.8
.15
The intercept coefficient tells you the mean depression of the 0 (female)
group
The gender coefficient tells you what to add to get the mean depression
of the 1 (male) group
If the gender coefficient is significant, the groups significantly differ
Categorical and Continuous Predictors
in Multiple Regression
Combining Types of Predictors
T-tests and ANOVAs use group variables to predict continuous outcomes
Correlations and simple regressions use continuous variables to predict
continuous outcomes
Multiple regressions allow you to use 1) information about group membership
and 2) information about other continuous measurements, in the same analysis
Combining Types of Predictors
WHY would we want this?
Imagine that we have a control group and a highly-provoked group, and we also
measure the “TypeA-ness” of each participant.
We noticed that because of streaky random sampling, we got more TypeA
people in the control group than in the provoked group.
Multiple regression allows us to see if there was an effect of our manipulation,
controlling for individual differences in TypeA-ness.
Basically, it allows us to put a situational manipulation and a personality scale
measurement into the same study.
Group
Provoke
TypeA
aggression
Control
0
3
3
Control
0
6
4
Control
0
10
6
Control
0
8
5
High
1
2
9
High
1
4
10
High
1
11
24
High
1
12
20
Predictor
Unstandardized
Partial
Coefficient
Intercept
-3.26
2.23
-
-1.46
.20
Provoke
10.675
1.89
.734
5.65
<.01
1.15
.26
.565
4.35
<.01
Type A
Standard Standardized
error
Partial
Coefficient
t
sig
There is a significant effect of experimental condition and a significant
effect of TypeA-ness
General Linear Model
General Linear Model
All of the techniques we’ve covered so far can be expressed as special cases of
multiple regression
If you run a multiple regression with an intercept and no slope, the t-test for the
intercept is the same as a single sample t-test.
If you put in a dichotomous (0/1) predictor, the t-test for your slope will be the
same as an independent samples t-test.
If you put in dummy variables for multiple groups, your regression ANOVA will be
the same as your one-way ANOVA or two-way ANOVA.
If you put in one continuous predictor, your β will be the same as your r.
General Linear Model
Plus multiple regression can do so much more!
Looking at several continuous predictors together in one model.
Controlling for confounds.
Using covariates to “soak up” residual variance.
Looking at categorical and continuous predictors together in one model.
Looking at interactions between categorical and continuous variables.