Multiple Regression

A single numerical response variable, Y.

Multiple numerical explanatory variables, X
1
, X
2
,…, X k
1

Simple Linear Regression
Y
Y


 y | x

0
 
 
1 x
 
2

3

Multiple Regression
Y
Y



Y | x
1
, x
2

0

,...,

1 x
1 x k

 

2 x
2

...
  k x k
 
4

5

Conditions



Independent
Identically distributed

6

Example

Y, Response – Effectiveness score based on experienced teachers’ evaluations.

Explanatory – Test 1, Test 2,
Test 3, Test 4.
7

Bivariate Fit of EVAL By Test1
700
600
500
400
300
200
0 50
Test1
100 150
8

Bivariate Fit of EVAL By Test2
700
600
500
400
300
200
110 120 130 140 150 160 170
Test2
9

Bivariate Fit of EVAL By Test3
700
600
500
400
300
200
30 40 50 60
Test3
70 80 90
10

Bivariate Fit of EVAL By Test4
700
600
500
400
300
200
35 40 45 50 55
Test4
60 65 70
11

Individual Tests

None of the individual tests
(explanatory variables) appears to be strongly related to the evaluation (response variable).
12

Method of Least Squares

Choose estimates of the various parameters in the multiple regression model so that the sum of squared residuals,
(SS
Error be.
), is the smallest it can
13

Method of Least Squares


Finding the estimates involves solving k+1 simultaneous equations with k+1 unknowns (the estimates of the parameters).
Do this with a statistical analysis computer package, like JMP.
14

JMP

Analyze – Fit Model

Pick Role Variables

Y – EVAL

Construct Model Effects

Add – Test1, Test2, Test3, Test4
15

JMP

Analyze – Fit Model

Personality – Standard Least
Squares

Emphasis – Minimal Report
16

Response EVAL
Summary of Fit
RSquare
RSquare Adj
Root Mean Square Error
Mean of Response
Observations (or Sum Wgts)
Analysis of Variance
Source
Model
Error
C. Total
DF
4
18
22
0.802861
0.759052
37.53627
444.4783
23
Sum of
Squares
103286.25
25361.49
128647.74
Mean Square
25821.6
1409.0
F Ratio
18.3265
Prob > F
<.0001*
Parameter Estimates
Term
Intercept
Test1
Test2
Test3
Test4
Estim ate
-193.4994
1.1158539
2.243267
-1.367001
6.0482387
Std Error
125.3074
0.319746
0.628449
0.563965
1.202281
t Ratio
-1.54
3.49
3.57
-2.42
5.03
Prob>|t|
0.1399
0.0026*
0.0022*
0.0261*
<.0001*
17

Prediction Equation

Predicted Evaluation = –193.50
+ 1.116*Test1 + 2.243*Test2
– 1.367*Test3 + 6.048*Test4
18