No Intercept Regression and
Analysis of Variance
Example Data Set
Y
X
5
20
6
23
7
27
8
33
8
31
9
35
10
43
5
19
6
25
7
29
8
31
Estimate two models
• Model with y-intercept
Y=a+b*X
• Model no y-intercept
Y=b*X
Regression Statistics
Multiple R
0.984
Regression Statistics
Multiple R
0.999
R Square
Adjusted R Square
Standard Error
R Square
Adjusted R Square
Standard Error
Observations
0.969
0.965
0.300
11
Observations
0.998
0.898
0.333
11
Observations
• The model with a y-intercept is more complex
than the model with no y-intercept.
• One would expect then that the R2 of the model
would decline when the y-intercept is removed.
BUT, the R2 actually increases.
• If the explanatory power of the model, R2,
increases, then the error of the model, Standard
Error, should decrease. But, the Standard Error
actually increases.
Analysis of Variance Table
model with y-intercept
ANOVA
df
SS
MS
24.83 276.72
Regression
1
24.83
Residual
9
0.81
10
25.64
Total
0.09
F
Significance F
0.000000
Analysis of Variance Table
model no y-intercept
ANOVA
df
Regression
SS
MS
F
1 591.89 591.89 5338.74
Residual
10
1.11
Total
11 593.00
0.11
Significance F
0.000000
Comparison of Sum of Squares
Y-intercept
df
SS
Regression
1
24.83
Residual
9
0.81
10
25.64
Total
No yintercept
Regression
df
SS
1
591.89
Residual
10
1.11
Total
11
593.00
Revision of Sum of Squares
for no-intercept model
This is re-calculated.
No y-intercept
df
SS
Regression
1
SST – SSE
25.64 – 1.11 = 24.53
Residual
9
1.11
10
25.64
Total
These are from the model with a y-intercept.
Comparison of Revised Sum of
Squares
Y-intercept
df
SS
Regression
1
24.83
Residual
9
0.81
10
25.64
Total
No yintercept
df
SS
Regression
1
24.53
Residual
9
1.11
10
25.64
Total
Revised Statistics
• Model no y-intercept
Y=b*X
Regression Statistics
Multiple R
0.999
R Square
Adjusted R Square
Standard Error
Observations
0.998
0.898
0.333
11
SSR / SST
= 24.53 / 25.64
= 0.957
√ SSE / d.o.f.
= √ 1.11 / 9
= 0.351
Comparison of Revised Statistics
• Model with y-intercept
Y=a+b*X
• Model no y-intercept
Y=b*X
Regression Statistics
Multiple R
0.984
Regression Statistics
Multiple R
0.999
R Square
Adjusted R Square
Standard Error
R Square
Adjusted R Square
Standard Error
Observations
0.969
0.965
0.300
11
Observations
0.957
0.898
0.351
11
Revised Observations
• The model with a y-intercept is more complex
than the model with no y-intercept.
• One would expect then that the R2 of the model
would decline when the y-intercept is removed.
BUT, the R2 actually increases.
• If the explanatory power of the model, R2,
increases, then the error of the model, Standard
Error, should decrease. But, the Standard Error
actually increases.
Analysis of Variance Table
model no y-intercept
REVISED
SS / df
ANOVA
df
Regression
Residual
Total
1
SS
MS
24.53
24.53
9
1.11
10
25.64
0.123
F
Significance F
199.43
0.000000
FDIST
(199.43,
1,9)
MSR/ MSE
Summary
When comparing a model with a y-intercept to the same
model without a y-intercept
1. Revise the ANOVA table for the no-intercept model with
values from the y-intercept model (X).
2. Recalculate necessary items (Y), and the R2 and the
Standard Error.
ANOVA
df
SS
MS
F
Regression
1
Y
Y
Residual
X
1.11
Y
Total
X
X
Significance F
Y
Y