Topic 23: Diagnostics and
Remedies
Outline
• Diagnostics
– residual checks
• ANOVA remedial measures
Diagnostics Overview
• We will take the diagnostics and
remedial measures that we learned
for regression and adapt them to the
ANOVA setting
• Many things are essentially the same
• Some things require modification
Residuals
• Predicted values are cell
ˆ =Y
means, Y
ij
i.
• Residuals are the differences
between the observed values
and the cell means Yij- Yi.
Basic plots
• Plot the data vs the factor levels
(the values of the explanatory
variables)
• Plot the residuals vs the factor
levels
• Construct a normal quantile plot
and/or histogram of the residuals
KNNL Example
• KNNL p 777
• Compare 4 brands of rust inhibitor (X
has r=4 levels)
• Response variable is a measure of
the effectiveness of the inhibitor
• There are 10 units per brand (n=10)
Plots
• Data versus the factor
• Residuals versus the factor
• Normal quantile plot of the
residuals
Plots vs the factor
symbol1 v=circle i=none;
proc gplot data=a2;
plot (eff resid)*abrand;
run;
Data vs the factor
Means look different
…common spread in Y’s
Residuals vs the factor
Odd dist of points
QQ-plot
Due to odd
(lack of and
large)spread
Can try
nonparametric
analysis –
last slides
General Summary
• Look for
–Outliers
–Variance that depends on level
–Non-normal errors
• Plot residuals vs time and other
variables if available
Homogeneity tests
• Homogeneity of variance
(homoscedasticity)
• H0: σ12 = σ22 = … = σr2
• H1: not all σi2 are equal
• Several significance tests are
available
Homogeneity tests
• Text discusses Hartley, modified
Levene
• SAS has several including
Bartlett’s (essentially the
likelihood ratio test) and several
versions of Levene
Homogeneity tests
• There is a problem with assumptions
– ANOVA is robust with respect to
moderate deviations from Normality
– ANOVA results can be sensitive to
the homogeneity of variance
assumption
• Some homogeneity tests are sensitive
to the Normality assumption
Levene’s Test
• Do ANOVA on the squared residuals
from the original ANOVA
• Modified Levene’s test uses absolute
values of the residuals
• Modified Levene’s test is recommended
• Another quick and dirty rule of thumb
max( si2 ) / min( si2 )  2
KNNL Example
• KNNL p 785
• Compare the strengths of 5 types
of solder flux (X has r=5 levels)
• Response variable is the pull
strength, force in pounds required
to break the joint
• There are 8 solder joints per flux
(n=8)
Scatterplot
Levene’s Test
proc glm data=a1;
class type;
model strength=type;
means type/
hovtest=levene(type=abs);
run;
ANOVA Table
Sum of
DF
Squares
4 353.612085
35 73.7988250
Source
Model
Error
Corrected Total 39 427.410910
Mean
Square F Value Pr > F
88.4030213 41.93 <.0001
2.1085379
Common variance estimated to be 2.11
Output
Levene's Test
ANOVA of Absolute Deviations
Source DF
type
4
Error 35
F Value
3.07
Pr > F
0.0288
We reject the null hypothesis and
assume nonconstant variance
Means and SDs
Level
type
1
2
3
4
5
N
8
8
8
8
8
strength
Mean Std Dev
15.42
1.23
18.52
1.25
15.00
2.48
9.74
0.81
12.34
0.76
Remedies
• Delete outliers
– Is their removal important?
• Use weights (weighted regression)
• Transformations
• Nonparametric procedures
What to do here?
• Not really any obvious outliers
• Do not see pattern of increasing or
decreasing variance or skewed dists
• Will consider
– Weighted ANOVA
– Mixed model ANOVA
Weighted least squares
• We used this with regression
–Obtain model for how the sd
depends on the explanatory
variable (plotted absolute value
of residual vs x)
–Then used weights inversely
proportional to the estimated
variance
Weighted Least Squares
• Here we can compute the
variance for each level
• Use these as weights in PROC
GLM
• We will illustrate with the
soldering example from KNNL
Obtain the variances and
weights
proc means data=a1;
var strength;
by type;
output out=a2 var=s2;
data a2; set a2; wt=1/s2;
NOTE. Data set a2 has 5 cases
Proc Means Output
strength
Level of
type
1
N
Mean
Std Dev
8 15.4200000 1.23713956
2
8 18.5275000 1.25297076
3
8 15.0037500 2.48664397
4
8
5
8 12.3400000 0.76941536
9.7412500 0.81660337
Merge and then use the
weights in PROC GLM
data a3; merge a1 a2;
by type;
proc glm data=a3;
class type;
model strength=type;
weight wt;
lsmeans type / cl;
run;
Output
Sum of
DF
Squares Mean Square F Value Pr > F
4 324.213099 81.0532747
81.05 <.0001
Source
Model
Error
Corrected Total
35 35.0000000
39 359.213099
1.0000000
Data have been standardized to
have a variance of 1
LSMEANS Output
Because of weights, standard errors simply based
on sample variances of each level
type
1
strength
LSMEAN
15.4200000
Standard
Error
0.4373949
95% Confidence
Pr > |t|
Limits
<.0001 14.532041 16.307959
2
18.5275000
0.4429921
<.0001 17.628178 19.426822
3
15.0037500
0.8791614
<.0001 13.218957 16.788543
4
9.7412500
0.2887129
<.0001 9.155132 10.327368
5
12.3400000
0.2720294
<.0001 11.787751 12.892249
Mixed Model ANOVA
• Relax the assumption of constant
variance rather than including a
“known” weight
• This involves moving to a mixed
model procedure
• Topic will not be on exam but wanted
you to be aware of these model
capabilities
SAS Code
proc glimmix data=a1;
class type;
model strength=type / ddfm=kr;
random residual / group=type;
run;
This allows the variance to
differ in each level and a
degrees of freedom adjustment
is used to account for this
GLIMMIX OUTPUT
Fit Statistics
-2 Res Log Likelihood
AIC (smaller is better)
AICC (smaller is better)
BIC (smaller is better)
CAIC (smaller is better)
HQIC (smaller is better)
Generalized Chi-Square
Gener. Chi-Square / DF
122.11
132.11
134.18
139.88
144.88
134.79
35.00
1.00
Covariance Parameter Estimates
Standard
Cov Parm
Group Estimate
Error
Residual (VC) type 1
1.5305
0.8181
Residual (VC) type 2
1.5699
0.8392
Residual (VC) type 3
6.1834
3.3052
Residual (VC) type 4
0.6668
0.3564
Residual (VC) type 5
0.5920
0.3164
Type III Tests of Fixed Effects
Num Den
Effect
DF
DF F Value Pr > F
type
4 14.81
71.78 <.0001
Really 3 groups
of variances
SAS Code
proc glimmix data=a1;
class type;
model strength=type / ddfm=kr;
random residual / group=type1;
run;
Type1 was created to identify
Type 1 and 2, Type 3, and Type
4 and 5 as 3 groups
GLIMMIX OUTPUT
Fit Statistics
-2 Res Log Likelihood
AIC (smaller is better)
AICC (smaller is better)
BIC (smaller is better)
CAIC (smaller is better)
HQIC (smaller is better)
Generalized Chi-Square
Gener. Chi-Square / DF
122.13
128.13
128.91
132.80
135.80
129.74
35.00
1.00
Covariance Parameter Estimates
Standard
Cov Parm
Group Estimate
Error
Residual (VC) Grp 1
1.5502
0.5859
Residual (VC) Grp 2
6.1834
3.3052
Residual (VC) Grp 3
0.6294
0.2379
Type III Tests of Fixed Effects
Num Den
Effect
DF
DF F Value Pr > F
type
4 19.8
77.68 <.0001
Better BIC but
same general type
conclusion
Transformation Guides
• When σi2 is proportional to μi, use Y
• When σi is proportional to μi, use
log(y)
• When σi is proportional to μi2, use 1/y
• For proportions, use arcsin( Y )
–arsin(sqrt(y)) in a SAS data step
• Box-Cox transformation
Example
• Consider study on KNNL pg 790
• Y: time between computer failures
• X: three locations
data a3;
infile 'u:\.www\datasets512\CH18TA05.txt';
input time location interval;
symbol1 v=circle;
proc gplot;
plot time*location;
run;
Scatterplot
Outlier or skewed
distribution? Can consider
transformation first
Box-Cox Transformation
• Can consider regression
log( si ) vs log( yi )
and 1-b1 is the power to raise Y by
• Can try various “convenient” powers
• Can use SAS directly to calculate the
power
E(logsig) = 0.90 + .79 logmu
Power should be 1-.79 ≈ 0.20
Using SAS
proc transreg data=a3;
model boxcox(time / lambda=-2 to 2
by .2) = class(location);
run;
Box-Cox Transformation Information for time
Lambda
R-Square
Log Like
-1.0
0.24
-73.040
-0.8
0.27
-67.330
-0.6
0.31
-62.316
-0.5
0.33
-60.144
-0.4
0.35
-58.239
-0.2
0.38
-55.346 *
0.0 +
0.39
-53.830 *
0.2
0.38
-53.769 <
0.4
0.36
-55.118 *
0.5
0.34
-56.273
0.6
0.32
-57.712
0.8
0.29
-61.314
1.0
0.25
-65.675
< - Best Lambda
* - 95% Confidence Interval
+ - Convenient Lambda
Output
Transforming data in SAS
data a3;
set a3;
transtime = time**0.20;
symbol1 v=circle i=none;
proc gplot;
plot transtime*location;
run;
Much more constant
spread in data!
Nonparametric approach
• Based on ranks
• See KNNL section 18.7, p 795
• See the SAS procedure
NPAR1WAY
Rust Inhibitor Analysis
Source
Model
Error
Corrected Total
DF
3
36
39
Sum of
Squares Mean Square F Value Pr > F
15953.4660
5317.82200 866.12 <.0001
221.03400
6.13983
16174.5000
Highly significant F test. Even
if there is a violation of
Normality, the evidence is
overwhelming
Nonparametric Analysis
Wilcoxon Scores (Rank Sums) for Variable eff
Classified by Variable abrand
Sum of Expected
Std Dev
abrand
N Scores Under H0 Under H0
1
10
128.0
205.0 32.014119
2
10
355.0
205.0 32.014119
3
10
255.0
205.0 32.014119
4
10
82.0
205.0 32.014119
Average scores were used for ties.
Kruskal-Wallis Test
Chi-Square
33.7041
DF
Pr > Chi-Square
3
<.0001
Mean
Score
12.80
35.50
25.50
8.20
Last slide
• We’ve finished most of
Chapters 17 and 18.
• We used program topic23.sas
to generate the output.