Logistic Regression in R
In this section of the notes we examine logistic regression in R. There are several
functions that I wrote for plotting diagnostics similar to what SAS does, although the
inspiration for them came from work Prof. Malone and I did for OLS as part of his senior
project.
Example 1: Oral Contraceptive Use and Myocardial Infarctions
Set up a text file with the data in columns with variable names at the top. The case and control
counts are in separate columns. The risk factor OC use and stratification variable Age follow.
> OCMI.data = read.table(file.choose(),header=T)
# read in text file
> OCMI.data
MI NoMI Age OCuse
1
4
62
1
Yes
2
2 224
1
No
3
9
33
2
Yes
4 12 390
2
No
5
4
26
3
Yes
6 33 330
3
No
7
6
9
4
Yes
8 65 362
4
No
9
6
5
5
Yes
10 93 301
5
No
> attach(OCMI.data)
> OC.glm <- glm(cbind(MI,NoMI)~Age+OCuse,family=binomial)
# fit model
> summary(OC.glm)
Call:
glm(formula = cbind(MI, NoMI) ~ Age + OCuse, family = binomial)
Deviance Residuals:
[1]
0.456248 -0.520517
[9] -0.045061
0.008822
1.377693
-0.886710
-1.685521
Coefficients:
Estimate Std. Error z value
(Intercept) -4.3698
0.4347 -10.054
Age2
1.1384
0.4768
2.388
Age3
1.9344
0.4582
4.221
Age4
2.6481
0.4496
5.889
Age5
3.1943
0.4474
7.140
OCuseYes
1.3852
0.2505
5.530
--Signif. codes: 0 `***' 0.001 `**' 0.01
0.714695
Pr(>|z|)
< 2e-16
0.0170
2.43e-05
3.88e-09
9.36e-13
3.19e-08
-0.130922
0.033643
***
*
***
***
***
***
`*' 0.05 `.' 0.1 ` ' 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 158.0085
Residual deviance:
6.5355
AIC: 58.825
on 9
on 4
degrees of freedom
degrees of freedom
54
Number of Fisher Scoring iterations: 3
Find OR associated with oral contraceptive use ADJUSTED for age.
Recall: CMH procedure gave 3.97.
> exp(1.3852)
[1] 3.995625
Find a 95% CI for OR associated with OC use.
> exp(1.3852-1.96*.2505)
[1] 2.445428
> exp(1.3852+1.96*.2505)
[1] 6.528518
Interpreting the age effect in terms of OR’s ADJUSTING for OC use.
Note: The reference group is Age = 1 which was women 25 – 29 years of age.
> OC.glm$coefficients
(Intercept)
Age2
-4.369850
1.138363
Age3
1.934401
Age4
2.648059
Age5
3.194292
OCuseYes
1.385176
> Age.coefs <- OC.glm$coefficients[2:5]
> exp(Age.coefs)
Age2
Age3
Age4
Age5
3.121653 6.919896 14.126585 24.392906
Find 95% CI for age = 5 group.
> exp(3.1943-1.96*.4474)
[1] 10.14921
> exp(3.1943+1.96*.4474)
[1] 58.62751
Example 2: Coffee Drinking and Myocardial Infarctions
CoffeeMI.data = read.table(file.choose(),header=T)
> CoffeeMI.data
Smoking Coffee MI NoMI
1
Never
> 5 7
31
2
Never
< 5 55 269
3
Former
> 5 7
18
4
Former
< 5 20 112
5
1-14 Cigs
> 5 7
24
6
1-14 Cigs
< 5 33 114
7 15-25 Cigs
> 5 40
45
8 15-25 Cigs
< 5 88 172
9 25-34 Cigs
> 5 34
24
10 25-34 Cigs
< 5 50
55
11 35-44 Cigs
> 5 27
24
12 35-44 Cigs
< 5 55
58
13
45+ Cigs
> 5 30
17
14
45+ Cigs
< 5 34
17
> attach(CoffeeMI.data)
> Coffee.glm = glm(cbind(MI,NoMI)~Smoking+Coffee,family=binomial)
55
> summary(Coffee.glm)
Call:
glm(formula = cbind(MI, NoMI) ~ Smoking + Coffee, family = binomial)
Deviance Residuals:
Min
1Q
Median
-0.7650 -0.4510 -0.0232
3Q
0.2999
Max
0.7917
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)
-1.2981
0.1819 -7.136 9.60e-13 ***
Smoking15-25 Cigs
0.6892
0.2119
3.253 0.00114 **
Smoking25-34 Cigs
1.2462
0.2398
5.197 2.02e-07 ***
Smoking35-44 Cigs
1.1988
0.2389
5.017 5.24e-07 ***
Smoking45+ Cigs
1.7811
0.2808
6.342 2.27e-10 ***
SmokingFormer
-0.3291
0.2778 -1.185 0.23616
SmokingNever
-0.3153
0.2279 -1.384 0.16646
Coffee> 5
0.3200
0.1377
2.324 0.02012 *
--Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 173.7899
Residual deviance:
3.7622
AIC: 84.311
on 13
on 6
degrees of freedom
degrees of freedom
Number of Fisher Scoring iterations: 3
OR for drinking 5 or more cups of coffee per day.
Note: CMH procedure gave OR = 1.375
> exp(.3200)
[1] 1.377128
95% CI for OR associated with heavy coffee drinking
> exp(.3200 - 1.96*.1377)
[1] 1.051385
> exp(.3200 + 1.96*.1377)
[1] 1.803794
Reordering a Factor
To examine the effect of smoking we might want to “reorder” the levels of smoking
status so that individuals who have never smoked are used as the reference group. To do
this in R you must do the following:
Smoking = factor(Smoking,levels=c("Never","Former","1-14 Cigs","15-25
Cigs","25-34 Cigs","35-44 Cigs","45+ Cigs"))
The first level specified in the levels subcommand will be used as the reference group,
“Never” in this case. Refitting the model with the reordered smoking status factor gives
the following:
56
> Coffee.glm2 <-glm(cbind(MI,NoMI)~Smoking+Coffee,family=binomial)
> summary(Coffee.glm2)
Call:
glm(formula = cbind(MI, NoMI) ~ Smoking + Coffee, family = binomial)
Deviance Residuals:
Min
1Q
Median
3Q
Max
-0.7650 -0.4510 -0.0232
0.2999
0.7917
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)
-1.61344
0.14068 -11.469 < 2e-16 ***
SmokingFormer
-0.01376
0.25376 -0.054
0.9568
Smoking1-14 Cigs
0.31533
0.22789
1.384
0.1665
Smoking15-25 Cigs 1.00451
0.17976
5.588 2.30e-08 ***
Smoking25-34 Cigs 1.56150
0.21254
7.347 2.03e-13 ***
Smoking35-44 Cigs 1.51417
0.21132
7.165 7.77e-13 ***
Smoking45+ Cigs
2.09646
0.25855
8.108 5.13e-16 ***
Coffee> 5
0.31995
0.13766
2.324
0.0201 *
--Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 173.7899
Residual deviance:
3.7622
AIC: 84.311
on 13
on 6
degrees of freedom
degrees of freedom
Number of Fisher Scoring iterations: 3
Notice that “SmokingNever” is now absent from the output so we know it is being used
as the reference group. The OR’s associated with the various levels of smoking are
computed below.
> Smoke.coefs = Coffee.glm$coefficients[2:7]
> exp(Smoke.coefs)
SmokingFormer Smoking1-14 Cigs Smoking15-25 Cigs Smoking25-34 Cigs
0.986338
1.370715
2.730561
4.765984
Smoking35-44 Cigs
Smoking45+ Cigs
4.545632
8.137279
Confidence intervals for each could be computed in the standard way.
57
Some Details for Categorical Predictors with More Than Two Levels
Consider the coffee drinking/MI study above. The stratification variable smoking has
seven levels. Thus it requires six dummy variables to define it. The level that is not
defined using a dichotomous dummy variable serves as the reference group. The table
below shows how the value of the dummy variables:
Level
D2
D3
D4
D5
D6
D7
Never
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
(Reference
Group)
Former
1 – 14 Cigs
15 – 24 Cigs
25 – 34 Cigs
35 – 44 Cigs
45+ Cigs
Example: Coffee Drinking and Myocardial Infarctions
CoffeeMI.data = read.table(file.choose(),header=T)
> CoffeeMI.data
Smoking Coffee MI NoMI
1
Never
> 5 7
31
2
Never
< 5 55 269
3
Former
> 5 7
18
4
Former
< 5 20 112
5
1-14 Cigs
> 5 7
24
6
1-14 Cigs
< 5 33 114
7 15-25 Cigs
> 5 40
45
8 15-25 Cigs
< 5 88 172
9 25-34 Cigs
> 5 34
24
10 25-34 Cigs
< 5 50
55
11 35-44 Cigs
> 5 27
24
12 35-44 Cigs
< 5 55
58
13
45+ Cigs
> 5 30
17
14
45+ Cigs
< 5 34
17
The Logistic Model
  ( x) 
     Coffee   D   D   D   D   D   D
~
ln 
o
1
2 2
3 3
4 4
5 5
6 6
7 7
 1   ( x) 
~


where Coffee is a dichotomous predictor equal to 1 if they drink 5 or more cups of coffee
per day.
Comparing the log-odds of a heavy coffee drinker who who smokes 15-25 cigarettes day
to a heavy coffee drinker who has never smoked we have.
58
 1 ( x) 
~
    
ln 
o
1
4
 1  1 ( x) 
~ 

  2 ( x) 
~
   
ln 
o
1
 1   2 ( x) 
~ 

Taking the difference gives,
 1 ( x) 
~


 1  1 ( x) 
~ 
ln 
 4
  2 ( x~ ) 


 1   2 ( x) 
~ 

thus
e 4  the odds ratio associated with smoking 15-24 cigarettes per day when compared to
individuals who have never smoked amongst heavy coffee drinkers. Because 1 is not
involved in the odds ratio the result is the same for non-heavy coffee drinkers as well!
You can also consider combinations of factors, e.g. if we compared heavy coffee drinkers
who smoked 15-24 cigarettes to a non-heavy coffee drinkers who have never smoked the
associated OR would be given by e1  4 .
Using our fitted model the OR’s ratios discussed above would be.
> summary(Coffee.glm)
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)
-1.61344
0.14068 -11.469 < 2e-16 ***
SmokingFormer
-0.01376
0.25376 -0.054
0.9568
Smoking1-14 Cigs
0.31533
0.22789
1.384
0.1665
Smoking15-25 Cigs 1.00451
0.17976
5.588 2.30e-08 ***
Smoking25-34 Cigs 1.56150
0.21254
7.347 2.03e-13 ***
Smoking35-44 Cigs 1.51417
0.21132
7.165 7.77e-13 ***
Smoking45+ Cigs
2.09646
0.25855
8.108 5.13e-16 ***
Coffee> 5
0.31995
0.13766
2.324
0.0201 *
--Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
OR for 15-24 cigarette smokers vs. never smokers (regardless of coffee drinking status)
> exp(1.00451)
[1] 2.730569
59
OR for 15-24 cigarette smokers who are also heavy coffee drinkers vs. non-smokers who
are not heavy coffee drinkers
> exp(.31995 + 1.00451)
[1] 3.760154
Similar calculations could be done for other combinations of coffee and cigarette use.
Example 3: Risk Factors for Low Birth Weight
Response
Y = low birth weight, i.e. birth weight < 2500 grams (1 = yes, 0 = no)
Set of potential predictors
X1
X2
X3
X4
X5
X6
X7
=
=
=
=
=
=
=
previous history of premature labor (1 = yes, 0 = no)
hypertension during pregnancy (1 = yes, 0 = no)
smoker (1 = yes, 0 = no)
uterine irritability (1 = yes, 0 = no)
minority (1 = yes, 0 = no)
mother’s age in years
mother’s weight at last menstrual cycle
Analysis in R
> Lowbirth = read.table(file.choose(),header=T)
> Lowbirth[1:5,]
# print first 5 rows of the data set
Low Prev Hyper Smoke Uterine Minority Age Lwt race bwt
1
0
0
0
0
1
1 19 182
2 2523
2
0
0
0
0
0
1 33 155
3 2551
3
0
0
0
1
0
0 20 105
1 2557
4
0
0
0
1
1
0 21 108
1 2594
5
0
0
0
1
1
0 18 107
1 2600
Make sure categorical variables are interpreted as factors by using the factor command
>
>
>
>
>
>
Low = factor(Low)
Prev = factor(Prev)
Hyper = factor(Hyper)
Smoke = factor(Smoke)
Uterine = factor(Uterine)
Minority = factor(Minority)
Note: This is not really necessary for dichotomous variables that are coded (0,1).
Fit a preliminary model using all available covariates
> low.glm = glm(Low~Prev+Hyper+Smoke+Uterine+Minority+Age+Lwt,family=binomial)
> summary(low.glm)
Call:
glm(formula = Low ~ Prev + Hyper + Smoke + Uterine + Minority +
Age + Lwt, family = binomial)
Deviance Residuals:
Min
1Q
Median
-1.6010 -0.8149 -0.5128
3Q
1.0188
Max
2.1977
60
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.378479
1.170627
0.323 0.74646
Prev1
1.196011
0.461534
2.591 0.00956 **
Hyper1
1.452236
0.652085
2.227 0.02594 *
Smoke1
0.959406
0.405302
2.367 0.01793 *
Uterine1
0.647498
0.466468
1.388 0.16511
Minority1
0.990929
0.404969
2.447 0.01441 *
Age
-0.043221
0.037493 -1.153 0.24900
Lwt
-0.012047
0.006422 -1.876 0.06066 .
--Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
Null deviance: 232.40
Residual deviance: 196.71
AIC: 212.71
on 185
on 178
degrees of freedom
degrees of freedom
Number of Fisher Scoring iterations: 3
It appears that both uterine irritability and mother’s age are not significant. We can fit
the reduced model eliminating both terms and test whether the model is significantly
degraded by using the general chi-square test (see the SAS example).
> low.reduced = glm(Low~Prev+Hyper+Smoke+Minority+Lwt,family=binomial)
> summary(low.reduced)
Call:
glm(formula = Low ~ Prev + Hyper + Smoke + Minority + Lwt, family =
binomial)
Deviance Residuals:
Min
1Q
Median
-1.7277 -0.8219 -0.5368
3Q
0.9867
Max
2.1517
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.261274
0.885803 -0.295 0.76803
Prev1
1.181940
0.444254
2.661 0.00780 **
Hyper1
1.397219
0.656271
2.129 0.03325 *
Smoke1
0.981849
0.398300
2.465 0.01370 *
Minority1
1.044804
0.394956
2.645 0.00816 **
Lwt
-0.014127
0.006387 -2.212 0.02697 *
--Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 232.40
Residual deviance: 200.32
AIC: 212.32
on 185
on 180
degrees of freedom
degrees of freedom
Number of Fisher Scoring iterations: 3
61
  ( x) 
~
    X  X  X  X  X
H o : ln 
o
1 1
2
2
3
3
5
5
7
7
 1   ( x) 
~ 

  ( x) 
~
    X  X  X  X  X  X  X
H 1 : ln 
o
1 1
2
2
3
3
4
4
5
5
6
6
7
7
 1   ( x) 
~ 

* Recall:  ( x)  P( Low  1 | X )
~
~
DH o  200.32
df = 180
Residual Deviance Alternative Hypothesis Model: DH1  196.71
df = 178
Residual Deviance Null Hypothesis Model:
General Chi-Square Test
 2  DH 0  DH1  200.32  196.71  3.607
p  value  P(  2  3.607)  .1647
Fail to reject the null, the reduced model is adequate.
2
Interpretation of Model Parameters
OR’s Associated with Categorical Predictors
> low.reduced
Call: glm(formula = Low ~ Prev + Hyper + Smoke + Minority + Lwt,
family = binomial)
Coefficients:
(Intercept)
Prev1
-0.26127 1.18194
Hyper1
1.39722
Smoke1
0.98185
Degrees of Freedom: 185 Total (i.e. Null);
Null Deviance:
232.4
Residual Deviance: 200.3
AIC: 212.3
Minority1
1.04480
Lwt
-0.01413
180 Residual
Estimated OR’s
> exp(low.reduced$coefficients[2:5])
Prev1
Hyper1
Smoke1 Minority1
3.260693 4.043938 2.669388 2.842841
95% CI for OR Associated with History of Premature Labor
> exp(1.182 - 1.96*.444)
[1] 1.365827
> exp(1.182 + 1.96*.444)
[1] 7.78532
Holding everything else constant we estimate that the odds of having an infant with low birth
weight are between 1.366 and 7.785 times larger for mothers with a history of premature labor.
62
95% CI for OR Associated with Hypertension
> exp(1.397 - 1.96*.6563)
[1] 1.117006
> exp(1.397 + 1.96*.6563)
[1] 14.63401
Holding everything else constant we estimate that the odds of having an infant with low birth
weight are between 1.117 and 14.63 times larger for mothers with hypertension during
pregnancy.
95% CI for OR Associated with Smoking
> exp(.981849 - 1.96*.3983)
[1] 1.222846
> exp(.981849 + 1.96*.3983)
[1] 5.827086
Holding everything else constant we estimate that the odds of having an infant with low birth
weight are between 1.223 and 5.827 times larger for mothers who smoked during pregnancy.
95% CI for OR Associated with Minority Status
> exp(1.0448 - 1.96*.3950)
[1] 1.310751
> exp(1.0448 + 1.96*.3950)
[1] 6.16569
Holding everything else constant we estimate that the odds of having an infant with low birth
weight are between 1.311 and 6.166 times larger for non-white mothers.
OR Associated with Mother’s Weight at Last Menstrual Cycle
Because this is a continuous predictor with values over 100 we should use an increment
larger than one when considering the effect of mother’s weight on birth weight. Here we
will use an increment of c = 10 lbs., although certainly there are other possibilities.
> exp(-10*.014127)
[1] 0.8682549
i.e. 13.2% decrease in the OR for each additional 10 lbs. in premenstrual weight.
A 95% CI for this OR is:
> exp(10*(-.014127) - 1.96*10*.006387)
[1] 0.7660903
> exp(10*(-.014127) + 1.96*10*.006387)
[1] 0.9840439
Create a sequence of weights from smallest observed weight to the largest observed weight by ½
pound increments.
> x = seq(min(Lwt),max(Lwt),.5)
63
Here I have set the other covariates as follows: previous history (1 = yes), hypertension
(0 = no), smoking status (1 = yes), and minority (0 = no).
> fit = predict(low.reduced,data.frame(Prev=factor(rep(1,length(x))),
Hyper=factor(rep(0,length(x))),Smoke=factor(rep(1,length(x))),Minority=
factor(rep(0,length(x))),Lwt=x),type="response")
plot(x,fit,xlab=”Mother’s Weight”,ylab=”P(Low|Prev=1,Smoke=1,Lwt)”)
This is a plot of the effect of
premenstrual weight for smoking
mothers with a history of premature
labor. Using the predict command
above similar plots could be
constructed by examining other
combinations of the categorical
predictors.
64
Case Diagnostics (Delta Deviance and Cook’s Distance)
As in the case of ordinary least squares (OLS) regression we need to be wary of cases
that may have unduly high influence on our results and those that are poorly fit. The
most common influence measure is Cook’s Distance and a good measure of poorly fit
cases is the Delta Deviance.
Essentially Cook’s Distance ( ˆ( i ) or 𝐷𝑖 ) measures the changes in the estimated
parameters when the ith observation is deleted. This change is measured for each of the
observations and can be plotted versus ˆ( x) or observation number to aid in the
~
identification of high influence cases. Several cut-offs have been proposed for Cook’s
Distance, the most common being to classify an observation as having large influence if
ˆ( i )  1 or, in case of large sample size n, ˆ( i )  4 / n .
Cook’s Distance
 ( i ) 
2
1  eˆ i
k  1  hi
where eˆ χ i 
 hi

 1  hi

y i  yˆ i
is the Pearson’s residual defined above.
n i θˆ (~
x i )(1  θˆ (~
x i ))
Delta deviance measures the change in the deviance (D) when the ith case is deleted.
Values around 4 or larger are considered to cases that are poorly fit.
These cases correspond to cases to individuals where yi  1 but ˆ( x) is small, or cases
~
where yi  0 but ˆ( x) is large.
~
In cases of both high influence and poor fit it is good to look at the covariate values for
these individuals and we can begin to address the role they play in the analysis. In many
cases there will be several individuals with the same covariate pattern, especially if most
or all of the predictors are categorical in nature.
> Diagplot.glm(low.reduced)
65
> Diagplot.log(low.reduced)
Cases 11 and 13 have the highest Cook’s distances although they are not that large. It
should be noted also that they are also somewhat poorly fit. Cases 129, 144, 152, and
180 appear to be poorly fit. The information on all of these cases is shown below.
> Lowbirth[c(11,13,129,144,152,180),]
Low Prev Hyper Smoke Uterine Minority Age Lwt race bwt
11
0
0
1
0
0
1 19 95
3 2722
13
0
0
1
0
0
1 22 95
3 2750
129
1
0
0
0
1
0 29 130
1 1021
144
1
0
0
0
1
1 21 200
2 1928
152
1
0
0
0
0
0 24 138
1 2100
180
1
0
0
1
0
0 26 190
1 2466
66
Case 152 had a low birth weight infant even in the absence of the identified potential risk
factors. The fitted values for all four of the poorly fit cases are quite small.
> fitted(low.reduced)[c(11,13,129,144,152,180)]
11
13
129
144
152
180
0.69818500 0.69818500 0.10930602 0.11486743 0.09877858 0.12307383
Cases 11 and 13 have high predicted probabilities despite the fact that they had babies
with normal birth weight. Their relatively high leverage might come from the fact that
there were very few hypertensive minority women in the study. These two facts
combined lead to the relatively large Cook’s Distances for these two cases.
Plotting Estimated Conditional Probabilities ~ P( Low  1 | x~ )
A summary of the reduced model is given below:
> low.reduced
Call: glm(formula = Low ~ Prev + Hyper + Smoke + Minority + Lwt,
family = binomial)
Coefficients:
(Intercept)
Prev1
-0.26127 1.18194
Hyper1
1.39722
Smoke1
0.98185
Degrees of Freedom: 185 Total (i.e. Null);
Null Deviance:
232.4
Residual Deviance: 200.3
AIC: 212.3
Minority1
1.04480
Lwt
-0.01413
180 Residual
To easily plot probabilities in R we can write a function that takes covariate values and
compute the desired conditional probability.
> x <- seq(min(Lwt),max(Lwt),.5)
>
+
+
+
+
>
+
>
>
>
>
PrLwt <- function(x,Prev,Hyper,Smoke,Minority) {
L <- -.26127 + 1.18194*Prev + 1.39722*Hyper + .98185*Smoke +
1.0448*Minority - .01413*x
exp(L)/(1 + exp(L))
}
plot(x,PrLwt(x,1,1,1,1),xlab="Mother's Weight",ylab="P(Low=1|x)",
ylim=c(0,1),type="l")
title(main="Plot of P(Low=1|X) vs. Mother's Weight")
lines(x,PrLwt(x,0,0,0,0),lty=2,col="red")
lines(x,PrLwt(x,1,1,0,0),lty=3,col="blue")
lines(x,PrLwt(x,0,0,1,1),lty=4,col="green")
67
R Function – Diagplot.log
Plot Cook’s Distance and Delta Deviance for Logistic Regression Models
Diagplot.log = function(glm1)
{
k <- length(glm1$coef)
h <- lm.influence(glm1)$hat
fv <- fitted(glm1)
pr <- resid(glm1, type = "pearson")
dr <- resid(glm1, type = "deviance")
par(mfrow = c(2, 1))
n <- length(fv)
index <- seq(1, n, 1)
Ck <- (1/k)*((pr^2) * h)/((1 - h)^2)
Cd <- dr^2/(1 - h)
plot(index, Ck, type = "n", xlab = "Index", ylab =
"Cook's Distance", cex = 0.7, main =
"Plot of Cook's Distance vs. Index", col = 1)
points(index, Ck, col = 2)
identify(index, Ck)
plot(index, Cd, type = "n", xlab = "Index", ylab =
"Delta Deviance", cex = 0.7, main =
"Plot of Delta Deviance vs. Index")
points(index, Cd, col = 2)
identify(index, Cd)
par(mfrow = c(1, 1))
invisible()
}
68
Interactions and Higher Order Terms (Note ~ uses data frame: Lowbwt)
Working with a slightly different version of the low birth weight data available which
includes an additional predictor, ftv, which is a factor that indicates the number of first
trimester doctor visits the woman (coded as: 0, 1, or 2+). We will examine how the
model below was developed in the next section where we discuss model development.
In the model below we have added an interaction between age and the number of first
trimester visits. The logistic model is:
  ( x) 
~
     Age   Lwt   Smoke   Pr ev   HT   UI 
log 
o
1
2
3
4
5
6
 1   ( x) 
~ 

 7 FTV 1   8 FTV 2   9 Age * FTV 1  10 Age * FTV 2  11Smoke * UI
> summary(bigmodel)
Call:
glm(formula = low ~ age + lwt + smoke + ptd + ht + ui + ftv +
age:ftv + smoke:ui, family = binomial)
Deviance Residuals:
Min
1Q
Median
-1.8945 -0.7128 -0.4817
3Q
0.7841
Max
2.3418
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.582389
1.420834 -0.410 0.681885
age
0.075538
0.053945
1.400 0.161428
lwt
-0.020372
0.007488 -2.721 0.006513 **
smoke1
0.780047
0.420043
1.857 0.063302 .
ptd1
1.560304
0.496626
3.142 0.001679 **
ht1
2.065680
0.748330
2.760 0.005773 **
ui1
1.818496
0.666670
2.728 0.006377 **
ftv1
2.921068
2.284093
1.279 0.200941
ftv2+
9.244460
2.650495
3.488 0.000487 ***
age:ftv1
-0.161823
0.096736 -1.673 0.094360 .
age:ftv2+
-0.411011
0.118553 -3.467 0.000527 ***
smoke1:ui1 -1.916644
0.972366 -1.971 0.048711 *
--Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 234.67
Residual deviance: 183.07
AIC: 207.07
on 188
on 177
degrees of freedom
degrees of freedom
Number of Fisher Scoring iterations: 4
> bigmodel$coefficients
(Intercept)
age
lwt
smoke1
prev1
ht1
-0.58238913 0.07553844 -0.02037234 0.78004747 1.56030401 2.06567991
ui1
ftv1
ftv2+
age:ftv1
age:ftv2+ smoke1:ui1
1.81849631 2.92106773 9.24445985 -0.16182328 -0.41101103 -1.91664380
69
Calculate P(Low|Age,FTV) for women of average pre-pregnancy weight with all other
risk factors absent. Similar calculations could be done if we wanted to add in other
factors as well.
First we calculate the logits as function of age for three levels of FTV 0, 1, and 2+
respectively.
> L <- -.5824 + .0755*agex - .02037*mean(lwt)
> L1 <- -.5824 + .0755*agex - .02037*mean(lwt) + 2.9211 - .16182*agex
> L2 <- -.5824 + .0755*agex - .02037*mean(lwt) + 9.2445 - .4110*agex
Next we calculate the associated conditional probabilities.
> P <- exp(L)/(1+exp(L))
> P1 <- exp(L1)/(1+exp(L1))
> P2 <- exp(L2)/(1+exp(L2))
Finally we plot the probability curves as function of age and FTV.
> plot(agex,P,type="l",xlab="Age",ylab="P(Low|Age,FTV)",ylim=c(0,1))
> lines(agex,P1,lty=2,col="blue")
> lines(agex,P2,lty=3,col="red")
> title(main="Interaction Between Age and First Trimester
Visits",cex=.6)
The interaction between in age and
FTV produces differences in
direction and magnitude of the age
effect. For women with no first
trimester doctor visits their
probability of low birth weight
increases with age. However for
women with at least one first
trimester visit the probability of low
birth weight decreases with age.
The magnitude of that drop is
largest for women with 2 or more
first trimester visits.
We also have an interaction between smoking and uterine irritability added to the model.
This will affect how we interpret the two in terms of odds ratios. We need to consider
the OR associated with smoking for women without uterine irritability, the OR associated
with uterine irritability for nonsmokers, and finally the OR associated with smoking and
having uterine irritability during pregnancy.
70
These estimated odds ratios are given below:
OR for Smoking with No Uterine Irritability
> exp(.7800)
[1] 2.181472
OR for Uterine Irritability with No Smoking
> exp(1.8185)
[1] 6.162608
OR for Smoking and Uterine Irritability
> exp(.7800+1.8185-1.91664)
[1] 1.977553
This result is hard to explain physiologically and so this interaction term might be
removed from the model.
Model Selection Methods
Stepwise methods used in logistic regression are the same as those used in ordinary least
square regression however the measure is the AIC (Akaike Information Criteria) as
opposed to Mallow’s Ck statistic. Like Mallow’s statistic, AIC balances residual
deviance and the number of parameters in the model.
AIC = D + 2k ˆ
Where D = residual deviance, k = total number of estimated parameters, and ˆ is an
estimate of the dispersion parameter which is taken to be 1 in models where
overdispersion is not present. Overdispersion occurs when the data consists of the
number of successes out of mi > 1 trials and the trials are not independent (e.g. male birth
data from your last homework).
Forward, backward, both forward and backward simultaneously, and all possible subsets
regression methods can be employed to find models with small AIC values. By default R
uses both forward and backward selection simultaneously. The command to do this in R
has the basic form:
> step(current model name)
To have it select from models containing all potential two-way interactions use:
> step(current model name, scope=~.^2)
This sometimes will have problems with convergence due to overfitting (i.e. the
estimated probabilities approach 0 and 1 as in the saturated model). If this occurs you
can have R consider adding each of the potential interaction terms and then you can scan
the list and decide which you might want to add to your existing model. You can then
continue adding terms until the AIC criteria suggests additional terms do not improve
current model.
71
These commands are illustrated for the low birth weight data with first trimester visits
included in the output shown below.
Base Model
> low.glm <- glm(low~age+lwt+race+smoke+ht+ui+ptd+ftv,family=binomial)
> summary(low.glm)
Call:
glm(formula = low ~ age + lwt + race + smoke + ht + ui + ptd +
ftv, family = binomial)
Deviance Residuals:
Min
1Q
Median
-1.7038 -0.8068 -0.5009
3Q
0.8836
Max
2.2151
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.822706
1.240174
0.663 0.50709
age
-0.037220
0.038530 -0.966 0.33404
lwt
-0.015651
0.007048 -2.221 0.02637 *
race2
1.192231
0.534428
2.231 0.02569 *
race3
0.740513
0.459769
1.611 0.10726
smoke1
0.755374
0.423246
1.785 0.07431 .
ht1
1.912974
0.718586
2.662 0.00776 **
ui1
0.680162
0.463464
1.468 0.14222
ptd1
1.343654
0.479409
2.803 0.00507 **
ftv1
-0.436331
0.477792 -0.913 0.36112
ftv2+
0.178939
0.455227
0.393 0.69426
--Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 234.67
Residual deviance: 195.48
AIC: 217.48
on 188
on 178
degrees of freedom
degrees of freedom
Find “best” model that includes all potential two-way interactions.
> low.step <- step(low.glm,scope=~.^2)
Start: AIC= 217.48
low ~ age + lwt + race + smoke + ht + ui + ptd + ftv
+ age:ftv
- ftv
- age
<none>
- ui
+ smoke:ui
+ lwt:smoke
+ ui:ptd
+ lwt:ui
+ ptd:ftv
+ ht:ptd
Df Deviance
AIC
2
183.00 209.00
2
196.83 214.83
1
196.42 216.42
195.48 217.48
1
197.59 217.59
1
193.76 217.76
1
194.04 218.04
1
194.24 218.24
1
194.28 218.28
2
192.38 218.38
1
194.55 218.55
72
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
age:ptd
age:ht
age:smoke
race:ui
smoke
smoke:ht
smoke:ptd
race
race:smoke
lwt:ptd
lwt:ht
age:lwt
age:ui
ht:ftv
lwt:ftv
smoke:ftv
age:race
lwt:race
race:ptd
lwt
race:ht
ui:ftv
ht
ptd
race:ftv
1
1
1
2
1
1
1
2
2
1
1
1
1
2
2
2
2
2
2
1
2
2
1
1
4
194.58
194.59
194.61
192.63
198.67
195.03
195.16
201.23
193.24
195.35
195.44
195.46
195.47
194.00
194.19
194.47
194.58
194.63
194.83
200.95
195.19
195.32
202.93
203.58
193.81
218.58
218.59
218.61
218.63
218.67
219.03
219.16
219.23
219.24
219.35
219.44
219.46
219.47
220.00
220.19
220.47
220.58
220.63
220.83
220.95
221.19
221.32
222.93
223.58
223.81
Step: AIC= 209
low ~ age + lwt + race + smoke + ht + ui + ptd + ftv + age:ftv
+ smoke:ui
+ lwt:smoke
- race
<none>
+ ui:ptd
+ lwt:ui
+ ht:ptd
- smoke
+ age:smoke
+ race:ui
+ age:ptd
- ui
+ smoke:ht
+ lwt:ptd
+ smoke:ptd
+ age:ht
+ age:ui
+ age:lwt
+ lwt:ht
+ race:smoke
+ lwt:ftv
+ ptd:ftv
+ age:race
+ smoke:ftv
+ ht:ftv
+ lwt:race
+ race:ht
Df Deviance
AIC
1
179.94 207.94
1
180.89 208.89
2
186.99 208.99
183.00 209.00
1
181.42 209.42
1
181.90 209.90
1
182.06 210.06
1
186.11 210.11
1
182.16 210.16
2
180.32 210.32
1
182.50 210.50
1
186.61 210.61
1
182.71 210.71
1
182.75 210.75
1
182.82 210.82
1
182.90 210.90
1
182.96 210.96
1
183.00 211.00
1
183.00 211.00
2
181.23 211.23
2
181.44 211.44
2
181.57 211.57
2
181.62 211.62
2
181.65 211.65
2
181.82 211.82
2
182.55 212.55
2
182.78 212.78
73
+
+
+
-
race:ptd
lwt
ui:ftv
ht
ptd
race:ftv
age:ftv
2
1
2
1
1
4
2
182.85
188.88
182.94
190.13
191.05
181.69
195.48
212.85
212.88
212.94
214.13
215.05
215.69
217.48
Step: AIC= 207.94
low ~ age + lwt + race + smoke + ht + ui + ptd + ftv + age:ftv +
smoke:ui
- race
<none>
+ lwt:smoke
+ ht:ptd
- smoke:ui
+ ui:ptd
+ age:ptd
+ age:smoke
+ smoke:ptd
+ lwt:ptd
+ lwt:ui
+ age:ht
+ smoke:ht
+ age:lwt
+ age:ui
+ lwt:ht
+ lwt:ftv
+ ptd:ftv
+ smoke:ftv
+ race:smoke
+ age:race
+ ht:ftv
+ race:ui
+ ui:ftv
+ race:ht
+ lwt:race
+ race:ptd
- lwt
- ht
+ race:ftv
- ptd
- age:ftv
Df Deviance
AIC
2
183.07 207.07
179.94 207.94
1
178.34 208.34
1
178.89 208.89
1
183.00 209.00
1
179.07 209.07
1
179.35 209.35
1
179.37 209.37
1
179.58 209.58
1
179.61 209.61
1
179.76 209.76
1
179.78 209.78
1
179.82 209.82
1
179.84 209.84
1
179.86 209.86
1
179.94 209.94
2
178.25 210.25
2
178.53 210.53
2
178.64 210.64
2
178.73 210.73
2
178.84 210.84
2
178.89 210.89
2
179.13 211.13
2
179.50 211.50
2
179.52 211.52
2
179.68 211.68
2
179.86 211.86
1
187.15 213.15
1
187.66 213.66
4
178.51 214.51
1
188.83 214.83
2
193.76 217.76
Step: AIC= 207.07
low ~ age + lwt + smoke + ht + ui + ptd + ftv + age:ftv + smoke:ui
<none>
+ lwt:smoke
+ ui:ptd
+ ht:ptd
+ race
+ age:smoke
+ age:ht
Df Deviance
183.07
1
181.40
1
181.88
1
181.93
2
179.94
1
181.97
1
182.64
AIC
207.07
207.40
207.88
207.93
207.94
207.97
208.64
74
+
+
+
+
+
+
+
+
+
+
+
+
+
-
age:ptd
lwt:ptd
lwt:ui
smoke:ptd
age:lwt
smoke:ui
age:ui
smoke:ht
lwt:ht
smoke:ftv
lwt:ftv
ptd:ftv
ui:ftv
ht:ftv
ht
lwt
ptd
age:ftv
1
1
1
1
1
1
1
1
1
2
2
2
2
2
1
1
1
2
182.69
182.73
182.76
182.85
182.92
186.99
182.99
183.02
183.06
181.48
181.69
181.85
182.28
182.41
191.21
191.56
193.59
199.00
208.69
208.73
208.76
208.85
208.92
208.99
208.99
209.02
209.06
209.48
209.69
209.85
210.28
210.41
213.21
213.56
215.59
219.00
Summarize the model returned from the stepwise search
> summary(low.step)
Call:
glm(formula = low ~ age + lwt + smoke + ht + ui + ptd + ftv +
age:ftv + smoke:ui, family = binomial)
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.582389
1.420834 -0.410 0.681885
age
0.075538
0.053945
1.400 0.161428
lwt
-0.020372
0.007488 -2.721 0.006513 **
smoke1
0.780047
0.420043
1.857 0.063302 .
ht1
2.065680
0.748330
2.760 0.005773 **
ui1
1.818496
0.666670
2.728 0.006377 **
ptd1
1.560304
0.496626
3.142 0.001679 **
ftv1
2.921068
2.284093
1.279 0.200941
ftv2+
9.244460
2.650495
3.488 0.000487 ***
age:ftv1
-0.161823
0.096736 -1.673 0.094360 .
age:ftv2+
-0.411011
0.118553 -3.467 0.000527 ***
smoke1:ui1 -1.916644
0.972366 -1.971 0.048711 *
Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 234.67 on 188 degrees of freedom
Residual deviance: 183.07 on 177 degrees of freedom
AIC: 207.07
Number of Fisher Scoring iterations: 4
This is the model used to demonstrate model interpretation in the presence of
interactions.
An alternative to the full blown search above is to consider adding a single interaction
term to the “Base Model” from the set of all possible terms.
> add1(low.glm,scope=~.^2)
75
Single term additions
Model:
low ~ age + lwt + race
Df Deviance
<none>
195.48
age:lwt
1
195.46
age:race
2
194.58
age:smoke
1
194.61
age:ht
1
194.59
age:ui
1
195.47
age:ptd
1
194.58
age:ftv
2
183.00
lwt:race
2
194.63
lwt:smoke
1
194.04
lwt:ht
1
195.44
lwt:ui
1
194.28
lwt:ptd
1
195.35
lwt:ftv
2
194.19
race:smoke 2
193.24
race:ht
2
195.19
race:ui
2
192.63
race:ptd
2
194.83
race:ftv
4
193.81
smoke:ht
1
195.03
smoke:ui
1
193.76
smoke:ptd
1
195.16
smoke:ftv
2
194.47
ht:ui
0
195.48
ht:ptd
1
194.55
ht:ftv
2
194.00
ui:ptd
1
194.24
ui:ftv
2
195.32
ptd:ftv
2
192.38
+ smoke + ht + ui + ptd + ftv
AIC
217.48
219.46
220.58
218.61
218.59
219.47
218.58
209.00 *
220.63
218.04
219.44
218.28
219.35
220.19
219.24
221.19
218.63
220.83
223.81
219.03
217.76
219.16
220.47
217.48
218.55
220.00
218.24
221.32
218.38
We can than “manually” enter this term to our base model by using the update
command in R.
> low.glm2 <- update(low.glm,.~.+age:ftv)
> summary(low.glm2)
Call:
glm(formula = low ~ age + lwt + race + smoke + ht + ui + ptd +
ftv + age:ftv, family = binomial)
Deviance Residuals:
Min
1Q
Median
-2.0338 -0.7690 -0.4510
3Q
0.8354
Max
2.3383
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -1.636485
1.558677 -1.050 0.29376
age
0.085461
0.055734
1.533 0.12519
lwt
-0.017599
0.007653 -2.300 0.02147 *
race2
0.994134
0.550962
1.804 0.07118 .
race3
0.700669
0.491400
1.426 0.15391
76
smoke1
0.792972
ht1
1.936204
ui1
0.938620
ptd1
1.373390
ftv1
2.877889
ftv2+
8.264965
age:ftv1
-0.149619
age:ftv2+
-0.359454
--Signif. codes: 0 `***'
0.452303
0.747576
0.492240
0.495738
2.253710
2.594444
0.096342
0.115429
1.753
2.590
1.907
2.770
1.277
3.186
-1.553
-3.114
0.07957
0.00960
0.05654
0.00560
0.20162
0.00144
0.12043
0.00185
.
**
.
**
**
**
0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 234.67 on 188 degrees of freedom
Residual deviance: 183.00 on 176 degrees of freedom
AIC: 209
Number of Fisher Scoring iterations: 4
Next we could use add1 to consider the remaining interaction terms for addition to this
model.
> add1(low.glm2,scope=~.^2)
Single term additions
Model:
low ~ age + lwt + race + smoke + ht + ui + ptd + ftv + age:ftv
Df Deviance
AIC
<none>
183.00 209.00
age:lwt
1
183.00 211.00
age:race
2
181.62 211.62
age:smoke
1
182.16 210.16
age:ht
1
182.90 210.90
age:ui
1
182.96 210.96
age:ptd
1
182.50 210.50
lwt:race
2
182.55 212.55
lwt:smoke
1
180.89 208.89 *
lwt:ht
1
183.00 211.00
lwt:ui
1
181.90 209.90
lwt:ptd
1
182.75 210.75
lwt:ftv
2
181.44 211.44
race:smoke 2
181.23 211.23
race:ht
2
182.78 212.78
race:ui
2
180.32 210.32
race:ptd
2
182.85 212.85
race:ftv
4
181.69 215.69
smoke:ht
1
182.71 210.71
smoke:ui
1
179.94 207.94 **
smoke:ptd
1
182.82 210.82
smoke:ftv
2
181.65 211.65
ht:ui
0
183.00 209.00
ht:ptd
1
182.06 210.06
ht:ftv
2
181.82 211.82
ui:ptd
1
181.42 209.42
ui:ftv
2
182.94 212.94
ptd:ftv
2
181.57 211.57
77
Fitting Logistic Models in Arc and More Diagnostics
from website)
Again we consider the low birth weight case study.
(lowbirtharc.txt
Arc 1.03, rev Aug, 2000, Wed Oct 22, 2003, 12:10:14. Data set name:
Lowbw
Low birth weight study.
Name
Type
n
Info
AGE
Variate 189 Age of mother
BWT
Variate 189 Actual birthweight of child in grams
HT
Variate 189 Mother hypertensive during pregnancy (1 = yes, 0
= no)
ID
Variate 189
LOW
Variate 189 (1 = low birthweight, 0 = normal birthweight)
LWT
Variate 189 Mothers weight at last menstrual cycle
PTD
Variate 189 do not know
PTL
Variate 189 Previous history of premature labor (1 = yes, 0 =
no)
RACE
Variate 189 Race of mother (1 = white, 2 = black, 3 = other)
SMOKE
Variate 189 Mother smoke (1 = yes, 0 = no)
UI
Variate 189 Uterine irritability (1 = yes, 0 = no)
FTV
Text
189 # of doctor visits during 1st trimester
{F}FTV
Factor 189 Factor--first level dropped
{F}HT
Factor 189 Factor--first level dropped
{F}PTD
Factor 189 Factor--first level dropped
{F}RACE Factor 189 Factor--first level dropped
{F}SMOKE Factor 189 Factor--first level dropped
{F}UI
Factor 189 Factor--first level dropped
Select Fit binomial response… from
the Graph & Fit menu
In the resulting dialog box, specify the model as shown on the following page.
78
Give the model a name if you want.
Always include an intercept.
Use the Make Factors… option from the
data set menu to ensure all categorical
predictors are treated as factors.
Put dichotomous response in the
Response… box. The response may also
be the number of “successes” observed.
(see below)
If mi=1 for all cases then put the variable
Ones in the Trials… box. If your
response represented the number of
“successes” observed in mi > 1 trials then
you we need to import the number trials
and put that variable in this box.
The output below shows the results of fitting this initial model.
Data set = Lowbw, Name of Fit = B1
Binomial Regression
Kernel mean function = Logistic
Response
= LOW
Terms
= (AGE LWT {F}FTV {F}HT {F}PTD
Trials
= Ones
Coefficient Estimates
Label
Estimate
Std. Error
Constant
0.386634
1.27736
AGE
-0.0372340
0.0386777
LWT
-0.0156530
0.00707594
{F}FTV[0]
0.436379
0.479161
{F}FTV[2+]
0.615386
0.553104
{F}HT[1]
1.91316
0.720434
{F}PTD[1]
1.34376
0.480445
{F}RACE[2]
1.19241
0.535746
{F}RACE[3]
0.740681
0.461461
{F}SMOKE[1]
0.755525
0.424764
{F}UI[1]
0.680195
0.464216
{F}RACE {F}SMOKE {F}UI)
Est/SE
0.303
-0.963
-2.212
0.911
1.113
2.656
2.797
2.226
1.605
1.779
1.465
p-value
0.7621
0.3357
0.0270
0.3624
0.2659
0.0079
0.0052
0.0260
0.1085
0.0753
0.1429
Note: For FTV those
who went to the
doctor once during
the first trimester are
used as the reference
group
Scale factor:
1.
Number of cases:
189
Degrees of freedom:
178
Pearson X2:
179.059
Deviance:
195.476
(Note: AIC = D + 2k*(scale factor) = 195.48 + 22 = 217.48)
The results are identical those obtained from R.
Null deviance: 234.67
Residual deviance: 195.48
on 188
on 178
degrees of freedom
degrees of freedom
79
Examining Submodels – Backward Elimination and Forward Selection
Forward Elimination –
Select this option and click OK. It will
then show terms are sequentially added
to a model containing any base terms to
the model. By default the base contains
the intercept only.
Backward Elimination –
Simply select this option and click OK. It
will show how terms are sequentially
eliminated from the model along with the
resulting AIC for the deletion.
The other options do what they say.
The results of backward elimination for the current low birth weight model are shown
below.
Data set = Lowbw, Name of Fit = B1
Binomial Regression
Kernel mean function = Logistic
Response
= LOW
Terms
= (AGE LWT {F}FTV {F}HT {F}PTD {F}RACE {F}SMOKE {F}UI)
Trials
= Ones
Backward Elimination: Sequentially remove terms
that give the smallest change in AIC.
All fits include an intercept.
Current terms: (AGE LWT {F}FTV {F}HT {F}PTD {F}RACE {F}SMOKE {F}UI)
df
Deviance
Pearson X2 |
k
AIC
Delete: {F}FTV
180
196.834
180.989
|
9 214.834 *
Delete: AGE
179
196.417
181.401
|
10 216.417
Delete: {F}UI
179
197.585
180.753
|
10 217.585
Delete: {F}SMOKE
179
198.674
186.809
|
10 218.674
Delete: {F}RACE
180
201.227
183.365
|
9 219.227
Delete: LWT
179
200.949
177.855
|
10 220.949
Delete: {F}HT
179
202.934
177.447
|
10 222.934
Delete: {F}PTD
179
203.584
180.74
|
10 223.584
Current terms: (AGE LWT {F}HT {F}PTD {F}RACE {F}SMOKE {F}UI)
df
Deviance
Pearson X2 |
k
AIC
Delete: AGE
181
197.852
183.999
|
8 213.852
Delete: {F}UI
181
199.151
184.559
|
8 215.151
Delete: {F}RACE
182
203.24
182.815
|
7 217.240
Delete: {F}SMOKE
181
201.247
186.953
|
8 217.247
Delete: LWT
181
201.833
181.355
|
8 217.833
Delete: {F}PTD
181
203.948
181.536
|
8 219.948
Delete: {F}HT
181
204.013
179.069
|
8 220.013
*
80
Current terms: (LWT {F}HT {F}PTD {F}RACE {F}SMOKE
df
Deviance
Pearson X2
Delete: {F}UI
182
200.482
186.918
Delete: {F}SMOKE
182
202.567
189.716
Delete: {F}RACE
183
205.466
186.461
Delete: LWT
182
203.816
185.551
Delete: {F}PTD
182
204.217
182.499
Delete: {F}HT
182
205.162
182.282
{F}UI)
|
k
|
7
|
7
|
6
|
7
|
7
|
7
Current terms: (LWT {F}HT {F}PTD {F}RACE {F}SMOKE)
df
Deviance
Pearson X2 |
Delete: {F}SMOKE
183
205.397
189.925
|
Delete: {F}RACE
184
207.955
192.506
|
Delete: {F}HT
183
207.039
184.17
|
Delete: LWT
183
207.165
187.234
|
Delete: {F}PTD
183
208.247
184.45
|
Current terms: (LWT {F}HT {F}PTD {F}RACE)
df
Deviance
Pearson X2 |
Delete: {F}RACE
185
210.123
194.086
|
Delete: {F}HT
184
212.18
188.048
|
Delete: LWT
184
213.226
187.544
|
Delete: {F}PTD
184
216.295
191.533
|
k
6
5
6
6
6
k
4
5
5
5
AIC
214.482
216.567
217.466
217.816
218.217
219.162
*
AIC
217.397
217.955
219.039
219.165
220.247
AIC
218.123
222.180
223.226
226.295
Current terms: (LWT {F}HT {F}PTD)
df
Deviance
Delete: {F}HT
186
217.497
Delete: LWT
186
217.662
Delete: {F}PTD
186
221.142
Pearson X2 |
190.809
|
188.394
|
193.26
|
k
3
3
3
AIC
223.497
223.662
227.142
Current terms: (LWT {F}PTD)
df
Deviance
Delete: LWT
187
221.898
Delete: {F}PTD
187
228.691
Pearson X2 |
188.863
|
189.647
|
k
2
2
AIC
225.898
232.691
* indicates a potential “final” model using the AIC criteria, Arc does not add the *’s.
Making Interactions
To make interactions in Arc…
1st - Select Make
Interactions from
the data set menu.
2nd - Placing all
covariates in the righthand box will create
all possible two-way
interactions.
81
Deciding which interactions to include however is not as easy as in R. You could
potentially include all interactions and then backward eliminate, however things will get
unstable numerically with that many terms in the model. It is better to choose any
interactions you feel might make physiological sense and then backward eliminate.
If Arc does not use the reference group you would like to use, you can create dummy
variables for each level of the factor and then leave the one for the reference group out
when you specify the model.
Selecting these options will
create three dummy
variables one for each level
of FTV (0, 1, 2+).
The model with the age*recoded FTV and the smoking*uterine irritability interactions
we saw in the R handout is summarized below.
Data set = Lowbw, Name of Fit = B6
Binomial Regression
Kernel mean function = Logistic
Response
= LOW
Terms
= (AGE LWT {F}HT {F}PTD {F}SMOKE {F}UI {F}SMOKE*{F}UI
{T}FTV[1] {T}FTV[2+] {T}FTV[1]*AGE {T}FTV[2+]*AGE)
Trials
= Ones
Coefficient Estimates
Label
Estimate
Std. Error
Est/SE
p-value
Constant
-0.582374
1.42158
-0.410
0.6821
AGE
0.0755389
0.0539665
1.400
0.1616
LWT
-0.0203726
0.00749678
-2.718
0.0066
{F}HT[1]
2.06570
0.748727
2.759
0.0058
{F}PTD[1]
1.56032
0.496986
3.140
0.0017
{F}SMOKE[1]
0.780044
0.420371
1.856
0.0635
{F}UI[1]
1.81853
0.667517
2.724
0.0064
{F}SMOKE[1].{F}UI[1] -1.91668
0.973066
-1.970
0.0489
{T}FTV[1]
2.92109
2.28571
1.278
0.2013
{T}FTV[2+]
9.24491
2.66099
3.474
0.0005
{T}FTV[1].AGE
-0.161824
0.0968164
-1.671
0.0946
{T}FTV[2+].AGE
-0.411033
0.119117
-3.451
0.0006
Number of cases:
Degrees of freedom:
Pearson X2:
Deviance:
189
177
179.282
183.073
Notice: The recoding of FTV so FTV=0
is now the reference group.
82
Diagnostic Plots
There are several plotting options in Arc to help assess a models adequacy.
They are as follows:
 Residuals (deviance or chi-square) vs. the estimated logit ( Lˆ  ˆ T X )
Deviance Residual

 y 
 1  y i 

Di  2  sgn( yi  ˆ( xi ))   yi ln  i   (1  yi ) ln 
 ˆ( x ) 
 1  ˆ( x ) 

i
i




Chi-residual for the ith covariate pattern is defined as:
yi  yˆ i
eˆi 
the sum of the squared chi-residuals = Pearson’s 
m ˆ( x )(1  ˆ( x ))
i
i
~
~i
where yˆ i  miˆ( xi ) and yi  1 for cases and 0 for controls.
~



Plot of Cook’s distance vs. Case Number or some other quantity.
Plot of Leverage (potential for influence) vs. Case Number
Model checking plots
Eta’U ~ Estimated Logit ( Lˆi  ˆ T xi )
~
Obs-Fraction ~ yi / mi (1 and 0’s in the case mi =1)
ˆ
e Li
Fit-Fraction ~ ˆ( xi ) 
ˆ
~
1  e Li
Chi-Residuals ~ see above
Dev-Residuals ~ see above
eˆ i
T-Residuals ~
studentized chi-residual
1  hi
Leverages ~ hi = ith element of the hat matrix H
2
1  eˆ  i  hi
Cook’s Distance ~ Di 
measures
k  1  hi  1  hi
influence of the ith case.
Residuals vs. Estimated Logit (or some other function of the covariates)
If the model is adequate, a lowess ( = .6) smooth added to the plot should be constant,
i.e. flat. This plot will not work well when the number of replicates, mi are small, i.e.
close to 1. Model checking plots work better for checking model adequacy in those
cases.
83
As an example consider the simple, but reasonable, main effects model shown on the next
page.
Data set = Lowbw, Name of Fit = B3
Binomial Regression
Kernel mean function = Logistic
Response
= LOW
Terms
= (LWT {F}HT {F}PTD {F}RACE {F}SMOKE {F}UI)
Trials
= Ones
Coefficient Estimates
Label
Estimate
Std. Error
Est/SE
p-value
Constant
-0.125327
0.967238
-0.130
0.8969
LWT
-0.0159185
0.00695085
-2.290
0.0220
{F}HT[1]
1.86689
0.707212
2.640
0.0083
{F}PTD[1]
1.12886
0.450330
2.507
0.0122
{F}RACE[2]
1.30085
0.528349
2.462
0.0138
{F}RACE[3]
0.854413
0.440761
1.938
0.0526
{F}SMOKE[1]
0.866581
0.404341
2.143
0.0321
{F}UI[1]
0.750648
0.458753
1.636
0.1018
Scale factor:
Number of cases:
Degrees of freedom:
Pearson X2:
Deviance:
1.
189
181
183.999
197.852
The plots of the chi-square residuals vs. the estimated logit ( L̂ = ˆ T X ) and LWT are
shown below. The lowess smooth looks fairly flat and so no model inadequacies are
suggested.
84
Cook’s Distance vs. Case Number and Est. Probs - (no cases have high influence)
Leverages vs. Case Numbers
For leverages the average value is k/n, so values far exceeding the average have the
potential to be influential. The following is a good rule of thumb:
1/n < hi < .25 no worries
.25 < hi < .50 worry
.50 < hi < 1 worry lots
Model Checking Plots
For any linear combination b T x i of the predictors of terms imagine drawing two plots:
one of y / m vs. b T x , and one of ˆ( x ) vs. b T x . If the model is adequate lowess smooth
i
i
i
i
~
i
of each should match for any linear combination we choose. A model checking plot is a
plot with b T x i on the x-axis and both the lowess smooths described above added to the
plot. If they agree for a variety of choices of b T x i then we can feel reasonably confident
that our model is adequate. Large differences between these smooths can indicate model
deficiencies. Common choices for b T x i include the estimated logits ( L̂ ), the individual
predictors, and randomly chosen combinations of the terms in the model.
85
Here we see good
agreement between the
two smoothes for the
estimated logits.
Model checking plot
with the single term
LWT on the x-axis.
86
Model checking plot for
one random linear
combination of the terms
in the model. Again we
see good agreement.
87
Using Arc when the Number Trials is not 1
Example 1: Oral contraceptive use, myocardial infarctions, and age
To read these data in Arc it is easiest to create a text file that looks like:
Age OCuse MI NoMI Trials
1 Yes 4 62 66
1 No 2 224 226
2 Yes 9 33 42
2 No 12 390 402
3 Yes 4 26 30
3 No 33 330 363
4 Yes 6 9 15
4 No 65 362 427
5 Yes 6 5 11
5 No 93 301 394
The Trials column contains the total number of patients in each age and oral
contraceptive use category, i.e. the sum of the number of patients with MI and the
number of patients without MI (NoMI).
When read in Arc we have:
; loading D:\Data\Deppa Documents\Biostatistics (Biometry II)\Book
Data\OCMI.txt
Arc 1.06, rev July 2004, Mon Oct 16, 2006, 12:58:46. Data set name:
OCMI
Oral contraceptive use, age, and myocardial infarctions
Name
Type
n
Info
AGE
Variate 10
MI
Variate 10
NOMI
Variate 10
TRIALS Variate 10
OCUSE Text
10
In Arc we need to create to turn the Age variable into a factor as we don’t want to be
interpreted as an actual number and we need to create a factor based on OCuse. By
default Arc does things alphabetically so No would be used as “present” which is not
desirable. Thus it is best to create separate dichotomous dummy variables for each level
individually. This will allow us to use those who used oral contraceptives as having “risk
present”. To do this in Arc we need to use the Make Factors… option in the data menu.
88
For oral contraceptive use we want two separate dummy variables, one for each level of
use, i.e. Yes and No.
Fitting the logistic model in Arc with MI as the response and OCUSE[YES] as the risk
factor indicator.
89
Results for Fitted Logistic Model
Iteration 1: deviance = 6.69914
Iteration 2: deviance = 6.53561
Data set = OCMI, Name of Fit = B1
Binomial Regression
Kernel mean function = Logistic
Response
= MI
Terms
= ({F}AGE {T}OCUSE[YES])
Trials
= TRIALS
Coefficient Estimates
Label
Estimate
Std. Error
Constant
-4.36985
0.434642
{F}AGE[2]
1.13836
0.476782
{F}AGE[3]
1.93440
0.458227
{F}AGE[4]
2.64806
0.449627
{F}AGE[5]
3.19429
0.447386
{T}OCUSE[YES]
1.38518
0.250458
Scale factor:
Number of cases:
Degrees of freedom:
Pearson X2:
Deviance:
Est/SE
-10.054
2.388
4.221
5.889
7.140
5.531
p-value
0.0000
0.0170
0.0000
0.0000
0.0000
0.0000
1.
10
4
6.386
6.536
We can work with these parameter estimates as above to obtain OR’s of interest etc.
90
Motivating Example: Recumbant Cows
“The abiltiy of biochemical and haematolgical tests to predict recovery in
periparturient recumbent cows.” NZ Veterinary Journal, 35, 126-133 Clark, R. G.,
Henderson, H. V., Hoggard, G. K. Ellison, R. S. and Young,B. J. (1987).
Study Description:
For unknown reasons, many pregnant dairy cows become recumbant--they lay down-either shortly before or after calving. This condition can be serious, and may lead to
death of the cow. These data are from a study of blood samples of over 500 cows studied
at the Ruakura (N.Z.) Animal Health Laboratory during 1983-84. A variety of blood
tests were performed, and for many of the animals the outcome (survived, died, or animal
was killed) was determined. The goal is to see if survival can be predicted from the
blood measurements. Case numbers 12607 and 11630 were noted as having exceptional
care---and they survived.
Name
AST
Calving
CK
Daysrec
Inflamat
Myopathy
Outcome
PCV
Urea
CaseNo
Type
Variate
Variate
Variate
Variate
Variate
Variate
Variate
Variate
Variate
Text
n
429
431
413
432
136
222
435
175
266
435
Info
serum asparate amino transferase (U/l at 30C)
0 if measured before calving, 1 if after
Serum creatine phosphokinase (U/l at 30C)
Days recumbent
inflamation 0=no, 1=yes
Muscle disorder, 1 if present, 0 if absent
outcome: 1 if survived, 0 if died or killed (response)
Packed Cell Volume (Haemactocrit), %
serum urea (mmol/l)
case number
Because calving, inflammation, and myopathy are Bernoulli dichotomous predictors they
will not be transformed, although we might consider potential interactions involving
these predictors. We will not consider inflammation and myopathy however as most of
the cows have that information missing.
Guidelines for Transforming Predictors in Logistic Regression
Examining univariate conditional density plots for continuous predictors f ( xi | y )
(Cook & Weisberg)
Consider,
1 if success
f ( x | y )  the conditional density of x given outcome variable y  
0 if failure
Idea:
91
Univariate considerations
f ( x | y)
Normal, common variance
i.e. Var ( x | y  0)  Var ( x | y  1)
Normal, unequal variances
i.e. Var ( x | y  0)  Var ( x | y  1) v
Skewed right
x  [0,1]
x is dichotomous, Bernoulli
x ~ Poisson, i.e. x is a count
Suggested model terms
x
x and x2
x and log2(x)
base 2 is easier to interpret
log(x) , log(1-x)
x
x
Multivariate considerations
When considering multiple continuous predictors simultaneously we look at multivariate
normality.
If
f ( x | y ) ~ MVN (  y k , )
~
f ( x | y ) ~ MVN (  y  k ,  y  k )
~
then use the x’s themselves
then include x i2 ’s and x i x j terms
For example in the two predictor case (p = 2)
x1 x2 is needed if E ( x1 | x 2 )   o  1, y  k x 2
and if the variances are different for the xi across levels of y then we add x i2 terms as
well.
92
AST
Clearly AST has skewed distribution and using the log 2 ( AST ) in the model would be
recommended.
After transformation we have
In f (log 2 ( AST ) | Outcome) appears to approximately normal for both levels with a
constant variance so quadratic terms in the log scale are not suggested.
93
CK
Clearly CK is extremely right skewed and would benefit from log transformation.
Again the conditional densities appear approximately normal with equal variance, so we
will consider adding log 2 (CK ) only to the model.
PCV
f ( PCV | Outcome) is approximately normal for both outcome groups but the variation in
PCV levels appear to be higher for cows that survived. Thus we will consider PCV and
PCV2 terms in the model.
94
Daysrec
Despite the fact that Daysrec is right skewed we will not log transform it. It represents a
count of the number of days the cow was recumbent, so it could be modeled as a Poisson
and thus the only term recommended is the Daysrec itself.
Urea
Consider the log transformation of urea level.
f (log 2 (Urea) | Outcome) is approximately normal however the variation for cows that
survived appears larger so we will consider both log 2 (Urea) and log 2 (Urea) 2 terms.
95
Data set = Downer, Name of Fit = B2
372 cases are missing at least one value. (PCV has lots of missing values also)
Binomial Regression
Kernel mean function = Logistic
Response
= Outcome
Terms
= (AST log2[AST] CK log2[CK] Urea log2[Urea] log2[Urea]^2
PCV PCV^2 Daysrec Calving)
Trials
= Ones
Coefficient Estimates
Label
Estimate
Std. Error
Est/SE
p-value
Constant
-1.03935
6.35298
-0.164
0.8700
AST
-0.000720027
0.00242524
-0.297
0.7666
log2[AST]
-0.330179
0.554239
-0.596
0.5514
CK
-0.000109772
0.000135315
-0.811
0.4172
log2[CK]
-0.0121434
0.223648
-0.054
0.9567
Urea
-1.13453
1.05860
-1.072
0.2838
log2[Urea]
0.730468
2.89371
0.252
0.8007
log2[Urea]^2
0.660165
1.38757
0.476
0.6342
PCV
0.182480
0.224691
0.812
0.4167
PCV^2
-0.00165620
0.00325722
-0.508
0.6111
Daysrec
-0.391937
0.157490
-2.489
0.0128
Calving
1.28561
0.648089
1.984
0.0473
Scale factor:
Number of cases:
Number of cases used:
Degrees of freedom:
Pearson X2:
Deviance:
1.
435
165
153
127.410
141.988
Clearly we have some model reduction to do, as many of the current terms are not
significant. Before backward eliminating we will drop all of the non-transformed
versions of log scale predictors.
Coefficient Estimates
Label
Estimate
Std. Error
Est/SE
Constant
-3.82598
5.84498
-0.655
log2[AST]
-0.554005
0.293416
-1.888
log2[CK]
-0.118575
0.160536
-0.739
log2[Urea]
4.09939
3.12355
1.312
log2[Urea]^2 -0.978895
0.545929
-1.793
PCV
0.218085
0.213730
1.020
PCV^2
-0.00229912
0.00305947
-0.751
Daysrec
-0.383179
0.153758
-2.492
Calving
1.39322
0.647605
2.151
Scale factor:
1.
Number of cases:
435
Number of cases used:
165
Degrees of freedom:
156
Pearson X2:
134.154
Deviance:
145.123
Backward Elimination: Sequentially remove terms
that give the smallest change in AIC.
All fits include an intercept.
p-value
0.5127
0.0590
0.4601
0.1894
0.0730
0.3075
0.4524
0.0127
0.0314
96
Current terms: (log2[AST] log2[CK] log2[Urea] log2[Urea]^2
Daysrec Calving)
df
Deviance
Pearson X2 |
Delete: log2[CK]
157
145.671
134.797
|
Delete: PCV^2
157
145.786
134.995
|
Delete: PCV
157
146.392
135.415
|
Delete: log2[Urea]
157
148.141
140.787
|
Delete: log2[AST]
157
148.92
140.737
|
Delete: Calving
157
150.163
141.672
|
Delete: Daysrec
157
151.993
135.976
|
Delete: log2[Urea]^2
157
152.536
143.299
|
k
8
8
8
8
8
8
8
8
Current terms: (log2[AST] log2[Urea] log2[Urea]^2 PCV
Calving)
df
Deviance
Pearson X2
Delete: PCV^2
158
146.202
135.813
Delete: PCV
158
146.701
136.211
Delete: log2[Urea]
158
149.035
142.035
Delete: Calving
158
151.207
140.587
Delete: Daysrec
158
152.168
136.078
Delete: log2[Urea]^2
158
153.767
145.12
Delete: log2[AST]
158
161.383
144.17
PCV^2 Daysrec
k
7
7
7
7
7
7
7
Current terms: (log2[AST] log2[Urea] log2[Urea]^2 PCV
df
Deviance
Pearson X2
Delete: PCV
159
148.955
137.789
Delete: log2[Urea]
159
150.035
144.626
Delete: Calving
159
152.176
141.179
Delete: Daysrec
159
152.699
136.298
Delete: log2[Urea]^2
159
155.31
149.108
Delete: log2[AST]
159
163.059
140.738
Daysrec
|
k
|
6
|
6
|
6
|
6
|
6
|
6
|
|
|
|
|
|
|
|
PCV PCV^2
AIC
161.671
161.786
162.392
164.141
164.920
166.163
167.993
168.536
AIC
160.202 *
160.701
163.035
165.207
166.168
167.767
175.383
Calving)
AIC
160.955
162.035
164.176
164.699
167.310
175.059
Current terms: (log2[AST] log2[Urea] log2[Urea]^2 Daysrec Calving)
df
Deviance
Pearson X2 |
k
AIC
Delete: log2[Urea]
160
152.373
144.523
|
5 162.373
Delete: Daysrec
160
155.744
138.388
|
5 165.744
Delete: Calving
160
155.99
142.871
|
5 165.990
Delete: log2[Urea]^2
160
157.017
148.417
|
5 167.017
Delete: log2[AST]
160
164.785
143.03
|
5 174.785
Current terms: (log2[AST] log2[Urea]^2 Daysrec Calving)
df
Deviance
Pearson X2 |
Delete: Calving
161
160.932
150.399
|
Delete: Daysrec
161
162.036
146.037
|
Delete: log2[AST]
161
169.755
148.817
|
Delete: log2[Urea]^2
161
176.794
157.24
|
k
4
4
4
4
AIC
168.932
170.036
177.755
184.794
Current terms: (log2[AST] log2[Urea]^2 Daysrec)
df
Deviance
Pearson X2 |
Delete: Daysrec
162
167.184
150.961
|
Delete: log2[AST]
162
178.021
150.618
|
Delete: log2[Urea]^2
162
181.641
162.028
|
k
3
3
3
AIC
173.184
184.021
187.641
Current terms: (log2[AST] log2[Urea]^2)
df
Deviance
Delete: log2[Urea]^2
163
182.688
Delete: log2[AST]
163
192.479
k
2
2
AIC
186.688
196.479
Pearson X2 |
162.386
|
151.943
|
97
Forward selection suggests the same model.
“Final” Model
Data set = Downer, Name of Fit = B5
372 cases are missing at least one value.
Binomial Regression
Kernel mean function = Logistic
Response
= Outcome
Terms
= (log2[AST] log2[Urea] log2[Urea]^2 PCV Daysrec Calving)
Trials
= Ones
Coefficient Estimates
Label
Estimate
Std. Error
Est/SE
p-value
Constant
-1.12404
5.01853
-0.224
0.8228
log2[AST]
-0.733670
0.196044
-3.742
0.0002
log2[Urea]
4.44950
3.17044
1.403
0.1605
log2[Urea]^2 -1.05918
0.554282
-1.911
0.0560
PCV
0.0514512
0.0335256
1.535
0.1249
Daysrec
-0.386695
0.153067
-2.526
0.0115
Calving
1.44641
0.623820
2.319
0.0204
Scale factor:
Number of cases:
Number of cases used:
Degrees of freedom:
Pearson X2:
Deviance:
1.
435
170
163
138.509
148.269
Diagnostics and Model Checking Plots
Chi-residuals vs. estimated logits ~ Looks good.
98
Cook’s Distance and Leverage vs. Case Numbers
Model Checking Plots (Estimated Logits and Marginals)
LOGIT
99
AST
UREA
100
PCV
DAYSREC
All of these plots look OK. The largest departure observed is in the case of urea but the
discrepancy there is primarily due to one observation stands out from the rest.
101
In R
To replicate the analysis above in R you will need the following functions to look and the
conditional densities:
𝑓(𝑥|𝑦 = 0) and 𝑓(𝑥|𝑦 = 1)
The first two functions are used to make pretty histograms in the conplot function. The
function conplot replicates the fitting density estimates conditional on the value of the
outcome variable Y, by taking the predictor X and the Y as arguments. If there are missing
values on either the response or the predictor those cases are automatically removed before
constructing the plot.
nclass.FD = function (x)
{
r <- quantile(x, c(0.25, 0.75))
names(r) <- NULL
h <- 2 * (r[2] - r[1]) * length(x)^{
-1/3
}
ceiling(diff(range(x))/h)
}
bandwidth.nrd = function (x)
{
r <- quantile(x, c(0.25, 0.75))
h <- (r[2] - r[1])/1.34
4 * 1.06 * min(sqrt(var(x)), h) * length(x)^(-1/5)
}
conplot = function (x, xname = deparse(substitute(x)),y)
{
xname <- deparse(substitute(x))
data = na.omit(cbind(x,y))
x = data[,1]
y = data[,2]
par(err = -1)
dens0 <- density(x[y==0], width = bandwidth.nrd(x[y==0]))
dens1 <- density(x[y==1], width = bandwidth.nrd(x[y==1]))
ylim <- range(c(dens0$y,dens1$y))
xlim <- range(c(dens0$x,dens1$x))
hist(x, nclass.FD(x), prob = T, xlab = xname, xlim = xlim,
ylim = ylim,main=paste("Conditional X|Y Plot of ",xname))
lines(dens0,col="blue")
lines(dens1,col="red")
invisible()
}
102
> conplot(x=AST,y=Outcome)
> conplot(x=log(AST),y=Outcome)
Etc…
To obtain model checking plots in R you will need to install the package car from the
CRAN which essentially is a collection of functions to replicate Arc in R. The two
functions that create model checking plots in the car library are called mmp and mmps,
the latter creates model checking plots for each predictor as well as the overall fit.
103
Downer Example in R
> mod1 = glm(Outcome~AST+Urea+PCV+Calving+Daysrec+CK,family="binomial")
> summary(mod1)
Call:
glm(formula = Outcome ~ AST + Urea + PCV + Calving + Daysrec +
CK, family = "binomial")
Deviance Residuals:
Min
1Q
Median
-1.7678 -0.7541 -0.1928
3Q
0.7546
Max
2.0696
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.3313771 1.1987644
0.276 0.78222
AST
-0.0022405 0.0014726 -1.521 0.12815
Urea
-0.3140380 0.0770497 -4.076 4.59e-05 ***
PCV
0.0601745 0.0339726
1.771 0.07652 .
Calving
1.3192777 0.6238318
2.115 0.03445 *
Daysrec
-0.4804961 0.1498000 -3.208 0.00134 **
CK
-0.0001435 0.0001121 -1.280 0.20068
--Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 212.71 on 164 degrees of freedom
Residual deviance: 146.39 on 158 degrees of freedom
(270 observations deleted due to missingness)
AIC: 160.39
Number of Fisher Scoring iterations: 7
> mmps(mod1)
104
Using the same approach as in the analysis in Arc we might use the model with several
terms based on predictor transformations.
>
>
>
>
>
logAST = log2(AST)
logCK = log2(CK)
logUrea = log2(Urea)
logUrea2 = logUrea^2
PCV2 = PCV^2
> Downer2 = data.frame(Outcome,logAST,logCK,logUrea,logUrea2,PCV,PCV2,Daysrec,Calving)
> Downer2 = na.omit(Downer2)
> attach(Downer2)
> mod2 = glm(Outcome~logAST+logCK+logUrea+logUrea2+PCV+PCV2+
Daysrec+Calving,family="binomial",data=Downer2)
> summary(mod2)
Call:
glm(formula = Outcome ~ logAST + logCK + logUrea + logUrea2 +
PCV + PCV2 + Daysrec + Calving, family = "binomial", data =
Downer2)
Deviance Residuals:
Min
1Q
Median
-1.9522 -0.7094 -0.2869
3Q
0.7109
Max
2.0585
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -3.826289
5.856671 -0.653
0.5135
logAST
-0.554007
0.293529 -1.887
0.0591 .
logCK
-0.118574
0.160621 -0.738
0.4604
logUrea
4.099642
3.137295
1.307
0.1913
logUrea2
-0.978940
0.548487 -1.785
0.0743 .
PCV
0.218083
0.213771
1.020
0.3076
PCV2
-0.002299
0.003060 -0.751
0.4525
Daysrec
-0.383178
0.153795 -2.491
0.0127 *
Calving
1.393222
0.647759
2.151
0.0315 *
--Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 212.71
Residual deviance: 145.12
AIC: 163.12
on 164
on 156
degrees of freedom
degrees of freedom
Number of Fisher Scoring iterations: 7
105
Backwards eliminate using the step() function
> mod3 = step(mod2)
Start: AIC=163.12
Outcome ~ logAST + logCK + logUrea + logUrea2 + PCV + PCV2 +
Daysrec + Calving
- logCK
- PCV2
- PCV
<none>
- logUrea
- logAST
- Calving
- Daysrec
- logUrea2
Df Deviance
AIC
1
145.67 161.67
1
145.79 161.79
1
146.39 162.39
145.12 163.12
1
148.14 164.14
1
148.92 164.92
1
150.16 166.16
1
151.99 167.99
1
152.54 168.54
Step: AIC=161.67
Outcome ~ logAST + logUrea + logUrea2 + PCV + PCV2 + Daysrec +
Calving
- PCV2
- PCV
<none>
- logUrea
- Calving
- Daysrec
- logUrea2
- logAST
Df Deviance
AIC
1
146.20 160.20
1
146.70 160.70
145.67 161.67
1
149.03 163.03
1
151.21 165.21
1
152.17 166.17
1
153.77 167.77
1
161.38 175.38
Step: AIC=160.2
Outcome ~ logAST + logUrea + logUrea2 + PCV + Daysrec + Calving
<none>
- PCV
- logUrea
- Calving
- Daysrec
- logUrea2
- logAST
Df Deviance
146.20
1
148.96
1
150.03
1
152.18
1
152.70
1
155.31
1
163.06
AIC
160.20
160.96
162.03
164.18
164.70
167.31
175.06
> summary(mod3)
Call:
glm(formula = Outcome ~ logAST + logUrea + logUrea2 + PCV + Daysrec +
Calving, family = "binomial", data = Downer2)
Deviance Residuals:
Min
1Q
Median
-2.0329 -0.6836 -0.2644
3Q
0.7002
Max
2.0893
106
Coefficients:
Estimate Std. Error z value
(Intercept) -1.49051
5.04944 -0.295
logAST
-0.72986
0.19514 -3.740
logUrea
4.61037
3.19802
1.442
logUrea2
-1.08728
0.55899 -1.945
PCV
0.05489
0.03370
1.629
Daysrec
-0.37191
0.15422 -2.411
Calving
1.45572
0.62845
2.316
--Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01
Pr(>|z|)
0.767853
0.000184
0.149406
0.051768
0.103394
0.015888
0.020538
***
.
*
*
‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 212.71
Residual deviance: 146.20
AIC: 160.2
on 164
on 158
degrees of freedom
degrees of freedom
Number of Fisher Scoring iterations: 7
> mmps(mod3)
107
We could consider adding interaction terms to our “final” model. This is easily done
using the scope option.
> mod3 = step(mod2,scope=~.^2)
> summary(mod3)
Call:
glm(formula = Outcome ~ logAST + logUrea + logUrea2 + PCV + PCV2
+
Daysrec + Calving + PCV:Calving + logAST:PCV2 + logAST:PCV,
family = "binomial", data = Downer2)
Deviance Residuals:
Min
1Q
Median
-1.8712 -0.6639 -0.1012
3Q
0.6784
Max
2.5298
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 44.950653 39.720672
1.132
0.2578
logAST
-9.472551
5.720701 -1.656
0.0978 .
logUrea
6.107698
3.425871
1.783
0.0746 .
logUrea2
-1.315449
0.602659 -2.183
0.0291 *
PCV
-3.814468
2.386123 -1.599
0.1099
PCV2
0.069737
0.036368
1.918
0.0552 .
Daysrec
-0.388998
0.162716 -2.391
0.0168 *
Calving
12.376954
6.037254
2.050
0.0404 *
PCV:Calving -0.322466
0.168853 -1.910
0.0562 .
logAST:PCV2 -0.010067
0.005179 -1.944
0.0519 .
logAST:PCV
0.608959
0.344917
1.766
0.0775 .
--Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 212.71
Residual deviance: 134.15
AIC: 156.15
on 164
on 154
degrees of freedom
degrees of freedom
Number of Fisher Scoring iterations: 7
108
> mmps(mod3)
There is a slight improvement in fit. The same model fit in JMP produces the following
ROC curve. The resulting classification is very good using this model.
109