Correlation CoeÆcient
Y
=0
X=1
X
( )
E (Y )
E (XY )
V (X )
V (Y )
E X
=
=
=
=
=
=0
Y
=1
11
12
21
22
Product moment correlation
coeÆcient:
q
= 21 + 22
2+ = 21 + 22
2+
22
(1
+2(1
2+
= Corr(X; Y )
= E (XYV)(XE)V(X(Y)E) (Y )
+2
= p22 2+
1+ 2++1+2
) = 1+2+
+2) = +1+2
2+
530
529
Estimation:
p p
r = p11 22
p p
p12p21
2+p+1p+2
Properties:
1. 1 1
2. Independence model ) = 0
3. = 1 when 12 = 21 = 0
= 01 when 11 = 22 = 0
4. is margin sensitive
=
1+
Y11Y22 Y12Y21
r
Y1+Y2+Y+1Y+2
Note that
and
X2
X2
2
r =
n
= n(YY11YY22
r
1+
for 2 2 tables.
531
)2
Y12Y21
2+Y+1Y+2
532
Measures of Association for
I J tables based on Pearson's
X2
2 =
Note that
X2
=
=
(ij
I X
J
X
i=1 j =1
Yij
I X
J
X
i=1 j =1
I X
J
X
n
i=1 j =1
where
)
i++j 2
i++j
Yi+Y+j
n
Yi+Y+j
n
(Pij
= Y++
Pi+ = Yi+=n
Then
2
)
Pi+P+j 2
Pi+P+j
^2 = X 2
n
is a consistent estimator of 2,
but
0 2 minfI 1; J 1g
= Yij =n
P+j = Y+j =n
n
Pij
533
Pearson's measure of mean
square contingency
P
v
u
u
u
u
t
V
r
Estimation:
^=
Cramer's V
2
= 2+ 1
Note that
0 P I I 1 for I I tables
P
534
v
u
u
u
u
t
2
= minfI 1; J 1g
0V 1
Esitmation:
X 2=n
^V =
minfI 1; J 1g
v
u
u
u
u
t
v
u
u
u
u
t
X2
X2
+
n
535
536
/*
This program is stored as
assoc.sas
Example: Association between diagnosis and treatment prescribed by psychiatrists in New Haven, Conn. (1950)
/*
*/
This program uses PROC FREQ
in SAS to test for independence
between diagnosis and perscribed
Treatment
Psycho- Organic Custodial
therapy Therapy
Care
Aective
30
102
280
Alcoholic
48
23
20
Organic
19
80
75
Schizophrenic 121
344
382
Senile
18
11
141
treatment in the 1950 New Haven
study. */
DATA SET1;
INPUT D T X;
LABEL D = DIAGNOSIS
T = TREATMENT;
CARDS;
1 1 30
1 2 102
538
537
5 2 11
5 3 141
RUN;
PROC FREQ DATA=SET1 ORDER=INTERNAL;
PROC PRINT DATA=SET1;
TITLE 'DATA FOR THE NEW HAVEN STUDY';
TABLES D*T / CHISQ MEASURES
SCORES=TABLE ALPHA=.05
NOPERCENT NOCOL EXPECTED;
PROC FORMAT;
WEIGHT X;
VALUE DFMT 1='AFFECTIVE'
FORMAT D DFMT.;
2='ALCOHOLIC'
FORMAT T TFMT.;
3='ORGANIC'
4='SCHIZOPHRENIC'
TITLE 'ANALYSIS OF THE NEW HAVEN DATA';
5='SENILE';
RUN;
VALUE TFMT 1='PSYCHOTHERAPY'
2='ORGANIC THERAPY'
3='CUSTODIAL CARE';
RUN;
539
540
ANALYSIS OF THE NEW HAVEN DATA
DATA FOR THE NEW HAVEN STUDY
Obs
D
T
X
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
1
1
1
2
2
2
3
3
3
4
4
4
5
5
5
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
30
102
280
48
23
20
19
80
75
121
344
382
18
11
141
Table of D by T
D(DIAGNOSIS)
Frequency
Expected
Row Pct
T(TREATMENT)
PSYCHO- ORGANIC CUSTODIAL Total
THERAPY THERAPY CARE
AFFECTIVE
30
57.398
7.28
102
136.2
24.76
280
218.4
67.96
412
ALCOHOLIC
48
23
12.678 30.083
52.75 25.27
20
48.24
21.98
91
ORGANIC
19
80
75
24.241 57.521 92.238
10.92 45.98 43.10
174
542
541
SCHIZOPHRENIC
SENILE
Total
382
449
45.10
847
Statistic
18
11
141
23.684 56.198 90.118
10.59
6.47 82.94
170
121
118
14.29
236
344
280
40.61
560
898
1694
Statistics for Table of D by T
Statistic
DF
Value
Prob
Chi-Square
8 259.9367 <.0001
Likelihood Ratio Chi-Square 8 238.6614 <.0001
Mantel-Haenszel Chi-Square 1
3.0511 0.0807
Phi Coefficient
0.3917
Contingency Coefficient
0.3647
Cramer's V
0.2770
543
Value
ASE
Gamma
Kendall's Tau-b
Stuart's Tau-c
-0.0066
-0.0042
-0.0040
0.0345
0.0219
0.0206
Somers' D C|R
Somers' D R|C
-0.0040
-0.0045
0.0206
0.0233
Pearson Correlation
Spearman Correlation
-0.0425
-0.0067
0.0235
0.0244
Lambda Asymmetric C|R
Lambda Asymmetric R|C
Lambda Symmetric
0.0415
0.0000
0.0201
0.0184
0.0000
0.0090
Uncertainty Coefficient C|R
Uncertainty Coefficient R|C
Uncertainty Coefficient Sym
0.0721
0.0537
0.0616
0.0091
0.0067
0.0077
Sample Size = 1694
544
Proportional Reduction in Error
(PRE)
2
6
6
6
6
6
6
4
3
7
7
7
7
7
7
5
2
6
6
6
6
6
6
4
minimum prob.
minimum prob.
of erroneous
of erroneous
prediction
prediction
assuming
under the
independence
alternative
2
3
P RE =
minimum probability of 7
6 erroneous prediction
4
5
assuming independence
3
7
7
7
7
7
7
5
Predicting the column category from
the row category
I
(1 +;max) (1 X i;max)
i=1
C=R =
1 +;max
with estimate
I
(1 P+;max) (1 X Pi;max)
i=1
^C=R =
1 P+;max
The large sample variance is estimated
as
2
^1
(^
C=R
2
6
6 (1
1 6
=
6
)
N6
4
I
X
i=1
X
Pi;max)(
i
Pi;max + P+;max
(1
P+;max)3
2
X
i
3
Pi;max ) 7
7
7
7
7
5
546
545
New Haven (1950) study:
P11 = :0177
P12 = :0602
P13 = :1653
P21 = :0283
P22 = :0136
P23 = :0118
P31 = :0112
P32 = :0472
P32 = :0443
P41 = :0714
P42 = :2031
P42 = :2255
P51 = :0106
P52 = :0065
P52 = :0832
P+1 = :1393
P+2 = :3306
P+3 = 5301
^
C=R
^1;^C=R
=
(1
:5301)
=
:4699 :0505
:4699
=
018
(1
P1;max
^R=C = (1
:5000)
= 0
(1 [:0714 + :2031 + 2255])
(1 :5000)
and
P+;max
(:1653 + :0283 + :0472 + :2255 + :0832))
(1 :5301)
= 041
2
^1
(^
R=C )
=0
In this case the 4-th row contains the
largest proportion of patients for each
column.
So it appears that C=R > 0.
547
548
Properties:
1. 0 C=R 1
2. C=R = 0
if knowledge of
the row category
does not improve
the probability of
correctly predicting
column category
3. C=R = 1
0
1/3
0
0
1/12
0
1/2
0
0
0
0
1/12
549
550
Other measures of association
for nominal categorical variables:
4. C=R is not always equal to
R=C
Agresti(1990, pp 22-26)
Lloyd(1999, pp 70-71)
Concentration coeÆcient:
5. C=R is not aected by permuting rows and/or columns.
This makes C=R a suitable
measure for tabales dened by
nominal variables.
XX
C=R
=
i j
2 =i+
ij
1
X
j
X
j
2j
+
2j
+
Uncertainty coeÆcient:
ij log(ij =i++j )
i j
UC=R =
+j log(+j )
j
XX
X
551
552
Measures of association for I J tables
dened by two ordinal variables.
Example:
Each student in a \random"
sample of N = 21 sociology students at the Univ. of
Michigan was cross-classied
with respect to responses to
two items.
Willingness
to join an
organization
Low
moderate
high
Concern with
proper behavior
Low moderate high
22
20
5 47
26
60
27 113
8
31
18 57
50
111
56
554
553
Consider the responses to these two
items for two dierent students
(i1; j1) gives the row and column categories for the responses of
student 1.
(i2; j2) gives the row and column categories for the responses of
student 2.
Concordant pair of students:
either i1 > i2 j1 > j2
or i1 < i2 j1 < j2
Discordant pair of students:
either i1 > i2 j1 > j2
or i1 < i2 j1 < j2
Neither concordant nor discordant:
i1 = i2 and/or j1 = J2
555
Concordant Pairs
Low ! High
Low 22
#
60 27
High
31 18
22(60 + 27 + 31 + 18)
20
27
18
20(27 + 18)
556
Low ! High
Low 22
#
60
High
18
5
none
26
60(66 + 18)
22 20
31 18
26(31 + 18)
27
27(22 + 20)
557
8
22
26
558
22 20
26 60
31
None
18(22 + 20 +26 + 60)
31
31(22 + 26)
559
560
Discordant Pairs
26 60
8 31
22
5(26 + 60 + 8 + 31)
None
26
8
5
20
20
26
5
26(20 + 5)
20(26 + 8)
562
561
8
60
8 31
5
27
27(8 + 31)
60(5 + 8)
8
20 5
60 27
8(20 + 5 + 60 + 27)
563
564
31
Number of concordant pairs:
5
27
P
=
2
X X
Yij 64
Yk`
i=1 j =1
k>i `>j
I X
J
X
+
= 12; 492
31(5 + 27)
X X
k<i `<j
Number of discordant pairs:
Q
None
=
2
X X
Yij 64
Yk`
i=1 j =1
k>i `<j
I X
J
X
+
= 5; 676
18
X X
k<i `>j
3
Yk`75
566
565
Willingness
to join an
organization
Low
moderate
high
3
Yk`75
Kendall's Taub:
Concern with
proper behavior
b =
r
N (N
1)
X
i
Low moderate high
22
20
5 47
26
60
27 113
8
31
18 57
50
111
56
567
P
Yi+(Yi+
sQ
1)
N (N
1)
X
Y+j (Y+j
j
Goodman-Kruskal Gamma:
^ = PP + QQ
12; 492 5; 676 = 0:375
= 12
; 492 + 5; 676
568
1)
Properties of ^ :
Large sample variance estimate: ^1; =
2
16
(P +Q)4
hXX
P
j
j
Low
1. ^ = 1
#
#
^
n X X
XX
Yij Q
Yk` +
Yk`
X X
k>i `<j
k>i `>j
Yk` +
XX
k<i `>j
High
k<i `<j
Yk`
Low
0
10
0
0
o2i
Low
2. ^ = 1
#
#
High
! High
0
0
15
0
Low
0
27
0
0
0
0
0
30
! High
18
0
0
0
0
0
0
0
570
569
3. ^ = 0 for the case where
Pij = Pi+P+j .
Also for other cases,
e.g.,
Spearman's rho (corrected for
ties)
10 0 10
0 47 0
10 0 10
XX
^ =
i
v
u
uh 3
1 t
N
12
4. j^ j j^bj
j
Yij
N
hX
k<i
Y
Yk+ + i+
2
X
i
Yi3+
Yi+
Ni hX
2
ih
`<j
N3
Y+e +
N
Y+j
Ni
2
2
J X
j =1
Y+3j
5. For 2 2 tables, ^ is the estimate
of Yule's Q.
571
572
Y+j
i
Product-moment correlation
coeÆcient:
I X
J Y
X
ij
(r r)(c c)
i
j
1
j1 N
v
r=v
uI
u J
uX Y
u X Y+j
u
i
+
2
t
t
(
ri r) u
(cj
i1 N
j =1 N
i
c)2
where r1; r2; ; rI are the row scores
and c1; c2; ; cJ are the column scores
I
r = 1 X Y r
N i=1
c = N1
Consider ri = i
cj = j
J
X
j =1
i+
i
Measure of agreement for I I
tables
A
A
B
Judge 1 C
D
Y+j cj
Judge 2
B C
D
11
12
13
14
1+
21
22
23
24
2+
31
32
33
34
3+
41
42
43
44
4+
+1
+2
+3
+4
= 1; 2; : : : ; I
= 1; 2; : : : ; J
i
j
573
574
Cohen's Kappa:
I
X
K
= i=1
1
where
1
2
=
I
X
ii
I
X
i=1
I
X
i=1
i++i
i++i
= 1
1
Estimation:
2
2
\actual"
probability
of agreement
I
X
= i++i probability of
i=1
chance agreement for independent
classicatons
i=1
ii
575
^=
K
=
I
X
i
=1
Pii
1
I
X
=1 i+ +i
I
X
Pi+P+i
i=1
i
X
P
N i Yii
N2
X
i
i
P
P
Yi+Y+i
Yi+Y+i
576
Estimate of the large sample
variance:
^1 =
Pii
i
^2 =
Pi+P+i
i
^3 =
Pii(Pi+ + P+i)
i
^4 =
Pij (P+i + Pj +)2
i j
^12 ;^k = N1 [^(11(1 ^ ^)12)
2
^1)(2^1^2 ^3)
+2(1 (1
^2)3
2
2
+ (1 ^(11) (^^4)4 4^2)]
2
X
X
Rating of student teachers by two supervisors (Gross, 1971, BFH, Chap.
11)
Supervisor 2
X
XX
Supervisor 1
Authoritarian
Authoritarian
Democratic
Permissive
democratic
Permissive
17
4
8
29
5
12
0
17
10
3
13
26
32
19
21
N = 72
^ = 0:361
^1;^ = p:0084 = :0915
577
578
Properties of Kappa:
An approximate 95% condence
interval for Kappa:
1. ^ = 0
if Pii = Pi+P+i;
i = 1 ; 2; : : :
2. 1 ^ 1
I
3. ^ = 1 if i=1
Pii = 1
^ (1:96)^1;^
:361 :179
X
0
0 P22 0
0 0 P33
P11
) (:18; :54)
579
0
580
^ = 1
X
Pi+P+i
iX
i
Pi+P+i
when there is \no agreement".
Kappa is sensitive to
marginal distributions.
40 9 49
6 45 51
46 54
^ = :70
P11 + P22
= :85
80 10 90
5 5 10
85 15
^ = :32
P11 + P22
= :85
45 15 60
25 15 40
70 30
^ = 0:13
P11 + P22
= :60
25 35 60
5 35 40
30 70
^ = 0:26
P11 + P22
= :60
582
581
Weighted Kappa
/* This program is stored as kappa.sas */
XX
w
=
i j
wij ij
1
XX
j j
XX
i j
wij i++j
wij i++j
Choices of weights:
1 i = j ) Kappa
wij =
0 i 6= j
1 i=j
wij = 1=2 j = i + 1 or j = j 1
0 otherwise
(i j )2
wij = 1
(I 1)2
8
>
<
>
:
8
>
>
>
>
>
<
>
>
>
>
>
:
583
/* First Use PROC FREG in SAS to
compute kappa for the student
teacher ratings. There are two
options for specifying weights */
data set1;
input sup1 sup2 count;
cards;
1 1 17
1 2 4
1 3 8
2 1 5
2 2 12
2 3 0
3 1 10
3 2 3
3 3 13
run;
584
/* This part of the program uses PROC IML
in SAS to compute either Kappa or a
weighted kappa and the corresponding
standard errors. It is applied to
the student teacher rating data. */
proc format;
value rating 1=Authoritarian
2=Democratic
3=Permissive;
run;
PROC IML;
START KAPPA;
proc freq data=set1;
tables sup1*sup2 / agree(wt=ca)
printkwt alpha=.05 nocol norow;
weight count;
format sup1 rating. sup2 rating.;
run;
/* ENTER TABLE OF COUNTS */
X = { 17 4 8,
5 12 0,
10 3 13};
proc freq data=set1;
tables sup1*sup2 / agree(wt=fc)
printkwt alpha=.05 nocol norow;
weight count;
format sup1 rating. sup2 rating.;
run;
/* ENTER THE TABLE OF WEIGHTS;
USE AN IDENTITY MATRIX IF YOU
DO NOT WANT WEIGHTED KAPPA */
W = { 1.0 0.5 0.0 ,
0.5 1.0 0.5 ,
0.0 0.5 1.0 };
585
586
/* COMPUTE KAPPA */
/* BEGINNING OF MODULE TO COMPUTE
KAPPA AND WEIGHTED KAPPA */
K1 = SUM(DIAG(X))/T;
K2 = SUM(DIAG(E))/(T**2);
K3 = SUM(DIAG(X)*DIAG(XR+XC`))/(T**2);
J1 = J(1,NR);
J2 = J(NC,1);
TT1 = ((XC`*J1+J2*XR`)##2)#X;
K4 = SUM(TT1)/(T**3);
KAPPA = (K1 - K2)/(1 - K2);
/* COMPUTE NUMBER OF ROWS AND
NUMBER OF COLUMNS FOR X */
NR = NROW(X);
NC = NCOL(X);
/* COMPUTE ROW AND COLUMN TOTALS
FOR THE MATRIX OF COUNTS */
/* COMPUTE STANDARD ERRORS:
S1 DOES NOT ASSUME INDEPENDENCE:
S2 ASSUMES THE NULL HYPOTHESIS
OF INDEPENDENCE IS TRUE */
XR = X(| ,+|);
XC = X(|+, |);
/* COMPUTE TABLE OF EXPECTED COUNTS
FOR INDEPENDENT RANDOM AGREEMENT */
S1 = (K1*(1-K1)/((1-K2)**2)+2*
(1-K1)*(2*K1*K2-K3)/((1-K2)**3)+
((1-K1)**2)*(K4-4*K2*K2)/
((1-K2)**4))/T;
S1 = SQRT(S1);
S2 = (K2 + K2*K2 -(SUM(DIAG(E)*DIAG(XR+XC`))
/(T**3)))/(T*(1-K2)**2);
S2 = SQRT(S2);
E = XR*XC;
/* COMPUTE OVERALL TOTAL COUNT */
T = SUM(X);
587
588
/* COMPUTE WEIGHTED KAPPA */
XW = X#W;
EW = E#W;
WR = (W*XC`) / T;
WC = (XR`*W) / T;
KW1 = SUM(XW)/T;
KW2 = SUM(EW)/(T**2);
KAPPAW = (KW1 - KW2)/(1 - KW2);
TT2 = (WR*J2`+J1`*WC);
TT3 = (W*(1-KW2)-TT2*(1-KW1))##2;
/* COMPUTE 95% CONFIDENCE INTERVALS
AND TESTS OF THE HYPOTHESIS THAT
THERE IS ONLY RANDOM AGRREMENT */
TK = KAPPA/S2;
TKW = KAPPAW/SW2;
TT4 = TK**2;
PK = 1 - PROBCHI(TT4,1);
TT4 = TKW**2;
PKW = 1 - PROBCHI(TT4,1);
CKL = KAPPA - (1.96)*S1;
CKU = KAPPA + (1.96)*S1;
CKWL = KAPPAW - (1.96)*SW1;
CKWU = KAPPAW + (1.96)*SW1;
/* COMPUTE STANDARD ERRORS:
SW1 DOES NOT ASSUME INDEPENDENCE:
SW2 ASSUMES THE NULL HYPOTHESIS
OF INDEPENDENCE IS TRUE */
SW1 = SUM(X#TT3)/T;
SW1 = (SW1 -(KW1*KW2-2*KW2+KW1)##2)
/(T*(1-KW2)**4);
SW1 = SQRT(SW1);
SW2 = (W-TT2)##2;
SW2 = ((SUM(E#SW2)/T**2)-(KW2##2))
/(T*(1-KW2)**2);
SW2 = SQRT(SW2);
589
590
The FREQ Procedure
/* PRINT RESULTS */
PRINT ,,,"Unweighted Kappa statistic " KAPPA;
PRINT ,"
Standard error " S1;
PRINT ,"95% confidence interval (" CKL "," CKU ")";
PRINT ,,,"Standard error when there ";
PRINT
"is only random agreement " S2;
PRINT ,,,"P-value for test of ";
PRINT
"completely random agreement" PK;
PRINT ,,," Weighted Kappa statistic " KAPPAW;
PRINT ,"
Standard error " SW1;
PRINT ,"95% confidence interval (" CKWL "," CKWU ")";
PRINT ,,,"Standard error when there";
PRINT
"is only random agreement " SW2;
PRINT ,,,"P-value for test of ";
PRINT
"completely random agreement" PKW;
FINISH;
RUN KAPPA;
591
Table of sup1 by sup2
Frequency
Percent
Authoritarian
Authori Democra Permiss Total
tarian
tic
ive
17
23.61
4
5.56
8
11.11
29
40.28
Democratic
5
6.94
12
16.67
0
0.00
17
23.61
Permissive
10
13.89
3
4.17
13
18.06
26
36.11
Total
32
44.44
19
26.39
21
72
29.17 100.00
592
Statistics for Table of sup1 by sup2
Test of Symmetry
Statistic (S)
DF
Pr > S
Kappa Coefficient Weights
(Fleiss-Cohen Form)
3.3333
3
0.3430
Kappa Coefficient Weights
Authori
tarian
Authoritarian 1.0000
Democratic
0.5000
Permissive
0.0000
Democra
tic
Permiss
ive
0.5000
1.0000
0.5000
0.0000
0.5000
1.0000
Simple Kappa
Weighted Kappa
Value
ASE
0.3623
0.2842
0.0907
0.1042
Democra
tic
Permiss
ive
1.0000
0.7500
0.0000
0.7500
1.0000
0.7500
0.0000
0.7500
1.0000
Kappa Statistics
Statistic
Value
Simple Kappa
Weighted Kappa
Kappa Statistics
Statistic
Authoritarian
Democratic
Permissive
Authori
tarian
95% Confidence
Limits
0.1844 0.5401
0.0800 0.4883
0.3623
0.2156
ASE
95% Confidence
Limits
0.0907 0.1844 0.5401
0.1250 -0.0295 0.4606
Sample Size = 72
593
KAPPA
Unweighted Kappa statistic 0.3622675
KAPPAW
Weighted Kappa statistic 0.2841756
S1
Standard error 0.0907466
SW1
Standard error 0.1041712
CKL
CKU
95% confidence interval ( 0.1844041 , 0.5401309 )
CKWL
CKWU
95% confidence interval ( 0.0800002 , 0.4883511 )
Standard error when there
Standard error when there
is only random agreement
594
S2
0.0836836
is only random agreement
SW2
0.0962934
P-value for test of
P-value for test of
PKW
completely random agreement 0.003166
PK
completely random agreement 0.000015
595
596
#
#
#
#
#
#
This file contains Splus code
for computing a Kappa statistic,
or weighted Kappa statistic,
standard errors and confidence
intervals. It is applied to the
student teacher data.
#
The file is stored as
#
Enter the observed counts
# Compute expected counts for random
# agreement
n <- sum(x)
xr <- apply(x, 1, sum)
xc <- apply(x, 2, sum)
one <- rep(1, length(xr))
e <- outer(xr, xc)/n
kappa.ssc
# Compute Kappa
x <- matrix(c(17, 4, 8, 5, 12, 0,
10, 3, 13), 3, 3, byrow=T)
#
k1
k2
k3
k4
Enter the weights
w<-matrix(c(1.0, 0.5, 0.0, 0.5, 1.0,
0.5, 0.0, 0.5, 1.0), 3, 3, byrow=T)
<<<<-
sum(diag(x))/n
sum(diag(e))/n
sum(diag(x)*diag(xr+xc))/(n*n)
sum(((outer(xc, one)+
outer(one, xr))**2)*x)/(n**3)
kappa <- (k1-k2)/(1-k2)
597
598
# Compute weighted Kappa
# Compute standard errors:
#
s1 does not assume random agreement
#
s2 assumes only random agreement
s11 <- (k1*(1-k1)/((1-k2)**2)+2*(1-k1)*
(2*k1*k2-k3)/((1-k2)**3)+
((1-k1)**2)*(k4-4*k2*k2)/((1-k2)**4))/n
s1 <- s11**.5
s22 <- (k2+k2*k2-(sum(diag(e)*diag(xr+xc))
/(n**2)))/(n*(1-k2)**2)
s2 <- s22**.5
599
xw
ew
wr
wc
<<<<-
x*w
e*w
apply(w*xc, 2, sum)/n
apply(w*xr, 2, sum)/n
kw1 <kw2 <tt2 <tt3 <kappaw
sum(xw)/n
sum(ew)/n
outer(wr, one)+outer(one, wc)
((w*(1-kw2))-(tt2*(1-kw1)))**2
<- (kw1-kw2)/(1-kw2)
# Compute standard errors:
#
sw11 does not assume random agreement
#
sw22 assumes only random agreement
sw11 <- sum(x*tt3)/n
sw11 <- (sw11-(kw1*kw2-2*kw2+kw1)**2)/
(n*(1-kw2)**4)
sw1 <- sw11**.5
sw22 <- (w-tt2)**2
sw22 <- ((sum(e*sw22)/n)-(kw2**2))/
(n*(1-kw2)**2)
sw2 <- sw22**.5
600
# print results
cat("\n", "
Unweighted Kappa = ",
signif(kappa,5))
cat("\n", "
Standard error = ",
signif(s1,5))
cat("\n", "95% confidence interval: ",
signif(ckl,5), signif(cku,5))
cat("\n", "p-value for test of random ",
"agreement = ", signif(pk,5))
cat("\n", "
Weighted Kappa = ",
signif(kappaw,5))
cat("\n", "
Standard error = ",
signif(sw1,5))
cat("\n", "95% confidence interval: ",
signif(ckwl,5), signif(ckwu,5))
cat("\n", "p-value for test of random ",
" agreement = ", signif(pkw,5))
# Construct 95% confidence intervals
# and tests for random agreement
tk <- kappa/s2
tkw <- kappaw/sw2
tt4 <- tk**2
pk <- (1-pchisq(tt4, 1))
tt4 <- tkw**2
pkw <-(1-pchisq(tt4, 1))
ckl <- kappa-(1.96)*s1
cku <- kappa+(1.96)*s1
ckwl <- kappaw-(1.96)*sw1
ckwu <- kappaw+(1.96)*sw1
601
You can source this code into S-PLUS and obtain
the results by issuing the following command at
the prompt in the S-PLUS command window:
source("yourdirectory/kappa.ssc")
The results are shown below.
Unweighted Kappa =
Standard error =
95% confidence interval:
p-value for test of
random agreement =
Weighted Kappa =
Standard error =
95% confidence interval:
p-value for test of
random agreement =
0.36227
0.090747
0.1844 0.54013
1.4978e-005
0.28418
0.10417
0.08 0.48835
0.00316
603
602