RAILS
Institution #3 Data Analysis
Bei Yu & Megan Oakleaf
Table: librarian: 1; faculty: 2
institute
3
A
1
B
1
C
1
D
2
E
2
F
2
G
2
H
2
J
2
K
2
L
2
(1) Kripendorff’s Alpha measure:
All judges:
criterion
124366
124367
124368
Total score
Kripendorff’s Alpha
Librarian(1)
.1740
.1967
.1173
.2274
All judges
.1881
.2265
.1874
.2395
Faculty(2)
.1788
.2501
.2068
.2430
(2) Cohen’s Kappa
pairwise Kappa agreement on criterion code 124366:
Mean Kappa for all judges=.122
3A
3B
3C
3D
3E
3F
3G
3H
3J
3K
3B
-.001
3C
.038
.333
3D
.052
.118
.194
3E
.011
-.066
.002
.076
3F
.008
.161
.152
.146
-.136
Mean Kappa for librarian=.123
3A
3B
3B
-.001
3C
.038
.333
1
3G
.062
.340
.294
.015
-.084
.223
3H
-.031
.322
.245
.077
-.052
.232
.236
3J
.090
.125
.170
.150
.077
.098
.192
.138
3K
.114
.004
.107
.409
.028
.093
.035
.083
.274
3L
-.084
.358
.367
.039
-.145
.100
.454
.271
.061
.118
Mean Kappa for faculty=.115
3D
3E
3F
3G
3H
3J
3K
3E
.076
3F
.146
-.136
3G
.015
-.084
.223
3H
.077
-.052
.232
.236
3J
.150
.077
.098
.192
.138
3K
.409
.028
.093
.035
.083
.274
3L
.039
-.145
.100
.454
.271
.061
.118
pairwise Kappa agreement on criterion code 124367:
Mean Kappa for all judges=.170
3A
3B
3C
3D
3E
3F
3G
3H
3J
3K
3B
.022
3C
.107
.244
3D
-.001
.168
.183
3E
.081
-.088
-.130
.048
3F
.055
.203
.177
.211
-.123
3G
.036
.304
.327
.132
-.037
.170
3H
.079
.351
.269
.197
-.067
.146
.314
3J
.029
.329
.236
.164
.031
.116
.352
.238
3K
.053
.206
.184
.332
.070
.300
.199
.208
.305
3L
.128
.306
.424
.170
.002
.175
.490
.415
.242
.286
Mean Kappa for librarian=.124
3A
3B
3B
.022
3C
.107
.244
Mean Kappa for faculty=.182
3D
3E
3F
3G
3H
3J
3K
3E
.048
3F
.211
-.123
3G
.132
-.037
.170
3H
.197
-.067
.146
.314
pairwise Kappa agreement on criterion code 124368:
2
3J
.164
.031
.116
.352
.238
3K
.332
.070
.300
.199
.208
.305
3L
.170
.002
.175
.490
.415
.242
.286
Mean Kappa for all judges=.138
3A
3B
3C
3D
3E
3F
3G
3H
3J
3K
3B
-.074
3C
.112
.199
3D
-.009
.121
.114
3E
-.049
.032
-.017
.098
3F
.034
.255
.275
.178
-.006
3G
-.024
.279
.291
.135
.008
.276
3H
-.170
.349
.186
.164
-.039
.259
.269
3J
.073
.181
.142
.119
-.083
.206
.166
.175
3K
.010
.152
.032
.284
-.112
.231
.195
.206
.212
3L
-.005
.322
.421
.174
-.006
.157
.386
.274
.182
.239
Mean Kappa for librarian=.079
3B
-.074
3A
3B
3C
.112
.199
Mean Kappa for faculty=.155
3E
.098
3D
3E
3F
3G
3H
3J
3K
3F
.178
-.006
3G
.135
.008
.276
3H
.164
-.039
.259
.269
3J
.119
-.083
.206
.166
.175
3K
.284
-.112
.231
.195
.206
.212
3L
.174
-.006
.157
.386
.274
.182
.239
Summary of Cohen’s Kappa
criterion
124366
124367
124368
Total score
Librarian(1)
.123
.124
.079
n/a
Faculty(2)
.115
.182
.155
n/a
All
.122
.170
.138
n/a
(3) Pearson’s correlation
Pairwise correlation regarding criterion code “124366”
Mean correlation for all judges=.212
3B
3C
3D
3E
3F
3G
3
3H
3J
3K
3L
3A
3B
3C
3D
3E
3F
3G
3H
3J
3K
**.
*.
.046
.160
.506**
.098
.190
.267**
-.067
-.051
-.069
.055
-.181
.346**
.301**
-.004
-.221*
.090
.517**
.472**
.179
-.175
.461**
-.076
.529**
.354**
.212*
-.086
.307**
.436**
.159
.328**
.398**
.269**
-.103
.039
.335**
.285**
.068
.192
.267**
.551**
-.103
.114
.283**
.182
.367**
.035
.535**
.541**
.235**
-.132
.476**
.608**
.428**
.363**
.343**
Correlation is significant at the 0.01level
Correlation is significant at the 0.05 level
Mean correlation for librarian=.237
3B
.046
3A
3B
**.
3C
.160
.506**
Correlation is significant at the 0.01level
Mean correlation for faculty=.204
3D
3E
3F
3G
3H
3J
3K
**.
*.
3E
.055
3F
-.004
-.221*
3G
.179
-.175
.461**
3H
.212*
-.086
.307**
.436**
3J
.269**
-.103
.039
.335**
.285**
3K
.551**
-.103
.114
.283**
.182
.367**
3L
.235**
-.132
.476**
.608**
.428**
.363**
.343**
Correlation is significant at the 0.01level
Correlation is significant at the 0.05 level
Pairwise correlation regarding criterion code “124367”
Mean correlation for all judges=.276
3A
3B
3C
3D
3E
3F
3G
3H
3J
3K
**.
*.
3B
.112
3C
.117
.447**
3D
-.023
.247*
.286**
3E
.061
-.024
-.145
-.022
3F
.004
.342**
.359**
.290**
-.090
Correlation is significant at the 0.01level
Correlation is significant at the 0.05 level
4
3G
-.011
.507**
.490**
.329**
-.153
.367**
3H
.047
.555**
.410**
.363**
-.009
.356**
.440**
3J
.162
.468**
.386**
.366**
.079
.253*
.468**
.446**
3K
.118
.385**
.343**
.524**
-.050
.434**
.378**
.418**
.429**
3L
.077
.496**
.574**
.423**
-.083
.371**
.626**
.585**
.426**
.446**
Mean correlation for librarian=.225
3B
.112
3A
3B
**.
3C
.117
.447**
Correlation is significant at the 0.01level
Mean correlation for faculty=.300
3D
3E
3F
3G
3H
3J
3K
**.
*.
3E
-.022
3F
.290**
-.090
3G
.329**
-.153
.367**
3H
.363**
-.009
.356**
.440**
3J
.366**
.079
.253*
.468**
.446**
3K
.524**
-.050
.434**
.378**
.418**
.429**
3L
.423**
-.083
.371**
.626**
.585**
.426**
.446**
Correlation is significant at the 0.01level
Correlation is significant at the 0.05 level
Pairwise correlation regarding criterion code “124368”
Mean correlation for all judges=.214
3A
3B
3C
3D
3E
3F
3G
3H
3J
3K
**.
*.
3B
-.047
3C
.141
.317**
3D
-.142
.227*
.226*
3E
-.088
-.038
-.160
.170
3F
-.075
.417**
.468**
.109
-.231*
3G
-.105
.489**
.497**
.194
-.115
.521**
3H
-.150
.487**
.409**
.157
-.123
.350**
.448**
3J
-.023
.363**
.118
.291**
-.042
.221*
.295**
.315**
3K
-.084
.339**
.197*
.387**
-.132
.366**
.284**
.272**
.424**
3L
.040
.563**
.533**
.314**
-.148
.549**
.578**
.482**
.412**
.497**
Correlation is significant at the 0.01level
Correlation is significant at the 0.05 level
Mean correlation for librarian=.137
3A
3B
**.
3B
-.047
3C
.141
.317**
Correlation is significant at the 0.01level
Mean correlation for faculty=.244
3D
3E
3F
3G
3H
3E
.170
3F
.109
-.231*
3G
.194
-.115
.521**
3H
.157
-.123
.350**
.448**
5
3J
.291**
-.042
.221*
.295**
.315**
3K
.387**
-.132
.366**
.284**
.272**
3L
.314**
-.148
.549**
.578**
.482**
.424**
3J
3K
**.
*.
.412**
.497**
Correlation is significant at the 0.01level
Correlation is significant at the 0.05 level
Pairwise correlation regarding total scores
Mean correlation for all judges=.270
3A
3B
3C
3D
3E
3F
3G
3H
3J
3K
**.
*.
3B
.073
3C
.146
.491**
3D
-.017
.234*
.317**
3E
-.056
-.037
-.135
.056
3F
-.085
.415**
.447**
.135
-.205*
3G
-.038
.577**
.560**
.261**
-.169
.528**
3H
-.071
.604**
.437**
.308**
-.097
.392**
.487**
3J
.113
.471**
.348**
.380**
-.018
.197*
.447**
.402**
3K
.063
.360**
.345**
.611**
-.105
.314**
.387**
.383**
.454**
3L
.066
.581**
.583**
.354**
-.134
.519**
.655**
.539**
.480**
.522**
Correlation is significant at the 0.01level
Correlation is significant at the 0.05 level
Mean correlation for librarian=.237
3A
3B
**.
3B
.073
3C
.146
.491**
Correlation is significant at the 0.01level
Mean correlation for faculty=.289
3D
3E
3F
3G
3H
3J
3K
**.
*.
3E
.056
3F
.135
-.205*
3G
.261**
-.169
.528**
3H
.308**
-.097
.392**
.487**
3J
.380**
-.018
.197*
.447**
.402**
3K
.611**
-.105
.314**
.387**
.383**
.454**
Correlation is significant at the 0.01level
Correlation is significant at the 0.05 level
Summary of Mean Pearson’s Correlation
Criterion
124366
124367
124368
Total
Librarian(1)
.237
.225
.137
.237
Faculty(2)
.204
.300
.244
.289
6
All judges
.212
.276
.214
.270
3L
.354**
-.134
.519**
.655**
.539**
.480**
.522**