Sensitivity and Specificity
Part II – Computations and
Examples
This video is designed to accompany
pages 81-94
in
Making Sense of Uncertainty
Activities for Teaching Statistical Reasoning
Van-Griner Publishing Company
ImPACT Example
Concern about concussions in young athletes is at an alltime high. A variety of screening tests are available for
assessing concussion status, some are to be used on
the field; others take more time. One such test is the
“ImPACT” test, which stands for “Immediate PostConcussion Assessment and Cognitive Testing.”
How well does ImPACT perform as a screening test?
ImPACT Example
“Sensitivity and specificity of the ImPACT Test Battery for concussion in
athletes.” Archives of Clinical Neuropsychology, 2005
Rule: Beyond a certain level on the
ImPACT scale you are said to be
concussed.
Actual Status
Athlete was, in
reality, not
concussed
Athlete was, in
reality, concussed
Totals
ImPACT said “not
concussed”
59
13
72
ImPACT said “concussed”
7
59
66
Totals
66
72
138
ImPACT Prediction
FPR = 7/66 = 0.11 or 11%
FNR = 13/72 = 0.18 or 18%
Specificity =
Sensitivity =
89%
82%
Beck Inventory Example
The original BDI, first published in 1961, consisted of
twenty-one questions about how the subject has been
feeling in the last week. A typical question is below.
Choose the most appropriate response
(0) I do not feel sad.
(1) I feel sad.
(2) I am sad all the time and I can't snap out of it.
(3) I am so sad or unhappy that I can't stand it.
How well does Beck perform as a screening test?
Beck Inventory Example
“Sensitivity and Specificity of Depression Questionnaires in a College-Age
Sample.” Journal of Genetic Psychology, 2008, 169(3), 281–288
Rule: If your total score on the Beck
Inventory is 10 or greater then you will be
categorized as clinically depressed.
Actual Status
as determined by
Diagnostic and Statistical Manual of Mental Disorders
IV
Patient was, in
reality, not
depressed
Patient was, in
reality, depressed
Totals
Beck said “not
depressed”
66
5
71
Beck said “depressed”
12
12
24
Totals
78
17
95
Beck Inventory
Indication
FPR = 12/78 = 0.14 or 14%
FNR = 5/17 = 0.29 or 29%
Specificity =
Sensitivity =
85%
71%
Field Sobriety Testing
There are three common testing procedures implemented
during an FST: horizontal gaze nystagmus (HGN), one-leg
stand (OLS), and walk and turn (WAT).
Data from 296 subjects who participated in the NHTSA’s
1998 San Diego field sobriety test validation study, ordered
by total scores on these three FST procedures, are
available .
Let’s set a cutoff for the field rule and assess how well the
rule is working as a screening test for intoxication.
Rules
For The Test: assume that a 4 or above on the FST will
tag a participant as drunk.
Concerning The Truth: assume that a BAC of 0.04% or
greater means a participant is legally drunk. There were
267 study participants with BACs of 0.04% or greater.
Task at Hand
Fill out this table!
Using Total FST >=4 to Designate
“Drunk”
Actual BAC
FST Decision
Totals
< 0.04 (Sober) >= 0.04 (Drunk)
Sober
Drunk
Totals
29
267
296
Field Sobriety Testing
Using Total FST >=4 to Designate
“Drunk”
FST Decision
Sober
Actual BAC
< 0.04
Totals
9
11
20
20
256
27
6
Drunk
Totals
>= 0.04
29
267
296
FPR = 20/29 or 69% Specificity = 9/29 or 31%
FNR = 11/267 or 4% Sensitivity = 256/267 or 96%
Case
229
254
66
142
217
191
182
109
259
199
113
22
67
145
53
15
287
89
123
258
35
11
247
6
294
231
74
214
58
14
34
12
211
232
293
130
297
271
119
HGN
0
0
2
2
2
2
2
2
0
2
2
2
0
2
2
2
2
0
2
2
2
4
2
2
2
2
2
4
2
4
2
2
4
2
2
4
2
OLS
0
0
1
0
0
WAT
1
1
0
0
0
1
0
0
1
0
3
0
0
0
1
1
1
0
1
1
0
1
1
1
0
2
0
2
1
0
0
0
1
1
2
2
2
1
1
1
0
0
1
0
2
1
1
0
2
1
0
0
1
0
2
2
2
0
0
Total FST Actual BAC
1
0
1
0.02
1
0.067
2
0.005
2
0.03
2
0.034
2
0.038
2
0.04
2
0.04
2
0.048
2
0.05
2
0.06
3
0.022
3
0.03
3
0.032
3
0.04
3
0.04
3
0.05
3
0.05
3
0.053
4
0
4
0.01
4
0.016
4
0.02
4
0.02
4
0.03
4
0.04
4
0.04
4
0.05
4
0.058
4
0.058
4
0.06
4
0.06
4
0.06
4
0.06
4
0.07
4
0.08
4
0.1
4
0.121
What If?
Change the rule: assume that a 2 or above on the FST will
tag a participant as drunk.
Changes that occur: FPR will surely go up and FNR will
surely go down.
So the point is: how well a screening test performs is
directly related to the cutoff it uses to identify a “positive.”
One-Sentence Reflection
Simple fractions are used to compute
sensitivity and specificity in many
situations where both test results and
the truth are arrayed in 2x2 tables.