ISEN 330 (Homework Assignment 1)
Due date: September 11th, at 11:59PM
Submitted via Canvas
Signal Detection Theory Applications:
1.
Wolfpack Winery wants to evaluate the performance of a wine taster whose task is to detect if a wine
labeled as made from “Pinot Noir" has been tempered by the addition of some Gamay (generally
considered an inferior grape). Here, the SIGNAL corresponds to presence of Gamay and the NOISE
condition corresponds to pure Pinot. The wine taster was blindfolded and tasted forty (40) glasses of
Pinot, half of them tempered with some Gamay and half without. Below are the performance data sets
of the taster in trying to identify Gamay in a Pinot Noir wine. A “NO” response indicates the
perceived absence of Gamay, whereas a “YES” response indicates perception of Gamay in the wine
(tainted product).
Data Set 1
TASTER’S RESPONSE
Yes (Gammy)
No (Pure Pinot)
Total (T)
Reality
Tempered by Gammy
(Signal)
4 (H)
16 (M)
20
Pure Pinot
(Noise)
1 (FA)
19 (CR)
20
Data Set 2
TASTER’S RESPONSE
Yes (Gammy)
No (Pure Pinot)
Total (T)
Reality
Tempered by Gammy
14
6
20
Data Set 3
TASTER’S RESPONSE
Yes (Gammy)
No (Pure Pinot)
Total (T)
Reality
Tempered by Gammy
19
1
20
0.20
0.80
0.05
0.95
Pure Pinot
6
14
20
0.70
0.30
0.30
0.70
Pure Pinot
16
4
20
0.95
0.05
0.80
0.20
(a) (3 pts.) Calculate the probabilities of the signal (Gamay) detection outcomes for each data set
(As Shown Above)
(b) (3 pts.) Based on the probabilities in each data set, classify taster responding (Data Set 1, 2, and
3) as “risky” or “conservative” or “unbiased”.
Data Set 1: Very conservative (P(FA)=0.05 low; P(H)=0.20 low)
Data Set 2: unbiased (P(FA)=0.30; P(H)=0.70)
Data Set 3: Very risky (P(FA)=0.80 high; P(H)=0.95 high)
(c) (2 pts.) Plot an ROC curve using data sets 1, 2, and 3. Next, consider the additional signal
detection data set 4. Add the corresponding data point for data set 4 to the ROC curve, and
calculate the probabilities of the signal detection outcomes for data set 4.
Data Set 4
TASTER’S RESPONSE
Yes (Gammy)
No (Pure Pinot)
Total (T)
Reality
Tempered by Gammy
18
2
20
Pure Pinot
2
18
20
0.90
0.10
0.10
0.90
(d) (1 pt.) If I told you the same person generated data sets 1-3 and 4, what could you tell me about
their functional state during the signal detection task, when data set 4 was recorded?
More alert and attentive (improved functional state) with high sensitivity and a neutral criterion
during Set 4.
2.
Inspector A had P(H) = 0.82 and P(FA) = 0.24. Inspector B had P(H) = 0.82 and P(FA) = 0.40.
(a) (4 pts.) Compute sensitivity (d’) and response criterion (c) for inspector A and B, respectively.
(b) (1 pt.) Who is more sensitive, and who is riskier?
Inspector A is more sensitive; Inspector B is riskier.
3.
A shooting range worker is suing his employer over a work-related injury because he claimed the
loud noises is making him practically deaf. The insurance company for the shooting range wanted to
check out the damage of his hearing. So, a “yes-no” hearing test was devised. Twenty trials
contained an audio signal that is readily perceivable to someone with normal hearing (but not loud),
and 20 contain no signal. The worker said that he heard an audio signal 12 times in total but was
only correct twice.
(a) (1 pt.) Create a decision/response matrix that reflects the worker’s response.
Reality
Worker’s Response
Say “Signal”
Say “No Signal”
Total (T)
Signal
2
18
20
No Signal
10
10
20
(b) (1 pt.) Calculate the probabilities of the signal detection outcomes.
0.10
0.90
0.50
0.50
(c) (2 pts.) Compute sensitivity (d’) and decision criterion (c).
(As Shown in work below)
(d) (1 pt.) Based on d’ give your conclusion as to whether the worker really is deaf.
d′≈−1.28 meaning his “signal” / “no signal” responses are opposite of the evidence. That
pattern is not consistent with genuine deafness and suggests a misunderstanding of the task or
non-credible responding rather than true insensitivity.
(e) (1 pt.) Indicate whether the probability measures (i.e., P(H) and P(FA)) support your earlier
conclusion based on d’.
Saying “signal” half the time when there is no signal but only detecting 10% when there is a
signal supports the conclusion that his responses aren’t guided by actual hearing of the tone.
~
~
~
~
~
~Ji,•.--
) (fU~))
'3
L ~ ( V (H))- z. ( ?(f A)) 1 L=--\[4 ~(t-() -n
~
~
--it\' (1-1) ~o. iv ) :). 1- Lo. 1b )-= - t, 111 b
-s
-s
,..,~
s
d )~ l - J. 1,~ I 11 )- O -= -, .1-~j h --.: - I. l.,~
~
,
G
~ -~ [ - I , 1-b I H d J "'= 0 · &Ifoi --:- 0, b ~
- ---•- 4!»_
'
~-=~~-=-.-=-=--=--/~-~_c"c'i_ ~)- c A~~\ -t ;~, ~ w"
•
·-
L . ··_
1}-t.flf,t,._I_ _ _ _ _ _ _ _
--
2,.
Y('"
.i~_1"
i l-
·-
-
(.,--:; ~l [
-
~
"1- (
P(H)) -r::i (p (ii½
nd ~ -~ l l,1,rn, -l.zn~) c()
---
~