Decision making as a model
4. Signal detection: models and
measures
Nice theorems, but how to proceed in practice?
1. Hard work
Get several points on ROC-curve by inducing several
criteria (pay-off, signal frequency)
Compute hit rate and false alarm rate at every
criterion
Many trials for every point!
Measure or compute A using graphical methods
Certainly
no signal
0
1
2
3
4
5 Certainly
a signal
Variant: numeric (un)certainty scale: implies multiple criteria
– consumes many trials too
2. Rough approximation
Area measure for one point: A'
Average of those two
areas:
Hits
1-H
F
A' = 1 - ¼ ------- + ----1-F
H
(H-F)(1+H-F)
= ½ + ¼ ----------------H(1-F)
H
F
False Alarms
Iff H>F
B''= -.4
if H = 1, F≠0, F≠1, then B'' = -1
B''= .07
F
B''=.4
HIT RATE
B''= -.07
H
Comparable measure for
criterion/bias: Grier’s B''
Isobias curves
FALSE ALARM RATE
if H = 1 - F then B'' = 0
if F = 0, H≠ 0, H≠1 then B'' = 1
H(1 - H) – F(1 – F)
B'' = sign(H - F) -----------------------H(1 - H) + F(1 – F)
3. Introducing assumptions
Even when several points ara available, they may not lie on a nice
curve
Then you might fit a curve,
but which one?
Every curve reflects some
(implicit) assumptions about
distributions
Save labor:
more assumptions  less measurement
(but the assumptions may not be justified)
Normal distributions are popular
(there are other models!)
0,45
0,4
0,35
0,3
0,25
0,2
0,15
0,1
0,05
0
-4
-3
-2
-1
0
1
2
3
4
Simplest model:
noise and signal distributions normal with equal variance
One point (PH, PFA pair) is sufficient
5
Example: in an experiment with noise trials and signal trials
these results were obtained:
Hit rate: .933, False Alarm rate .309
(.067 misses and .691 correct rejections)
Normal distributions: via corresponding z-scores the
complete model can be reconstructed:
distance: d´ = 2
measure for “sensitivity”
z.933 = - 1.5
z.309 = .5
f
h
.933
.309
h
β = ---- = .37
f
Measure for
bias/criterion
-3
-2
-1
0
1
2
3
4
5
6
1
0,9
0,8
d'=0
0,7
proportion hits
-4
several d'-s and
corresponding
ROC-curves
d'=0.5
0,6
d'=1
0,5
d'=1.5
d'=2
0,4
d=2.5
0,3
d'=3
0,2
0,1
0
0
0,2
0,4
0,6
proportion false alarm s
0,8
1
Gaussian models: preliminary
Standaard normal curve
M=0, sd = 1
P
z
z
Φ(z) = -∞ φdx
φ(z)=
1 -z2/2
e
√2π
Transformations:
Φ(z)  P
Φ-1(P) or: Z(P)  z
see tabels and
standard software
Roc-curve
PH = f(PFA)
PH
PFA
λ
Z-transformation
ROC-curve
Pz
zH
-
zH = f(zFA)
Nice way to plot several (PFA, PH) points
zFA
Plotting with regression
line?
Regression line
zH = a1zFA+ b1,
Minimize (squared)
deviations zH :
Underestimation of a
Regresionline
zFA= a2zH + b2
Minimize (squared)
deviations zFA :
Overestimation of a
Compromise: average of regression
lines
ZH = ½(a1+1/a2)ZFA + ½(b1+b2/a2)
Equal variance model:
PFA = 1- Φ(λ),
= Φ(-λ),
PH = 1 – Φ(-(d' - λ)) = Φ(d' – λ),
zFA = -λ
zH = d' – λ
zH = zFA + d'
d' = zH –zFA
z-plot ROC 45°
0 λ
zH
d'
45°
d'
zFA
Criterion/bias:
β = h/f = φ(zH)/φ(zF)
f
h
1
-z2/2
φ(z) = ------ e
(standard-normal)
√(2π)
1
-zH2/2
zFA2 – zH2
φ(zH) = ------ e
-----------2
√
(2π)
Divide: -------------------- = e
1
-zFA2/2
φ(zFA) = ------ e
√(2π)
To get symmety a log transformation is often applied:
log β = log h – log f = ½(z2FA – z2H )
f
h
λ c
Alternatve: c (aka λcenter),
distance (in sd) between middle (were h=f) and criterion
c = -(d'/2 – λ)
zFA = -λ
d' = zH - zFA
zH – zFA 2zFA
c = - ---------+ ----2
2
zH + zFA
c = - ---------2
β
Isobias curves for β en c
c