PS3012: Advanced Research Methods
Lecture 9:
Psychophysics,
psychophysical methods,
and signal detection theory
Jonas Larsson
Department of Psychology
RHUL
Term 2: Lecture 9
PS3012: Advanced Research Methods
1 / 43
Today’s lecture
•
•
•
•
Introduction to psychophysics
Thresholds and psychometric functions
Psychophysical methods
Signal detection theory
Term 2: Lecture 9
PS3012: Advanced Research Methods
2 / 43
What is psychophysics?
• The study of the relationship between physical stimuli and
their subjective correlates, or percepts [Wikipedia]
• The scientific study of the relation between stimulus and
sensation [Gescheider, 1976]
• Central idea: measurements of behavioural parameters
(accuracy, reaction time, sensory thresholds) can be used
to infer mental state (percept) of subjects
Term 2: Lecture 9
PS3012: Advanced Research Methods
3 / 43
What can psychophysics be used for?
• Sensory system neurophysiology/neuropsychology
– Sensory limits of vision, hearing, touch…
– Interspecies comparison (e.g., monkeys vs humans)
– Inferring neuronal mechanisms (e.g. illusions, after-effects)
• Experimental psychology
– Visuomotor interactions
– Perception of speed, motion
– Attention
• Quantitative measurement of perceptual states
– Diagnostic tool (e.g., vision tests)
– Assessment tool (e.g., therapeutic effectiveness)
Term 2: Lecture 9
PS3012: Advanced Research Methods
4 / 43
Example: treatment of anorexia
• Distorted self-image in anorexia: subjects perceive
themselves as disproportionally overweight
• Suppose you want to test effectiveness of therapy to
improve self-image (reduce distortion). How can its
effectiveness be quantified?
• Use psychophysical methods to identify “ideal body
proportions” (using manipulated photos of subjects with
different shape/weight) as a threshold: perceptual boundary
between too fat / too thin)
• Measure ideal proportions before & after therapy
• Test difference (if any) statistically for effectiveness of
therapy
Term 2: Lecture 9
PS3012: Advanced Research Methods
5 / 43
Example: treatment of anorexia
•
•
•
Show photos of
subjects manipulated
(Photoshop) to show
different body size (BMI)
Subjects have to rate
photos as “too thin” or
“too fat”; measure %
judged “too fat”
Fit psychometric
function to data
– Note shape (logistic)
Perceptual boundary
(threshold): BMI where
50% of photos judged
“too fat”
Term 2: Lecture 9
100
Perceived shape
(% images judged too fat)
•
Before therapy
After therapy
50 %
threshold
50
0
Body mass index (BMI)
PS3012: Advanced Research Methods
6 / 43
The power of psychophysics
• Quantitative - objective scale of measurement
• Does not suffer from subjectivity of introspection
• Can be used to study “pure” mental phenomena - e.g.
attention
• Valid inter-subject, inter-species, and inter-method
comparisons
– E.g. colour perception in humans and bees
– Sensitivity of neurons vs sensitivity of brains (humans)
• Can be used to study subliminal percepts (e.g. abovechance recognition without awareness)
• Can identify (possibly subconscious) response bias
Term 2: Lecture 9
PS3012: Advanced Research Methods
7 / 43
The concept of thresholds
• Detection threshold (classical definition): smallest
detectable stimulus intensity (energy) (that yields a sensory
percept)
– Threshold for sight (weakest detectable light): about 10 photons!
– Threshold for sound (weakest detectable air vibration): about the
diameter of an atom!
• Discrimination threshold : smallest detectable difference
between two stimuli (that yields a perceptual difference)
– Smallest detectable difference in orientation of two lines
– Smallest difference in colour corresponding to a colour category
change
• Thresholds correspond to a perceptual boundary
• Thresholds can be measured quantitatively
Term 2: Lecture 9
PS3012: Advanced Research Methods
8 / 43
Thresholds & psychometric functions
•
•
•
Psychometric function:
plot of proportion of
stimuli detected or
discriminated vs
stimulus intensity
Ideal psychometric
function: always 100%
above threshold,
always 0% below
threshold - a step
function
Why is the real
psychometric function
not a step function?
Because of NOISE
100
Proportion stimuli detected (%)
•
Ideal psychometric function:
Step function (fixed threshold)
Real psychometric
function:
S-shaped (sigmoid or
logistic) function
50
50%
threshold
0
Stimulus intensity
Term 2: Lecture 9
PS3012: Advanced Research Methods
9 / 43
Psychophysical methods
•
•
•
•
Method of limits
Method of adjustment
Method of constant stimuli
Adaptive methods
– Staircases
– Adaptive versions of constant stimuli
Term 2: Lecture 9
PS3012: Advanced Research Methods
10 / 43
Method of limits
• Stimulus intensity (for discrimination tasks, the difference
between two stimuli) is changed from trial to trial by a fixed
amount either upwards from very weak intensity (ascending
series) or downwards (descending series)
• Subjects report the stimulus intensity when they can no
longer detect or discriminate the stimuli (descending
series) or when they begin to be able to detect/discriminate
stimuli (ascending series)
• These stimulus intensities are averaged to give a threshold
estimate
• Ascending & descending series are done in alternation
Term 2: Lecture 9
PS3012: Advanced Research Methods
11 / 43
Method of limits
Stimulus intensity
Stimulus no
longer detected
descending series
Term 2: Lecture 9
Stimulus
detected
Threshold:
average
stimulus
intensity
ascending series
PS3012: Advanced Research Methods
12 / 43
Method of adjustment
• Subjects adjust stimulus intensity (or difference between
two stimuli) until they can just about detect or discriminate
the stimulus
• This stimulus intensity (or difference) is the threshold
• Usually done in ascending and descending series like
method of limits (but under subjects’ control)
Term 2: Lecture 9
PS3012: Advanced Research Methods
13 / 43
Method of constant stimuli
• Stimuli with a fixed range of intensity levels (or fixed range
of differences for discrimination tasks) are presented in
random order
• Subjects report stimulus absent/present (or for
discrimination tasks, same/different or weaker/stronger
than reference stimulus)
• Subjects’ reports are plotted against stimulus intensity /
difference magnitude to give a psychometric function
• Usually a psychometric function is then fit (by nonlinear
function fitting or logistic regression) to psychometric data
• Threshold is midway between chance level performance
(bottom of psychometric function, e.g. 50% for a 2AFC task)
and 100% detection / discrimination
Term 2: Lecture 9
PS3012: Advanced Research Methods
14 / 43
Method of constant stimuli
Stimulus intensity
Stimulus
detected
Stimulus not
detected
Term 2: Lecture 9
PS3012: Advanced Research Methods
15 / 43
•
•
•
•
For each level of stimulus
intensity, calculate and
plot proportion of stimuli
detected/discriminated
Fit psychometric (sigmoid)
function to data
Threshold is stimulus
intensity at inflection point
(middle of curve)
Corresponds to halfway
between 100%
performance and chance
level performance
(guessing)
Term 2: Lecture 9
Proportion stimuli detected (%)
Method of constant stimuli
100
50
50%
threshold
0
Stimulus intensity
PS3012: Advanced Research Methods
16 / 43
Adaptive methods: staircases
• Similar to method of limits, but series reverse direction
whenever decision changes (e.g. for a descending series,
when subject can no longer detect stimulus, series ascends
instead)
• More effective at “homing in” on threshold
• Threshold is average of reversal stimulus intensity
• More complex reversal rules are often used (“1-up, 2down”) with different methods for computing thresholds
• To avoid subject prediction, often uses several interleaved
staircases (series) randomly interspersed
Term 2: Lecture 9
PS3012: Advanced Research Methods
17 / 43
Adaptive methods:
adjusting constant stimuli
• Similar to constant stimuli, but range of stimulus intensity
levels to use are changed over course of experiment (not
fixed)
• Allows more time to be spent measuring responses near
threshold (like staircases)
• Unlike staircase methods, good for fitting psychometric
functions (samples responses over entire psychometric
function curve)
• Various methods exist (Best PEST, QUEST etc)
Term 2: Lecture 9
PS3012: Advanced Research Methods
18 / 43
The effect of noise on
psychometric functions
Detection or
discrimination of stimulus
is always subject to noise:
–
–
–
–
•
•
•
Neural
Stimulus (physical)
Attention
(Response)
On any trial, noise will
randomly increase or
decrease perceived signal
intensity
Subject perceives signal+
noise (cannot tell the
difference)
Changes step function to
sigmoid (logistic) function
Term 2: Lecture 9
100
Proportion stimuli detected (%)
•
Above threshold:
random noise will
weaken signal for
some trials, making
detection <100%
50
Below threshold:
random noise will
strengthen signal for
some trials, making
detection > 0%
0
Stimulus intensity
PS3012: Advanced Research Methods
19 / 43
Detecting stimuli in noise:
Signal Detection Theory (SDT)
• How stimuli are detected/discriminated against background
noise
• How to make decisions in the presence of uncertainty
• How to make optimal decisions from ambiguous data
• How to make good decisions from bad information
• SDT explains why shape of psychometric function varies
with noise
• SDT explains how a subject’s criterion (response bias)
affects decisions and how to measure it
• SDT allows measurement of sensitivity (ability to make
correct responses/decisions) regardless of criterion/bias
Term 2: Lecture 9
PS3012: Advanced Research Methods
20 / 43
Origin of SDT: WW2 radar operator
• Task: warn of incoming aircraft
• Are the blobs enemy aircraft?
Or just noise (e.g. clouds)?
• Decision depends on
subjective criterion: how big
must the blobs be to be aircraft
• Decision has consequences:
– If you miss an aircraft, people
might get killed
– If you mistake noise for
aircraft, fuel, manpower &
resources are wasted
Term 2: Lecture 9
PS3012: Advanced Research Methods
Radar screen
21 / 43
Decision outcomes & consequences
SIGNAL: are the blobs real enemy aircraft?
yes
DECISION:
should you alert
the air force?
Term 2: Lecture 9
no
yes
no
Hit
False
alarm
Miss
Correct
reject
PS3012: Advanced Research Methods
22 / 43
Decision depends on criterion
• Low criterion: alert for every blob: make sure you never
miss - but many false alarms
• High criterion: only alert for really big blobs: no false
alarms - but many misses
• Which criterion is “best” (optimal)?
• Depends on the costs of making errors...
• which errors are acceptable...
• but also on how good your information is (uncertainty)
Term 2: Lecture 9
PS3012: Advanced Research Methods
23 / 43
Example 2: mugger or friend?
• You’re walking alone on an empty street
• Somebody behind you calls out to you: “hey!”
• You don’t recognize the voice, and can’t see the person’s
face clearly
• Is it a friend or a mugger? (how familiar is the person’s
appearance?)
• Do you run or stay?
Term 2: Lecture 9
PS3012: Advanced Research Methods
24 / 43
Decision outcomes & consequences
SIGNAL: is the person a friend or a mugger?
DECISION:
should you
run or stay?
Term 2: Lecture 9
mugger
friend
run
Lucky
escape!
Friend
gets upset
stay
You got
mugged!
Head to
the pub
PS3012: Advanced Research Methods
25 / 43
Decision outcomes & consequences
SIGNAL: is the person a mugger or friend?
run
DECISION:
should you
run or stay?
Term 2: Lecture 9
stay
mugger
friend
Hit
False
alarm
Miss
Correct
reject
PS3012: Advanced Research Methods
26 / 43
Decision criterion depends on
penalties and uncertainty
• How would your decision to run or stay change if:
– it’s in the middle of the night on campus? (high uncertainty,
high penalty for false alarms)
– it’s the middle of the day on campus? (low uncertainty, high
penalty for false alarms)
– it’s in the middle of the night in the South Bronx? (high
uncertainty, high penalty for misses)
– it’s in the middle of the day in the South Bronx? (low
uncertainty, high penalty for misses)
• So how do you decide which decision criterion is best
(optimal)?
Term 2: Lecture 9
PS3012: Advanced Research Methods
27 / 43
Use Signal Detection Theory
criterion
run (mugger)
probability
stay (friend)
friend
mugger
unfamiliar appearance (stimulus intensity)
Term 2: Lecture 9
PS3012: Advanced Research Methods
28 / 43
SDT & effect of criterion:
radar operator example
criterion
yes (aircraft)
probability
no (noise)
noise
aircraft
Blob size (stimulus intensity)
Term 2: Lecture 9
PS3012: Advanced Research Methods
29 / 43
SDT & effect of criterion:
radar operator example
criterion
probability
no (noise)
noise
yes (aircraft)
correct
rejects
hits
aircraft
misses false alarms
Term 2: Lecture 9
PS3012: Advanced Research Methods
30 / 43
Low criterion:
few misses, many false alarms
criterion
probability
no (noise)
noise
yes (aircraft)
correct
rejects
hits
aircraft
false alarms
Term 2: Lecture 9
PS3012: Advanced Research Methods
31 / 43
High criterion:
many misses, few false alarms
criterion
probability
no (noise)
noise
correct
rejects
yes (aircraft)
hits
aircraft
misses
Term 2: Lecture 9
PS3012: Advanced Research Methods
32 / 43
Low noise: high discriminability &
sensitivity (few misses & false alarms)
discriminability d’
(distance between means)
probability
Small overlap
between
distributions of
noise and
stimulus+noise
(aircraft)
noise
aircraft
Blob size (stimulus intensity)
Term 2: Lecture 9
PS3012: Advanced Research Methods
33 / 43
High noise: low discriminability &
sensitivity (many misses & false
alarms)
discriminability d’
probability
Large overlap
between
distributions of
noise and
stimulus+noise
(aircraft)
noise
aircraft
Blob size (stimulus intensity)
Term 2: Lecture 9
PS3012: Advanced Research Methods
34 / 43
SDT & psychophysics
Decision criterion
probability
Response: No
Response: Yes
d’
Discriminability
(sensitivity): dprime (d’) - the
distance between
the means of (N)
and (SN) in units of
S.D.
Stimulus+Noise
(SN)
Noise (N)
Stimulus intensity
Term 2: Lecture 9
PS3012: Advanced Research Methods
35 / 43
Discriminability (d’) is independent of
criterion
Decision criterion
Response: No
Response: Yes
probability
d’
d’ depends only on
the distance
between the means
of (N) and (SN)
Stimulus+Noise
(SN)
Noise (N)
Stimulus intensity
Term 2: Lecture 9
PS3012: Advanced Research Methods
36 / 43
Discriminability (d’) is independent of
criterion
Decision criterion
Response: No
Response: Yes
probability
d’
d’ depends only on
the distance
between the means
of (N) and (SN)
Stimulus+Noise
(SN)
Noise (N)
Stimulus intensity
Term 2: Lecture 9
PS3012: Advanced Research Methods
37 / 43
Estimation of d’
•
•
•
d’ is the difference between the means of the noise (N) and
stimulus+noise (SN) distributions, in units of standard deviations
of the noise (N) distribution:
d’ = [mSN - mN] / sN
But these distributions are not usually known!
d’ is more easily computed from the hit rate (proportion of stimuli
reported when present, [yes|SN] ) and the false alarm rate
(proportion of stimuli reported when not present, [yes|N] ):
– Convert hit & false alarm rates (which are probabilities) to z scores
from tables of z distribution:
• Hit rate = P(yes|SN) => z( yes|SN )
• False alarm rate = P( yes|N ) => z( yes|N )
•
d’ = z( yes|SN ) - z( yes|N )
Decision criterion must be fixed!
Term 2: Lecture 9
PS3012: Advanced Research Methods
38 / 43
Interpreting d’
• Low d’ means stimulus (signal) + noise (SN) distribution is
highly overlapping with noise (N) distribution
– d’ = 0: chance level performance (N and SN overlap exactly)
• High d’ means SN and N distributions are far apart
– d’ = 1: moderate performance
– d’ = 4.65: “optimal” (corresponds to hit rate=0.99, false alarm
rate=0.01)
Term 2: Lecture 9
PS3012: Advanced Research Methods
39 / 43
Example
• Performance on visual detection task before drinking
alcohol:
– Hit rate 0.7, false alarm rate 0.2
• Performance of task after drinking alcohol:
– Hit rate 0.8, false alarm rate 0.3
• Did performance or sensitivity (discriminability) improve?
• Before drinking alcohol:
d’ = z(hit rate) - z(false alarm rate) = 0.542 - (-0.842) = 1.366
• After drinking alcohol:
d’ = z(hit rate) - z(false alarm rate) = 0.842 - (-0.542) = 1.366
• Alcohol did not improve performance (d’)
• Alcohol did change criterion (by lowering it)
Term 2: Lecture 9
PS3012: Advanced Research Methods
40 / 43
Controlling decision criterion
•
•
•
•
•
Criterion influenced by stimulus probability and decision
consequences (payoffs - rewards & penalties)
Need to know chance level performance (performance when no
stimulus present)
Present noise stimuli on some constant proportion of trials - this
proportion is then equal to chance level performance
Use fixed payoff (e.g. reward for hits, penalties for false alarms)
Best: use forced choice methods:
– Most common: use two-alternative forced choice (2AFC); present
two stimuli on each trial (one with stimulus, one with just noise) and
force subject to decide which one contained the stimulus - chance
level performance is then 50%
– Performance often above chance even when subject is guessing
Term 2: Lecture 9
PS3012: Advanced Research Methods
41 / 43
Summary of SDT
•
•
•
•
•
•
•
•
Decisions (perceptual judgments) are always made in the
presence of noise (internal/neural and external/physical)
Decisions are made with respect to a criterion (response bias)
Criterion is variable & reflects probability of stimulus and payoffs/
consequences of decision
Performance (hit rate) is a biased measure - depends on criterion
There is a trade-off between hit rate and false alarm rate
Sensitivity/discriminability - the ability to discriminate a stimulus
from noise - is independent of the criterion
d’ is a measure of discriminability that is insensitive to criterion
d’ can be computed from the hit rate (proportion of stimuli
detected when present) and the false alarm rate (proportion of
stimuli reported when not present)
Term 2: Lecture 9
PS3012: Advanced Research Methods
42 / 43
And finally…
• Reading: see course web page
– Ehrenstein & Ehrenstein: Psychophysical Methods
• Next week: last lecture (DW)
Term 2: Lecture 9
PS3012: Advanced Research Methods
43 / 43