1
FITTING
PSYCHOMETRIC
FUNCTIONS
Florian Raudies
11/17/2011
Boston University
2
Overview
Definitions
Parameters
Fitting
Example: Visual Motion
Goodness of Fit
Conclusion
3
Definitions
Labels for the axes of a psychometric function
Proportion correct
Examples
Experiment
Design
Proportion
Correct
2AFC
50…100%
3AFC
33…100%
2IAFC
50…100%
2AFC
Two alternative forced choice
Stimulus level
4
Definitions
Proportion correct
Special points of an psychometric function
Psychometric function
75%
PSE
Point of subjective equivalence
50%
JND
Just noticeable difference
25%
PSE
2JND
Stimulus level
5
Definitions
Weibull function
𝑓 𝑊𝑒𝑖𝑏𝑢𝑙𝑙
𝑥
𝑥|𝛼, 𝛽 = 1 − exp(−
𝛼
𝛽
)
Cumulative normal distribution function
𝑓 𝑐𝑛𝑑𝑓
𝑥|μ, 𝜎 = erf(𝑥; 𝜇, 𝜎)
Logit function
𝑓𝑙𝑜𝑔𝑖𝑡 𝑥|μ, 𝜃 =
1
1 + exp −
erf 𝑥; 𝜇, 𝜎 =
𝑥−𝜇
𝜃
1
2𝜋𝜎
𝑥
−∞
𝑠−𝜇 2
exp(−
) 𝑑𝑠
2𝜎
6
Definitions in Matlab
function Y = weibullFunction(X, alpha, beta)
% weibullFunction
%
X
- Input values.
%
alpha
- Parameter for scale.
%
beta
- Parameter for shape.
%
% RETURN
%
Y
- Return values.
%
% DESCRIPTION
%
See http://en.wikipedia.org/wiki/Weibull_distribution.
Y = 1 - exp(-(X/alpha).^beta);
7
Definitions in Matlab
function Y = cndFunction(X, mu,sigma)
% cndFunction - Cumulative normal distribution function
% Shift by one up and rescale because the integral for erf
% ranges from 0 to value whereas the distribution uses the
% boundaries -inf to value.
Y = (1+erf( (X-mu)/(sqrt(2)*sigma) ))/2;
function Y = logitFunction(X, mu,theta)
% logitFunction…
Y = 1./( 1 + exp( -(X-mu)/theta ) );
8
Parameters
Psychometric Functions
1
proportion correct
0.8
0.6
0.4
Weibull,  =5, =7
cndf, =7, =1
logit, =2, =0.5
0.2
0
0
2
4
6
stimulus level
8
10
9
Proportion correct
Proportion correct
Proportion correct
Parameters
Weibull
1
 =5, =2
 =5, =7
 =1, =7
0.5
0
0
1
2
3
4
5
6
Stimulus level
Cndf
7
1
0
1
2
3
4
5
6
Stimulus level
Logit
7
1
10
8
9
10
=5,  =1
=5,  =0.25
=1,  =0.25
0.5
0
9
=5,  =1
=5,  =0.25
=1,  =0.25
0.5
0
8
0
1
2
3
4
5
6
Stimulus level
7
8
9
10
10
Parameters
Additional parameters for a psychometric function
𝑓 𝑝𝑠𝑦 𝑥|Θ = 𝛾 + 1 − 𝛾 − 𝜆 ∙ 𝑓 𝑊𝑒𝑖𝑏𝑢𝑙𝑙 (𝑥|𝛼, 𝛽)
with the parameter vector Θ = (𝛼, 𝛽, 𝛾, 𝜆).
𝛼 - scale
𝛽 - shape
𝛾 - guessing rate. For nAFC 𝛾 = 1/𝑛.
𝜆 - miss rate. For a perfect observer 𝜆 = 0.
11
Definition in Matlab
function Y = psycFunctionMissGuess(psycFunction,X,Theta,Const)
% psycFunctionMissGuess
%
psycFunction - Function handle for the psychometric function.
%
X
- Input values.
%
Theta
- Parameter values.
%
Const
- Constants, here guess rate and miss rate.
%
% RETURN
%
Y
- Output values.
Y = Const(1) …
+ (1 - Const(1) - Const(2)) * psycFunction(X,Theta(1),Theta(2));
12
Fitting
Assume: 𝑛 independent measurements (𝑥𝑖 , 𝑟𝑖 ) with
𝑥𝑖 strength of the percept and
𝑟𝑖 𝜖 {0,1} response of participant in a 2AFC task.
Problem: Maximum likelihood estimation for parameters of
the psychometric function
𝑛
𝑃 Θ|𝑥1 , 𝑟1 , … , 𝑥𝑛 , 𝑟𝑛 =
𝑃(Θ|𝑥𝑖 , 𝑟𝑖 )
𝑖=1
with
𝑃 Θ|𝑥𝑖 , 𝑟𝑖 = 𝑐𝑜𝑟𝑟𝑒𝑐𝑡 = 𝑓 𝑝𝑠𝑦 (𝑥𝑖 |Θ)
𝑃 Θ|𝑥𝑖 , 𝑟𝑖 = 𝑓𝑎𝑙𝑠𝑒 = 1 − 𝑓 𝑝𝑠𝑦 (𝑥𝑖 |Θ)
13
Fitting
Small values and log-likelihood
The term 𝑃(… ) can lead to small values below the range of
single or double precision. Thus, for optimization take the
negative and apply the monotonic log-function function:
𝐿 Θ|𝑥1 , 𝑟1 , … , 𝑥𝑛 , 𝑟𝑛
log 𝑓 𝑝𝑠𝑦 𝑥𝑖 |Θ
=−
𝑟𝑖 =1
log(1 − 𝑓 𝑝𝑠𝑦 𝑥𝑖 |Θ )
−
𝑟𝑖 =0
This expression is maximized for the parameters Θ. Often
additional constraints for the parameters are available.
This is a constraint nonlinear optimization problem with also
referred to as nonlinear programming.
14
Fitting in Matlab
function Theta = fittingPsycFunction(X, Y, opt)
% fittingPsycFunction…
ThetaMin
= opt.ThetaMin; % Lower boundary for parameters.
ThetaMax
= opt.ThetaMax; % Upper boundary.
ThetaIni
= opt.ThetaIni; % Initial value for parameters.
Const
= opt.Const;
% Constants in the psychometric function.
psycFunction
= opt.psycFunction; % Function handle for the psychometric function.
% Optimization with the fmincon from the Matlab optimization toolbox.
Theta = fmincon(@(Theta)logLikelihoodPsycFunction(...
psycFunction, Theta, X,Y, Const), ...
ThetaIni, [],[],[],[], ThetaMin,ThetaMax, [], opt);
function L = logLikelihoodPsycFunction(psycFunction, Theta, X,Y, Const)
% logLikelihoodFunction…
Xtrue
= X(Y==1);
Xfalse = X(Y==0);
L
= -sum(log(
psycFunctionMissGuess(...
psycFunction, Xtrue, Theta, Const) + eps))...
-sum(log(1 - psycFunctionMissGuess(...
psycFunction, Xfalse, Theta, Const) + eps));
15
Example: Visual Motion
Objective: Measure the coherence threshold for motiondirection discrimination.
Design: 2AFC task between leftward and rightward
motion for varying motion coherence by a
limited dot lifetime in an random dot
kinematogram (RDK).
Use the method of constant stimuli for 11
coherence values. This requires usually more
samples than adaptive thresholding techniques.
Use 10 trials for each coherence value.
This is a very simplified example!
16
Example: Visual Motion
17
Example: Visual Motion
A single trial
Fixation
&
(Response) for
500ms
Fixation
&
Blank for
100ms
Fixation
&
1st Motion for
400ms
Fixation
&
2nd Motion for
400ms
Time
Overall time 10 x 11 x 1,9sec = 209sec or 3.48min.
A response is not expected before the first trail.
Are the motions equal?
18
Example: Visual Motion
Correct /
response
Motion coherence (%)
10
20
30
40
50
60
70
80
90
100
1
1/1
1/1
1/1
1/1
1/1
1/1
1/1
1
0
0
0
2
0/1
0/1
0/0
1/1
0/0
1/1
1/1
1
1
1
0
3
1/0
0/1
1/1
0/1
1/1
1/0
0/1
0
0
0
1
4
1/1
1/0
0/1
0/0
0/1
0/1
0/0
0
0
1
1
5
0/0
1/1
1/0
1/0
1/0
0/0
1/1
1
1
1
0
6
0/1
0/0
1/0
1/1
1/1
0/0
1/1
0
1
1
0
7
1/0
1/1
0/0
1/0
0/1
1/1
1/1
1
0
0
1
8
0/0
0/1
0/1
0/1
0/0
0/0
0/0
0
0
0
0
9
1/1
0/0
1/0
0/1
1/1
1/0
0/0
1
1
1
1
10
0/1
1/0
0/0
0/0
0/1
0/0
0/0
0
1
0
0
Correct
(%)
50
50
50
50
60
70
90
100
100
100
100
Trial
0
19
Example: Visual Motion
Fitted Weibull function
110
percentage correct
100
90
80
70
60
data
fitted Weibull
50
40
0
20
40
60
80
motion coherence (%)
100
20
Example in Matlab
% Load data files.
DataStimulus = dlmread('./DataMotionCoherenceStimulus00.txt');
DataObserver = dlmread('./DataMotionCoherenceObserver00.txt');
% "Response = 1" encodes correct and "Response = 0" incorrect.
Response = double(DataStimulus(2:end,:)==DataObserver(2:end,:));
trialNum = size(Response,2);
StimulusLevel = DataStimulus(1,:);
% Fit data.
opt.ThetaMin = [ 1.0 0.5]; % alpha, beta to optimize.
opt.ThetaMax = [100.0 10.0];
opt.ThetaIni = [ 5.0 1.0];
opt.Const
= [ 0.5 0.0]; % gamma, lambda are fixed.
opt.psycFunction = @weibullFunction;
StimulusLevelMatrix = repmat(StimulusLevel,[trialNum 1]);
Theta = fittingPsycFunction(StimulusLevelMatrix, Response, opt);
21
Goodness of Fit
Over dispersion or lack of fit
Dependency between trials
Non-stationary psychometric function (e.g. learning)
Under dispersion or fit is too god
Experimenter’s bias in removing outliers
22
Conclusion
Use maximum likelihood to fit your data, while leaving the
lapse rate as parameter being optimized.
This is not the case in the presented code but can be
adapted.
Assess goodness of fit to:
Ensure Parameter estimates and their variability are from a
plausible model to describe the data.
Identify uneven sampling of the stimulus level or outliers by
applying an objective criteria.
23
References
For fitting data
Myung, Journal of Mathematical Psychology 47, 2003
Treutwein & Strasburger, Perception & Psychophysics
61(1), 1999
For goodness of fit
Wichmann & Hill, Perception & Psychophysics 63(8), 2001
For detection theory
Macmillan & Creelman. Detection theory - A user’s guide
Psychology Press (2009)