Choice and Strict Matching
The Go-To Methodology: Conc VI VI
?
VI 30 s
VI 60 s
Concurrent schedules -- schedules of choice
Two ways of arranging concurrent schedules:
A.
Independent scheduling
B.
Dependent scheduling
and, for each, there are two ways in which the schedules
can be arranged:
1.
the two-key (or N-key) procedure
2.
the switching-key or Findlay or changeover
procedure
Independent vs. dependent scheduling
With independent scheduling, each time-based schedule
runs on its own. When a reinforcer is arranged on one
schedule, it stops, but the other continues timing and may
itself set up a reinforcer before the first one is taken.
With dependent scheduling, when one schedule sets up a
reinforcer, the other stops timing. The first reinforcer has
to be collected before the second schedule will continue
timing.
Independent scheduling
When you use independent scheduling, the subject
could get all of its reinforcers from just one alternative
and never change over.
But, if it did this, it would not get the maximum reinforcer
rate available.
When you arrange dependent scheduling, if the subject
responds only on one of the alternatives, the as soon as
a reinforcer sets up on the other alternative, no more
reinforcers at all will be obtained.
Dependent scheduling is good because if you set up (for
example) 4 times more reinforcers for one alternative than
for the other, the subject will get 4 times more on the first
alternative (within sampling error).
You have a closely controlled independent variable.
With independent scheduling, this may not happen so
precisely, and may not happen at all.
The other two procedures that can be used:
The two-key (or N-key) procedure
The two (or more) schedules are arranged at the same
time on two (or more) operanda (levers, keys).
The schedules run at the same time, and the subject is
free to move between the operanda and schedules at any
time.
VI x s
VI y s
Switching-key procedure:
Single peck - switching key -> changes the color and
schedule of the main key on which reinforcers can be
obtained.
Note that both the schedules still run all the time until
each one arranges a reinforcer -- even if the alternative is
not displayed.
VI x
Switch
Measures of choice in concurrent schedules
1. Response allocation: The number of responses
emitted on each alternative
Used to be measured as relative responses (the number
of responses on one alternative divided by the total
responses
B1
B1  B 2
Proportional measure: ranges from 0 (all
responses to B2) to 1 (all responses on B1)
Measures of choice in concurrent schedules
1. Response allocation: The number of responses
emitted on each alternative
or, more recently, as response ratio (the number of
responses on one key divided by the number of
responses on the other key):
B1
B2
This measure can range from 0 to infinity
(and is not homoscedastic – it does not have
constant variance) so…
Measures of choice in concurrent schedules
1. Response allocation: The number of responses
emitted on each alternative
Often used:
 B1 
log 
 B2 
This measure can range from minus infinity to infinity
(and probably IS homoscedastic – it does have constant
variance as we change things that affect choice)
Measures of choice in concurrent schedules - continued
2. Time allocation: The amount of time spent on each
alternative.
This is measured from changeover response to
changeover response in the switching-key procedure,
or from the first response on one alternative to the first
response on the other alternative in the two-key
procedure.
As above, the measures used may be relative time, or the
time-allocation ratio.
T1
T1  T 2
T1
T2
 T1 
log 
T2 
Relative time allocation
Time-allocation ratio
Log time-allocation ratio *
But are Time Measures that Simple?
• Traditionally measured time
– Is this missing something?
• Elements of time
– Pecking
– Switching
– Rf and post-rf
Switch vs. Peck Time in 2 Alt Choice
TIME PROPORTIONAL TO TOTAL TIME
1.0
Switch Proportion Time
Post Rf Proportion Time
Peck Proportion Time
0.8
0.6
0.4
0.2
0.0
-1.5
-1.0
-0.5
0.0
0.5
LOG REINFORCER RATIO
1.0
1.5
Response and time measures are
usually very similar…
• Independent of measures of choice
– relative response-allocation and timeallocation ratios
• Or procedure
– Two-key or switching-key procedure
– Independent versus dependent scheduling
The changeover delay (COD)
• COD: short period of time that must elapse
between changing between alternatives and
gaining a reinforcer that is already arranged.
• Changeover delays eliminate concurrent
superstitions -- which happens when a
reinforcer for one response also reinforces
responding on the other alternative.
– Remember superstition lecture
Reinforcer arranged on left
1 peck
>2s
1 peck
REINFORCER
Thinking about Choice Behavior
• The problem of choice
– Organisms constantly bombarded with
alternatives
• What controls allocation of behavior?
• How will behavior
allocate?
– When t’s are equal?
– Slightly different?
– Greatly different?
The initial empirical finding -- Herrnstein (1961) VI VI
1.0
HERRNSTEIN (1961)
BIRD 55
RELATIVE RESPONSES
0.8
BIRD 231
BIRD 641
0.6
0.4
0.2
STRICT MATCHING
0.0
0.0
0.2
0.4
0.6
RELATIVE REINFORCERS
0.8
1.0
Strict matching:
Relative responses equals
relative reinforcers obtained
Simple quantitative relation:
clearly something fundamental
and important is going on…
1.0
HERRNSTEIN (1961)
BIRD 55
RELATIVE RESPONSES
0.8
BIRD 231
BIRD 641
0.6
0.4
0.2
STRICT MATCHING
0.0
0.0
0.2
0.4
0.6
RELATIVE REINFORCERS
0.8
1.0
Why Study Choice?
•
•
1. The empirical relation -- Herrnstein (1961)
2. Because all behavior is fundamentally
choice behavior (Herrnstein, 1971)
–
There is always a choice
•
•
–
–
Other responses that can be emitted
Other reinforcers that can be obtained -- even when you
are reinforcing just one response (e.g., in a single VI
schedule).
An animal or a human can always do something
else.
Understand choice behavior -> understand all
behavior
So where do we start?: the strict matching law.
Approximately, the proportion of responses emitted to
one of the concurrent VI VI schedules equals the
proportion of reinforcers obtained at that alternative
1.0
HERRNSTEIN (1961)
BIRD 55
RELATIVE RESPONSES
0.8
BIRD 231
BIRD 641
0.6
0.4
0.2
STRICT MATCHING
0.0
0.0
0.2
0.4
0.6
RELATIVE REINFORCERS
0.8
1.0
The graph shows that, approximately, the proportion of
responses emitted to one of the concurrent VI VI
schedules equals the proportion of reinforcers obtained
at that alternative
This is the strict matching law, and its formula is:
B1
R1

B1  B 2 R1  R 2
B = responses, R = reinforcers, and the Subscripts 1 and
2 denote the alternatives.
Another way of writing the strict matching law:
B1
R1

B1  B 2 R1  R 2
Cross multiply:
B1( R1  R 2)  R1( B1  B 2)
B1R1  B1R 2  B1R1  B 2 R1
Subtract B1R1 from each side:
B1 R 2  B 2 R1
so:
B1 R1

B2 R2
B1
R1

B1  B 2 R1  R 2
is called the relative version of the strict matching law
B1 R1

B2 R2
is called the ratio version of the strict matching law
They both say exactly the same thing.
Play around a bit more and you can get:
B1 B 2

R1 R 2
What does this say?
Research on strict matching
Initially great empirical support. For example,
Conger & Killeen (1974) - conversation
Schroeder & Holland (1969) (over)
Rats, pigeons, horses, cows, mice, cockroaches, ravens
All sorts of responses, all sorts of reinforcers
-- but not always strict matching
Schroeder &
Holland, 1969
What’s the
organism?
If your COD is long
enough to eliminate
concurrent
superstitions, you
get strict matching.
Effect of
COD
Herrnstein’s Law
AKA Herrnstein’s Hyperbola, the quantitative law of effect,
the law of simple action
Herrnstein (1970) developed a theory to extend the strict
matching law to performance on single schedules:
For single schedules:
Herrnstein (1970) made the following assumptions:
Herrnstein’s Assumptions:
1. That even in single-schedule
situations, there are other responses
than the ones we measure, and other
reinforcers than the ones we provide.
They are extraneous (or other)
responses and reinforcers. These we
will designate Be and Re.
Herrnstein’s Assumptions:
2. The total rate of behavioral output is
constant, and this constant is called k.
Thus, for a two-alternative concurrent
schedule, B1 + B2 + Be = k.
Herrnstein’s Assumptions:
3. Strict matching applies. The subject
divides its behavior proportionally
between obtained R1 and R2 and Re
and so on.
Herrnstein’s Assumptions:
4. The value of Re is constant.
Now for the equation…
From Herrnstein's assumption of strict matching we know
that:
B1
R1

B1  Be R1  Re
and from Herrnstein's constant-total output assumption
we also assume that
B1  Be  k
Thus we can write:
B1
R1

,
k R1  Re
kR1
and thus B1 
R1  Re
kR 1
B1 
R1  R e
This is Herrnstein's equation, or Herrnstein's hyperbola,
or the law of simple action.
k and Re are assumed constants.
And they have units, too: k is responses per minute if
you measured B1 as responses per minute; Re is
reinforcers per minute if you measured R1 as reinforcers
per minute.
R1
B1  k
R1  Re
k is the total output of behavior, and it is allocated to B1
according to the proportion of all reinforcers that are for
this response (R1/(R1+R2)).
The Context of Reinforcement
Reinforcers for behaviors occur in the context of other
reinforcers for other behaviors -- the bottom line of the
equation:
rft rate for this
response
kR 1
B1 
R1  R e
the context of
reinforcement
The rate of a behavior is controlled by the reinforcer rate
for that behaviour in the context of all reinforcers in the
situation.
More on the context of reinforcement:
Let’s look at the response rate on one alternative of a 2alternative concurrent VI VI schedule.
The context of reinforcement is now:
R1  R 2  Re
So, the equation for B1 is:
R1
B1  k
.
R1  R 2  Re
Herrnstein’s Hyperbola -- Theory
kR 1
B1 
R1  R e
The equation describes an hyperbola —
variations in R1 when it is low change behavior a lot,
but the same size variations in R1 when it is high change
behavior only a little.
- Diminishing marginal returns -- the changes
decrease as the reinforcer rate increases.
k = 100 B/m, Re = 5 R/h
RESPONSES\MINUTE
100
80
Concave downwards
60
Negatively accelerated
Diminishing marginal
returns
40
20
0
0
20
40
60
RFS/HOUR
80
100
k = 100 B/m, Re = 5 R/h
k = 100 B/m, Re = 20 R/h
RESPONSES\MINUTE
100
80
60
40
20
0
0
20
40
60
RFS/HOUR
80
100
k = 100 B/m, Re = 5 R/h
k = 40 B/m, Re = 5 R/h
RESPONSES\MINUTE
100
80
60
40
20
0
0
20
40
60
RFS/HOUR
80
100
k = 100 B/M, Re = 20 R/H
k = 40 B/M, Re = 20 R/h
RESPONSES\MINUTE
100
80
60
40
20
0
0
20
40
60
RFS/HOUR
80
100
How well does Herrnstein’s Law fit the data?
Herrnstein (1970) looked at the available data and fitted
his hyperbola to them.
- take the data, and change the values of k and Re until
you get a good fit.
-VAC: Goodness of fit is measured by the percentage of
the variance in the data that is accounted for by the fitted
values of k and Re .
k and Re are free parameters
Free Parameters
• Free parameter: a value that you don’t
know in any other way except by fitting the
data -- there is no other way of knowing it.
– Differences in individuals and situations
• BUT can know how to change its value -for instance,
– Animal less hungry-> Re should get bigger
• Does Re change in the right direction
when we change deprivation?
– Yes
• Are the values of k and Re reasonable?
Herrnstein’s hyperbola: Data
120
RESPONSES PER MINUTE
100
DATA
BEST FIT
80
BIRD P118
60
40
k = 113 responses/minute
Re = 6.8 reinforcers per hour
20
VAC = 91%
0
0
50
100
150
200
REINFORCERS PER HOUR
250
300
Herrnstein’s hyperbola: Data
120
BIRD P129
DATA
RESPONSES PER MINUTE
100
BEST FIT
80
60
40
k = 81 responses/minute
Re = 13.8 reinforcers per hour
20
VAC = 77%
0
0
50
100
150
200
REINFORCERS PER HOUR
250
300
R e s p o n s e s p e r m i n u te
Herrnstein’s hyperbola: Data
120
Bird P121
k = 104 responses/minute
Re = 291 rfts/hour
VAC = 84%
96
Data
Fit
72
48
24
0
0
50
100
150
200
Reinforcers per hour
250
300
In general, the model fits the data well.
The values of the free parameters (k and Re) look
reasonable, and the different values seem to pick up very
clear individual differences between the subjects.
Modulus
When you fit the model, the value of k that you obtain is in
terms of the measure you have taken. If the pigeon was
key pecking for food, it is in pecks/minute; if it is lever
pressing, it is in terms of lever-presses per minute.
In other words, k is measured in the modulus of the
behavior that you are reinforcing.
Modulus
The same is true for Re— it is measured in terms of R1. If
R1 consists of 3-s access to grain, then Re is measured in
these terms.
So, in the previous graphs, Re is strictly "equivalent foodreinforcer per hour“ – if you have arranged 3-s access to
grain, Re will be measured in “3-s access to grain
equivalents per hour”
Avoidance
40
DE VILLIERS (1974) - BIRD R8
DATA
RESPONSES PER MINUTE
BEST FIT
30
k = 58 responses/minute
Re = 6.4 SHOCKS PER MINUTE
VAC = 100%
20
10
0
0.0
0.5
1.0
1.5
2.0
2.5
REINFORCERS PER HOUR
REINFORCERS PER MINUTE
3.0
3.5
Avoidance
40
DE VILLIERS (1974) - BIRD R3
DATA
RESPONSES PER MINUTE
BEST FIT
30
k = 24 responses/minute
Re = 1.3 SHOCKS PER MINUTE
VAC = 98%
20
10
0
0.0
0.5
1.0
1.5
2.0
2.5
REINFORCERS PER HOUR
REINFORCERS PER MINUTE
3.0
3.5
Avoidance
40
DE VILLIERS (1974) - BIRD R13
DATA
RESPONSES PER MINUTE
BEST FIT
30
20
k = 48 responses/minute
Re = 1.2 SHOCKS PER MINUTE
10
VAC = 97%
0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
REINFORCERS PER HOUR
REINFORCERS PER MINUTE
3.5
4.0
Herrnstein’s Hyperbola -- Application
kR 1
B1 
R1  R e
Given this equation:
What two ways could
we reduce B?
Herrnstein's law and concurrent schedules
The strict-matching equations for describing rates of
responding during either alternative in concurrent VI VI
schedules are:
Alternative 1:
B1
R1

,
B1  B 2  Be R1  R 2  Re
Alternative 2:
B2
R2

B1  B 2  Be R1  R 2  Re
According to Herrnstein's assumptions, the denominators
(B1 + B2 + Be) both equal k, so:
kR1
B1 
,
R1  R 2  Re
and
kR2
B2 
R1  R 2  Re
If now we divide B1 by B2 to see what relative behaviour
allocation would look like, we get:
B1 R1

B2 R2
As we saw already, this is just the same relation as:
B1 R1

B2 R2
which is the strict matching law.
B1
R1

B1  B 2 R1  R 2
So, the response rate on one of the schedules comprising
a concurrent VI VI schedule is:
kR1
B1 
,
R1  R 2  Re
How good is Herrnstein's law (1)?
A law is only as good as the assumptions are true.
It fits the data well,
But is that enough? -- many other equations could
be good fits.
How good is Herrnstein's law (2)?
The assumption about Re existing -- that there are other
reinforcers available -- cannot be faulted. Logically, it
seems that this must be correct.
BUT the assumption that Re remains constant when R1 is
varied is probably unreasonable.
It seems unlikely that Re could remain constant when Be
varies considerably. Re surely must fall when R1 is
increased.
How good is Herrnstein's law (3)?
The assumption that total output (k) is constant seems
counter-intuitive, but it could be correct if you measure all
the responses in the same modulus -- which Herrnstein's
theory does. Even doing nothing is behaving.
How good is Herrnstein's law (4)?
The assumption of strict matching is likely to be wrong.
Considerable research has shown that the behavior ratio
changes rather less with changes in reinforcer ratios than
is suggested by strict matching.
Extensions of strict matching
Application of strict matching
Experimental Analysis of Choice
• Methods: concurrent schedules, concurrent chains,
delay discounting, foraging contingencies, behavioral
economic contingencies.
• Models and Issues: matching/melioration,
maximizing/optimality, hyperbolic discounting, behavioral
economic/ecological models, behavior momentum, molar
versus molecular issue, concepts of response strength.
• Applications: self-control, drug abuse, gambling, risk,
economics, behavioral ecology, social/political decision
making.
B1
r1

B1  B 2 r1  r 2
Assume B1  B 2  k .
B1
r1

k
r1  r 2
Set B1= B, r1= r, and r2= ro, where ro
represents other sources of reinforcement.
Thus,
kr
B
r  ro
Herrnstein's Hyperbola