Choice and Matching
Chapter 10
1
Steven I. Dworkin, Ph.D.
Choice
• Can you think of a situation or behavior that
does not involve choice?
2
Steven I. Dworkin, Ph.D.
Choice and Matching
• Concurrent schedule of reinforcement
– Simultaneous presentation of two or more
independent schedules, each of which leads to a
reinforcer.
• Conc
– VR 20
– VI 30
VR 50
VI 60
3
Steven I. Dworkin, Ph.D.
The Matching Law
• Hernstein’s 1961
• Concurrent VI
schedules (eg. VI 135”
VI 270” – 13 versus
27 reinforcers/hr)
4
Steven I. Dworkin, Ph.D.
The Matching Law
• Proportion of
• Ra/Ra+Rb =
responses emitted on a
SRa/SRa+SRb
particular schedule
matches the proportion
of reinforcers obtained
on that schedule.
5
Steven I. Dworkin, Ph.D.
Matching Law
•
•
•
•
•
Data from pigeon on a conc VI 30 VI 60
Reinforcers VI 30 – 119, VI 60 – 58
Responses VI 30 – 2800, VI 60 – 1450
Proportion of reinforcers -119/177 = .67
Proportion of responses – 2800/4250 = .66
6
Steven I. Dworkin, Ph.D.
Matching
• Not just in the laboratory
• Not just rats and pigeons
7
Steven I. Dworkin, Ph.D.
Deviations from Matching
• Undermatching – proportion of responses
on the richer alternative versus poorer
alternative is less different then those
predicted from matching (less different)
• Little cost for switching from one
alternative to another.
8
Steven I. Dworkin, Ph.D.
Deviations from Matching
• Overmatching – proportion of responses on
the richer versus poorer of two alternatives
is more different than would be predicted by
matching (overmatching more different)
• Cost to move is high
9
Steven I. Dworkin, Ph.D.
Deviations from Matching
• Bias – one alternative attracts a higher
proportion of responses than would be
predicted by matching, regardless of
whether that alternative is the richer or
poorer of the two alternatives.
10
Steven I. Dworkin, Ph.D.
Deviations from Matching
11
Steven I. Dworkin, Ph.D.
Quality and Amount
• Matching holds
12
Steven I. Dworkin, Ph.D.
Application to Single Schedules
• As relative reinforcement for operant
response increases responding will increase
• Context is important
– Outdated magazines in doctors office
– Covering JEABs when graduate applications
are due
13
Steven I. Dworkin, Ph.D.
Melioration
• To make better
• The distribution of behavior in a choice
situation shifts toward those alternatives
that have a higher value regardless of the
effect on the overall amount of
reinforcement
14
Steven I. Dworkin, Ph.D.
Problems with Melioration
• Tendency towards richer alternatives can
result in reduction in obtained
reinforcements
• VR 100 VI 30
– Allocation of study time to different courses
15
Steven I. Dworkin, Ph.D.
Problems with Melioration
• Over indulgence in a highly reinforcing
alternative can often result in long-term
habituation to that alternative, thus reducing
its value as a reinforcer.
– Too much of a good thing……
– Be careful of what you wish for….
16
Steven I. Dworkin, Ph.D.
Problems with Melioration
• Often the result of behavior being too
strongly governed by immediate versus
delayed consequences.
17
Steven I. Dworkin, Ph.D.
Optimization Theory
•
•
•
•
Make decisions that maximize satisfaction
Matching law –description
Optimization theory – explanation
Matching occurs when it is optimal thing to
do
• Concurrent VIs matching maximizes the
rate of reinforcement
18
Steven I. Dworkin, Ph.D.
Optimization versus Matching
• Mazur 1981
– Concurrent chained
schedules
– Matching over
optimization
• Concurrent VI VR
– Matching
• Other studies
optimization
19
Steven I. Dworkin, Ph.D.
Molar versus Molecular Control
• Context
• History
• Bias
20
Steven I. Dworkin, Ph.D.
Momentary Maximization Theory
• Selection of alternative with highest value at the moment
– Size and quality of reinforcer
– State of deprivation
• Momentary best choice not always best in long run (selfcontrol)
• Order in moment to moment patterns?
• Gambling experiment
• Data suggest absence of momentary maximization if
number of responses considered
• Maybe maximization if time considered
21
Steven I. Dworkin, Ph.D.
Delay Reduction
• Choice related to reduction in delay to
reinforcement
22
Steven I. Dworkin, Ph.D.
Self-Control Choices
• Small immediate reinforcer versus delayed
larger reinforcer.
– Impulsivity versus self-control
23
Steven I. Dworkin, Ph.D.
Self-Control
• Controlling Responses (Skinner)
• Controlled response
–
–
–
–
Physical restraint
Deprivation satiation
Doing something else
Self-reinforcement and self-punishment
24
Steven I. Dworkin, Ph.D.
Self-Control
• Temporal Issue
– Lack of self-control arises from the fact that our
behavior is more heavily influenced by immediate
consequences as opposed to delayed consequences.
Immediate Consequence
Delayed Consequence
quitting
withdrawal
Improved health
smoking
Nicotine high
Deterioration of health
25
Steven I. Dworkin, Ph.D.
Self-Control
• Self-control – preference for larger later
reward
• Impulsiveness – preference for smaller
sooner reward
26
Steven I. Dworkin, Ph.D.
Anslie-Rachlin Model
• Preference for self-control versus impulsive
choice shifts over time.
• Value of reward is hyperbolic function of
delay
– Value of reward increases more sharply as
delay decreases and reward becomes more
imminent.
27
Steven I. Dworkin, Ph.D.
28
Steven I. Dworkin, Ph.D.
Which do you prefer
• $500 now or $1,000 in two years
• $500 in four years or $2,000 in six years
29
Steven I. Dworkin, Ph.D.
Changing the Shape of Delay
•
•
•
•
•
•
•
Biological factors
Behavioral disorders
Age
Drugs
History of delayed rewards
Availability of other reinforcers
Chaining or setting up subgoals
30
Steven I. Dworkin, Ph.D.
Improving Self-Control
• Precommitment
• Self-reinforcement
• Punishment for impulsive option
31
Steven I. Dworkin, Ph.D.
Other Choice Situations
• Preference for variability
– Pigeons - fixed delay has to be 3-4 secs longer
to be preferred over a variable delay.
– VR 60 over FR 30
32
Steven I. Dworkin, Ph.D.
Preference for Variable Delays
• Delay discounting
– Longer delay less value
of reinforcer
33
Steven I. Dworkin, Ph.D.
Tragedy of the Commons
• Freedom in a commons brings ruin to all.
• precommitment
• Punishers more immediate reinforcers
34
Steven I. Dworkin, Ph.D.