Operant Behavior
can involve BOTH

Obviously, reinforcement schedules can control
responding


So can “rules”:
heuristics
algorithms
concepts and concept formation



operant conditioning can have rules, for example, the
factors affecting reinforcement.
Rate of Reinforcement

In general, the faster the rate of
reinforcement the stronger and more rapid
the reinforcement

Peaks at some point: asymptotic
– Can no longer increase rate of responding
– Do risk satiation
Amount of Reinforcement

In general, the MORE reinforcement the
stronger and more rapid the responding.

Again, at some point increasing the
amount will not increase response ratesat asymptote

Again, worry about habituation/satiation
Delay of Reinforcement

Critical that reinforcer is delivered ASAP after the response has
occurred.

Important for establishing contingency
– Is really a contiguity issue
– Doesn’t HAVE to be contiguous, but helps

Why?
– Responses occurring between the target response and the reinforcer
may become paired with the reinforcer or punisher
– Inadvertently reinforce or punish in between responses

Example: Child hits sister, mother says “wait till your father gets
home”
– Child is setting table
– Father walks in, hears about misbehavior, and spanks
– Child connects table setting with spanking
Reinforcer Quality

Better quality = more and stronger
responding

BUT: Inverted U-shaped function
– Too poor a quality = low responding
– Too high a quality = satiation

Think of the tenth piece of fudge: As good
as the first one or two?
Response Effort

More effortful responses = lower response
rates

Must up the reinforcer rate, amount or
quality to compensate for increased effort

Again, an optimizing factor:
– Low quality reinforcer not worth an effortful
response
Post-Reinforcement Pause
Organism must have time to consume the
reinforcer
 Longer pauses for more involved
reinforcers

– M&M vs. salt water taffy!
– This is not disruptive as long as plan for it
Satiation Hypothesis


Responding decreases when animal “full”
Satiation or Habituation?

Satiation = satiety: animal has consumed as
much as can consume

Habituation = tired of it

BOTH affect operant behavior
often hard to tell which is which
Extinction of Intermittently
Reinforced Behavior

The less often and the more inconsistently
behavior is reinforced, the longer it will take to
extinguish the behavior, other things being
equal

Behaviors that are reinforced on a “thin”
schedule are more resistant to extinction than
behaviors reinforced on a more dense schedule

Behavior that is reinforced on a variable
schedule will be more resistant to extinction
than behavior reinforced on a fixed schedule
Reducing Reinforcer Density

Large amounts of behavior can be obtained
with very little reinforcement using intermittent
schedules
– Initially, behavior needs dense schedule of
reinforcement to establish it
– preferably continuous reinforcement
– As the behavior is strengthened, reinforcement can
be gradually reduced in frequency

Start with as low a density as the behavior can
tolerate and decrease the density as
responding is strengthened
Schedule or Ratio Strain

If reinforcement is reduced too quickly, signs of
extinction may be observed
– Response rate may slow down
– Inconsistent responding may be seen
– May see an increase in other responses

If this happens, retreat to a denser reinforcement
schedule

Adding a conditioned reinforcer in between
reinforcements can help bridge the gap
Learned Helplessness
What happens when animals lose
their perceived control over
consequences?
LH in dogs
• 2 separate shock locations: harness or alley-way/shuttlebox
• Several procedures:
–
–
–
–
Inescapable shock exposure
instrumental escape/avoidance training
Testing for chronic failure to escape
TREATMENT
• Removed barrier between sides
• Opened observation windows on opposite side of box
• Humans called dogs to safety side
– Forced escape/avoidance
– Recovery: testing again
Learned helplessness
Marty Seligman
• Four groups of dogs
Grp
Grp
Grp
Grp
Training I and II
I Escapable/escapeable
II Inescapable/inescapable
III Escapable/inescapable
IV Inescapable/escapable
result Lasting effects
run
None
not run
None
not run
None
not run
Severe
Remember, Seligman’s hypothesis was that NONE of the
dogs would be significantly harmed.
Did it work?
• YES!
• Learning contingency was critical
• When learned that contingency was to escape- could
overcome the first “learning rule”
• Was one of first attempts at treatment
• Why so important? One of first studies to suggest that
you could be TAUGHT to overcome aversive events in
your life!
Key Factor = inescapability
once learned not to escape (learned to be
helpless)= not change
Characteristics of L.H.
• inescapability that produces phenomenon,
not the shock itself
• works under variety of procedures,
conditions
• very generalizeable, transferable
• if take far enough, can make it a
contingency rule for the animal, rather
than specific contingency for specific
situation(s)
Symptoms of L.H.
•
•
•
•
•
passivity
learned laziness
retardation of learning
somatic effects
reduction of helplessness with time
Clinical expressions of
learned helplessness
• School phobias
and math anxiety
• Abusive
Relationships
• Depression
• Cultural learned
helplessness
Clinical expressions of
learned helplessness
• School phobias
and math anxiety
• Abusive
Relationships
• Depression
• Cultural learned
helplessness
“Curing” or eliminating learned
helplessness
• Unlearn the rule
• Reshape or recondition
• Must be done in situation where
organism cannot fail
• Difficult to do- animals can “not”
respond
• UPenn program on relearning thoughts
during test taking
Two theories to explain
• Competing response hypothesis
– Dog learned motor responses which alleviated or
attenuated shock
– Those were used in shuttle situation, but ineffective
– Thus learn the wrong responses
• Adaptation hypothesis
– Organisms sensitive to independence of events
– Most situations: understand that shock is not dependent
on their behavior
– In LH: not make this distinction, rather shock WAS
dependent on their behavior
Does Extrinsic Reinforcement
produce reductions in motivation?
The answer depends on
1) How we define reinforcement
2) Whether we consider Learned
Helplessness!
Questions from the peanut gallery!
• Massive literature on the effectiveness of
positive reinforcement across many settings
• BUT: concern…..is the effect of extrinsic
reinforcement to reduce intrinsic
reinforcement?
– Reduces individuals feeling of competence?
– Reliance on “things” rather than internal
motivation
Where did this debate come from?
• 1970’s: began with cognitive revolution and
backlash against behavior modification
• 1994: meta-analysis by Cameron and Pierce
suggested deleterious effects of positive
reinforcement programs
• Newer evidence in mixed
– Some say it does
– Some say it doesn’t
Intrinsic vs. Extrinsic motivation
• Intrinsic motivation: Deci (1975)
– Motivation for which there is no recognizable reward except the
activity itself
– Cannot be attributed to external control
• Extrinsic motivation =
– motivation with recognizable reward
– Behavior can be attributed to external reinforcer unrelated to
activity
• Harlow’s work from 1950’s:
– If monkeys solved puzzle task with no external reinforcement,
assumed that it was “internal”
Problem with this definition:
• Assume that reinforcers = things
– Newer models of reinforcement assume
reinforcers can be activities or access to activities
– Based on traditional Equipotentiality principle of
reinforcement:
• Reinforcers are things
• Reinforcers are a class of things
– Once a reinforcer for someone, always a reinforcer
– Once a reinforcer for a class of organisms, always a reinforcer
for that class of organisms
• Reinforcers are trans-situational
Could this explain the earlier data?
• Engage in behavior in anticipation of future
events
• Engage in behavior because activity is
reinforcing in and of itself
• This is NOT necessarily “intrinsic”, but due to
narrow definition of internal and external
reinforcers by others;
Other interpretations
• Horcones, 1987; Mawhinney, Dickinson and Taylor, 1989
• Intrinsic reinforcement = consequences which occur in absence of
programming by others
• Natural and automatic responses inevitably produced by the
structural characteristics of the physical environment in which the
human exists
• Extrinsic reinforcers are those which are programmed by others
• Intrinsically controlled behaviors = behavior controlled by
unprogrammed consequences
• Assumption is that intrinsic motivation is of greater value
Deci’s studies
• Examined changes in intrinsic motivation when extrinsic rewards were
given
• College students: solved “intrinsically interesting” puzzles:
– Rewarded or not rewarded extrinsically for solving
– Also condition where reward was offered but then unavailable
– Presentation of rewards increased time on task; no rewards decreased time on
task
• College students examined headlines:
– Rewards increased extrinsic motivation but withdrawal of rewards increased
intrinsic motivation
• Examined verbal praise for tangible rewards
– Extrinsic reward group spent more time doing the task
– But: both groups reported “liking” the task equally
Cognitive Evaluation Theory
• Deci and Ryan (1985)
• Assumption: self determination and competence are innate
human needs
• Events facilitate or hinder feelings of competence or selfdetermination depending on perceived informational,
controlling or amotivational significance
• Two reward categories:
– Task contingent
– Quality dependent
Eisenberger and Cameron (1996)
• Also subdivides task-completion rewards
category
– Performance independent
– Performance dependent
• Intrinsic motivation most degraded when given
tangible, anticipated rewards- devalues “self
worth”
• Alternative answer: learned helplessness
– When have no control over outcomes, can develop LH
Overjustification Hypothesis
• With extrinsic rewards, may justify behavior based on those
extrinsic rewards
– Engage in the behavior because it is rewarded
– If expect a reward, less likely to engage in behavior in absence of
reward
– Conclude that this means intrinsic reward is devalued
• Overjustify the response: if a person is already performing a
response and receiving no extrinsic reward, introduction of
extrinsic reward will decrease intrinsic motivation
– Performance becomes “overjustified” by external reward
Problems with these theories
• Methodological concerns:
– Not examine behavior over long time horizon;
only examine immediate effect
– Which would you choose? No reward or reward
for same activity?
• Does this devalue your intrinsic motivation
• Or does it make you an optimizing organism?
Problems with these theories
• Confusing rewards and reinforcement
– Rewards = pleasant occurrence of an event, may or may not strengthen a
response
– Reinforcement: contingency between response and consequence strengthens
the probability of a response
• Data do not support overjustification hypothesis
• Misinterpretation of behavioral principles:
– Reinforcers are not necessarily things
– Activity itself can be reinforcing
– Confusing lack of perceived control with loss of intrinsic motivation
• Alternative explanations from behavioral analysis: better theories of
reinforcement
Behavioral studies:
• Important to use within-subject rather than
across group comparisons
– Too many individual differences
– Examine change in reward value across an
individual
– Misinterpretation of “reward” vs reinforcer
• When use within subjects methods, the
devaluing tends to disappear.
Perhaps the answer is in how we
define reinforcement
• Trans-situational hypothesis:
– A reinforcer = a reinforcer across situations,
subjects, etc.
– A reinforcer MUST be a thing, not an activity
• Modern view of reinforcement:
– NOT transituational
– NOT necessarily a thing
– But we will leave that for another day!!!!!