Frode Svartdal
University of Tromsø
Oct. 2013

Extinction: Basics



Extinction is defined in terms of a reinforcement process
Extinction contingencies


The stimulus (S R or US) is discontinued
The learning contingency is discontinued
Extinction process

The conditioned response is reduced (strength, frequency, etc.)

Relearning, … not forgetting

Extinction: Basics
Operant conditioning
Catania, 1984)

Extinction: Basics
Classical conditioning





In general: Fast acquisition / high rate of responding  fast extinction
Amount of reward

High  fast extinction
Variability



Stimulus
Response
Reinforcement
= high ext. persistence
Some forms of learning do not extinguish
(easily)

Evaluative conditioning (e.g., Diaz, Ruiz, & Beyens, 2005)


Partial Reinforcement Extinction Effect


Partial (Intermittent) Reinforcement (PRF)
 increased extinction response
Continuous Reinforcement (CRF)
 reduced extinction persistence

First demonstrations
Operant conditioning; free operant; rats;
Skinner (1938)
Classical conditioning; blink response; students;
Humphreys (1939)
100%
50%

Free operant
Ferster & Culbertson, 1975

Free operant
Compared to CRF:
PRF 
• higher asymptotes
• more persistent responding under extinction
EXTINCTION
PRF
CRF

Rats, maze running speed under extinction
(Weinstock, 1954)
PRF (30%)
CRF

PRF response rate
LOWER than CRF response rate
Classical conditioning (rats): PREE
Extinction
100%
25%
50%
15%

Classical conditioning; eyelid; human subjects
(Svartdal & Flaten, in prep.)

Operant conditioning; humans;
Svartdal, 2003, Exp. 4


PREE is a very robust outcome


Measures & species





Bar pressing, rats
Maze running, rats
Pecking, pigeons
Blink reflex, humans, rabbits
…
Contingency


Operant/instrumental


Discrete trial
Free operant
Classical




How general is the PREE?


Reversed PREE observed under some conditions
Generalized PREE observed under some conditions
Alternative methods of analysis

Nevin (1988): ”PREE is an artefact because of wrong method of analzing extinction performance”
Response unit issue

PREE or not dependig on how the response is defined (Mowrer & Jones, 1945!

What happens if the subject is exposed to a mixture of PRF and CRF contingencies?

Reversed PREE
Pavlik & Carlton, 1965: Rats; bar pressing, free operant



Gr. 1: Single contingency; CRF
Gr. 2: Single contingency; PRF
Gr. 3: Two signalled schedules alternated for the same subjects; CRF
+ PRF

Reversed PREE
Conventional
PREE

Reversed PREE
Reversed
PREE

Reversed PREE


Pavlik & Carlton (1965):
Single reinforcement schedules (CRF vs. PRF) in betweengroups experiments  PREE
Two schedules (CRF vs. PRF) for the same subjects 
Reversed PREE



Other research
Reversed PREE observed
Generalized PREE (overall increased persistence, but no difference between conditions)
Conventional PREE rarely if ever observed in within-subjects manipulations of CRF - PRF

PREE as a generalization:
Ecological validity
If applied to a situation with a very specific schecule for a specific behavior  PREE
If applied to various situations with mixed contingencies 
Reversed PREE
Generalized PREE
Example:
Single mother – child is begging for toys only from mom
Example:
Mother and father – child begs for toys from both

Response unit issue

Free operant responding: What is the response unit?
Mowrer & Jones,1945:
What should be counted as the response unit - single responses or the unit of responses required for reinforcement?


Free-operant
Intermittent reinforcemet, e.g., FR4

Response unit
FR4
Reinforced responses

PREE
Total responses
Reversed PREE
Total responses / reinforcement ratio

Nevin: PREE is an artefact

PREE: Alternative analyses
•
•
Nevin, 1988: Behavioral momentum
”RPREE” is the rule – the response is stronger following CRF
•
• in free-operant responding (but not in discrete-trial experiments) following extended training
Extinction performance
•
Traditional measure: Number of responses
•
Nevin: Slope of the extinction curve

Nevin, 1988
Absolute number of responses
Relative to initial ext response level
SHORT LONG
PREE
RPREE

PREE vs. RPREE – important variables




Dependent measure

No. of responses vs. relative change
Type of situations

Free operant vs. discrete trial
Complexity of situation

One vs. more schedules (e.g., multiple schedule)
Design

Between groups vs. within subjects

Measure Number of responses
Situation
Schedule
Design
Other
Discrete trial
Single
Between-groups manipulation of reinforcer rate
CRF schedule must be 100%

PREE: My interests


Interaction PREE & Reversed PREE
Cognition (verbalization) related to behavioral PREE

The experimental situation
”Computer responses” presented
Left, right
Subject responses recorded
Left, right

The experimental situation




Task
Complete a four-response chain of responses started by the computer

E.g.: Computer: L R Subject: R L
Instructed task: Identify and apply the functional rule(s)

”Obtain as many correct answers as you can.”
Rules (depending on experiment)


”Repeat computer sequence”
”Reverse computer sequence”
Feedback (visual, autitory) for correct answer; nothing happens if answer is incorrect

The experimental situation
Manipulations (between groups and/or within groups)
Rule
Reverse (typically used)
Repeat
Contingency
CRF (100%)
PRF (20-60%)

The experimental situation


Reward rate manipulated


Between groups
Within subjects (multiple schedule)
Discrete trial situation; fixed number of trials


180 acquisition trials
40 extinction trials

Conventinal PREE; operant responding; students; Svartdal, 2003, Exp. 4

Reversed & conventional PREE; operant responding; students; Svartdal, 2000



Purpose: Explore the relationship between PREE and RPREE
PREE vs. RPREE: Contradiction or compatible effects?
Method


Independent groups: PRF and CRF
Within: CRF and PRF

Svartdal, 2000 ctd.




Multiple schedule, alternating
Group 40/40


Half trials (signalled): 40%
Half trials (signalled): 40%
Group 80/80

Half trials (signalled): 80%

Half trials (signalled): 80%
Group 80/40


Half trials (signalled): 80%
Half trials (signalled): 40%
PRF
”CRF”
”CRF” + PRF

PREE
* No. of responses: RPREE
* Relative change: No difference
80%
40%

Svartdal, 2000 ctd.


Relationship between schedule components
Simplest assumption: Modulation between component schedules:

60% + context = 60%  reference


60% + context = 100%  reduced persistence
60% + context = 20%  increaced persistence

Performance of a 60% schedule depending on other schedule = 100%, 60%, or 20%
Svartdal, 2000

Svartdal, F. (2000). Persistence during extinction:
Conventional and Reversed PREE under multiple schedules.
Learning and Motivation, 31, 21-40.

Cognition in PREE
•
•
Currently: Strong cognitive arguments to interpret conditioning in terms of cognition
•
•
•
•
Classical conditioning: Lovibond & Shanks, 2002
Operant conditioning: Shanks & St John, 1994
Implicit learning doubted: Shanks, 2005
Extinction: Lovibond, 2004
Basic argument:
CONTINGENCY  CONSCIOUS APPREHENSION  BEHAVIORAL CHANGE
CONTINGENCY  CONSCIOUS APPREHENSION  NO BEHAVIORAL CHANGE
•
Large number of studies supporting this assumption

Cognition in PREE

So, since the behvioral PREE is very robust, a ”cognitive PREE” must be easy to measure

Basic prosedure:

Behavioral acquisition under 100% vs. 60% reinforcer rate

Measurement of verbalized PREE

Cognition in PREE
Prediction of persistence:
”How likely is it that you will continue responding if reward no longer appears?”
Several experiments have demonstrated no sensitivity to learning history in predictions

3 extinction trials; immediate behavioral sensitivity
No difference in predictions
1,0
0,9
0,8
0,7
0,6
0,5
0,4
0,3
0,2
0,1
0,0
Trl1 Trl3 Trl5
100/100
60/60
Trl7 Trl9 Trl11 Trl13 Trl15 Trl17 TExt
Svartdal & Silvera, in prep.

Cognition in PREE
Retrospective judgments:
”How many responses did you emit after reward no longer appeared?”
Subjects are very accurate in descrbing their own behavior, including their own extinction persistence

Cognition in PREE
Svartdal, F. (2003). Extinction after partial reinforcement: Predicted vs. judged persistence.
Scandinavian Journal of Psychology , 44 , 55-64.

Meta-cognitive PREE?


We all have long experience with various contingencies
Maybe a ”meta-cognition” evolves:

Uncertain outcomes  Persist

Certain outcomes  Quit

Meta-cognitive PREE?


Scenarioes presented to subjects, manipulation


Reliable outcome vs.
Unreliable outcome
Persistence judgments of behavior

Meta-cognitive PREE?
Naive students: No effect of outcome manipulation

Meta-cognitive PREE?
Psychology students
(have read about PREE)
Naive students

Meta-cognitive PREE?
Svartdal, F. (2000). Persistence during extinction:
Are judgments of persistence affected by contingency information? Scandinavian Journal of Psychology,
41, 315-328.

PREE: Theory
Mowrer & Jones: Diskriminasjonshypotesen

PRF:


Læringbetingelsene  ekstinksjonsbetingelsene
Generalisering til ekstinksjon

CRF:


Læringbetingelsene # ekstinksjonsbetingelsene
Liten generalisering til ekstinksjon

PREE: Theory
Amsel: Frustrasjonshypotesen


PRF:

Forventning om belønning  frustrasjon når belønning uteblir



Frustrasjons-cues assosieres med læringssituasjonen
Under ekstinksjon: Frustrasjon pga uteblitt belønning
Læringssituasjonen  ekstinksjonssituasjonen
CRF:

Frustrasjon oppstår ikke under læring

Læringssituasjonen # ekstinksjonssituasjonen

PREE: Theory
Capaldi: Sequential hypothesis


PRF:



Ikke-belønnede trials blir signal på at belønning snart vil følge: … N N N R N N N R …
Dvs.: Det opparbeides en forventning om belønning når belønning uteblir
Under ekstinksjon: Mange responser pga forventning om belønning
CRF:


Ingen erfaring med uteblitt belønning under læring
Under ekstinksjon: Få responser

PREE: Theory



Status:

Diskriminasjonshypotesen står svakt



Amsels hypotese står rimelig sterkt
Capaldis hypotese står ganske sterkt
Nevins modell: Ingen hypotese i vanlig forstand
Discrete-trial-situasjonen

Capaldi og Amsel dominerende
Fri-operant-situasjonen

Svak teoretisk forståelse