Operant Conditioning
Pavlovian
learning how stimuli provide information
about other stimuli
Operant
learning how your behavior leads to good
and bad results
Natural selection
 effective organisms survive
 other ones die out
 results in complex well adapted organisms
Behavioral selection
 behavior leading to good outcomes
is strengthened
 other behavior dies out
 results in complex behavior that is
suitable for the environment
The Law of Effect
(voluntary responses, not reflexes)
Responses followed by good things
are strengthened
Responses followed by bad things
are weakened
R
responses
Re
(reward or reinforcer)
Good
Consequence
Bad
Consequence
Behavior
produces
consequences
Positive
reinforcement
Punishment
Behavior
eliminates
consequences
Omission
Negative
Reinforcement
The Law of Effect
(voluntary responses, not reflexes)
Responses followed by good things
are strengthened
Responses followed by bad things
are weakened
R
responses
Re
(reward or reinforcer)
Good
Consequence
Bad
Consequence
Behavior
produces
consequences
Positive
reinforcement
Punishment
Behavior
eliminates
consequences
Omission
Negative
Reinforcement
Increase
probability
of response
The Law of Effect
(voluntary responses, not reflexes)
Responses followed by good things
are strengthened
Responses followed by bad things
are weakened
R
responses
Re
(reward or reinforcer)
Good
Consequence
Bad
Consequence
Behavior
produces
consequences
Positive
reinforcement
Punishment
Behavior
eliminates
consequences
Omission
Negative
Reinforcement
Decrease
probability
of response
stop
reinforcement
bar press rate
spontaneous
recovery
acquisition
extinction
rat
out
of
box
time
Operant conditioning may also be subject
to inhibition.
Contingency vs. Contiguity
Does the response have to predict
the consequence?
Hammond (1980)
food
no
food
press bar
5
95
don’t press
0
100
food
no
food
press bar
5
95
don’t press
5
95
positive
contingency
Rat learns
to press!
zero
contingency
Rat stops
pressing or
doesn’t learn
Supports Contingency
Conditioned Reinforcement
Skinner (1938)
Phase 1
“click” . . . . . food
conditioned
reinforcer
primary
reinforcer
Phase 2
Rats learn to press a bar to hear
the “click”
Other examples
praise
money
grades
Conditioned reinforcers only develop when they
predict primary reinforcer
(more evidence for contingency)
Token Reinforcers
First train animals that tokens can be
exchanged for food
(e.g., vending machine)
Then see if animals will learn other responses
to get tokens
Chimpanzees
•tokens work as well as “real” reinforcers
•fight over the tokens
Learned Helplessness (Seligman)
Phase I - Learning to Escape
Shock
Control Dogs
panel
Yoked Dogs
•A long lasting shock is given to both groups
every once in a while
•Control dogs can turn shock off by pushing a
panel
•Yoked dogs’ shock turns off too, when control
dog pushes panel
•Yoked dogs can do nothing themselves to escape
shock
Phase 2 - Avoidance Learning
hurdle
•shock delivered to one side of box
•if dog jumps hurdle to other side
there is no shock
Control dogs learn to avoid shock
Yoked dogs don’t
Yoked dogs have learned that they can’t stop shock
They have learned to be helpless
“Curing” learned helplessness
•Drag the dog over the hurdle a few times
•Eventually learns to avoid shock
Curing learned helplessness requires the animal to
experience success
Depression is like learned helplessness
•passive
•lose weight
•have trouble learning new tasks
Preventing Learned Helplessness
If dogs are first exposed to shock that they can
turn off and then exposed to uncontrollable
shock, they don’t learn to be helpless.
Early Experience!
Methods for Studying Memory
STUDY
TEST
retention
interval
Recall Tests
Free recall
Serial recall
Cued recall
steel
thread
tights
universe
cob
batteries
eye
cow
pear
clarinet
nail
pitch
airplane
mittens
bunny
sprinkler
spaghetti
saucer
canary
microphone
Recognition
DRUM
CORD
KITE
Study
PARROT
RESPONSE
“YES”
“NO”
yes-no
Test
OLD
HIT
MISS
ITEM
CLASS
KITE
NEW
CORD
CORN
KITE
BIRD
Forced
Choice
FALSE
ALARM
CORRECT
REJECTION
Experiments with no study phase
Autobiographical Memory
•Recall all of your elementary school teachers
•Where were you when “Challenger” space
shuttle blew up?
How do you know if recall is accurate?
Semantic Memory
Tests of general knowledge
Easy questions
“Is an oak a tree?”
measure response time
Hard questions
Which is bigger, an eagle or a raccoon?
Savings in Relearning
Learn
something
Relearn
the same thing
How many
trials?
If it takes
few trials, you
have memory
% savings =
trials for
trials for
original learning
relearning
trials for original
learning
Example
learning ancient Greek passages
STUDY - 1-1/2 year old has Greek passages read
to him
TEST - Age 8
TEST
10 new passages
30 trials
10 old passages
21 trials
% savings =
30-21
= 30%
30
Conclude:
There is some memory for old passages
“Classic” or Standard Theory of Memory
Selectivity of Encoding
Only some information is stored
permanently
A
B
C
D
E
F
G
H
I
J
K
L
CD
H
E]
F
G
E
conveyor
Rejects
For closer
examination
Hi-grade
Information from sense organs
Sensory Memory
conveyor
Attention
System
Short-Term Store
Rejects
For closer
examination
Hi-grade
Long
Term
Store
Forgotten
Classic Theory
Sensory
Memories
•large capacity
•rapid turnover
•“raw”
information
eyes
ears
Iconic
Store
Echoic
Store
Attention Filter
Short-Term
Store
•small capacity
•symbolic
information
Long-Term
Store
•almost
unlimited
capacity
Short-Term Store
some metaphors
1. Box with Slots
(limited
capacity)
parrot pizza
jet
2. A work bench
where
thinking
takes place
parrot jet
3. STS is the activated part of long-term store
fly
plane
bird
parrot
jet
helicopter
pizza
food
hamburger
4. STS is your consciousness
jet
parrot
pizza
What is in STS?
Mental Symbols
sound of the word “dog”
appearance of the letter “A”
world
STS
LTS
retrieval from LTS
What happens to items in STS?
Sensory
Memory
STS
transferred
to LTS if
rehearsed
a lot or if
important
LTS
forgotten
if not
rehearsed
rehearsal
keeps item
in STS
What do symbols in
STS stand for?
(Coding)
Classic Theory
words & letters are coded by sound
(acoustic)
OR
internal articulation
not by meaning
Conrad (1964)
serial recall of letters
confusions are acoustic
code must be acoustic
F
D
K
Y
X
C
M
B
C
P
V
E
C
T
right rite wright right write rite
How is STS Limited?
1. Number Hypothesis
STS can hold only a fixed number
of symbols
7  2 (Miller)
2. Time Hypothesis
Any symbol in STS will be lost in
around 2 seconds if it is not
reactivated by rehearsal
Span of STS is determined by decay rate (2 sec)
and the rate of rehearsal
Baddeley, Thomson & Buchanan (1975)
tested memory span for words
memory span = number of items where immediate
serial recall is perfect
short words
long words
“pig” “bet”
“noon “rose” (take longer
to say to
yourself)
Memory span was greater for short words
Conclude:
Your memory span is the number of words
you can rehearse in 2 seconds
The Time Hypothesis is correct
Improving STS Capacity
1. Chunking with recoding
combine many symbols into one
THE DOG

“the dog”
0 0 1
1
0 1 0
1 1 1
2
7

2. Chunking without recoding
organize symbols into groups
716 - 275 - 7261
Are STS and LTS Really Different?
Classic Theory says “yes”!
STS
LTS
time
a few
seconds
“permanently”
capacity
a few
items
“unlimited”
cause of
forgetting
decay or
“getting
bumped
out”
retrieval
failure
nature of
symbols
(coding
acoustic/
articulatory
for words
anything
Immediate Free-Recall Serial Position Curve
100%
recency
primacy
recall
asymptote
|
| |
1 2 3
serial position
25
Immediate Free-Recall Serial Position Curve
STS
100%
LTS
recency
primacy
recall
asymptote
|
| |
1 2 3
serial position
25
Dissociations in free recall
serial position curve
1.
Recall
No dissociation
Condition A is
easier than B
over all positions
A
B
2.
A is better than
B for nonrecency
only
A
B
3.
A is better for LTS
A is better than B
for recency only
A
B
A is better for STS
Actual Dissociations
Supporting STS-LTS Distinction
Independent
variable
Effect on
Recency
Effect on
Nonrecency
presentation
rate
SLOW vs. FAST
SLOW =
FAST
SLOW >
FAST
NO TASK >
TASK
NO TASK =
TASK
YOUNG ≈
OLD
YOUNG >
OLD
AMNESIC ≥
NORMAL
AMNESIC
=0
intervening task
before recall
or
no task
old-age vs.
college-age
subjects
amnesic
vs.
normal
subjects
Immediate Free-Recall Serial Position Curve
100%
recency
primacy
recall
asymptote
|
| |
1 2 3
serial position
25
Immediate Free-Recall Serial Position Curve
100%
recency
primacy
recall
asymptote
FINAL FREE RECALL
| | |
1 2 3
serial position
Negative
Recency
Effect
25
Craik (1970)
Recency items are recalled worse than nonrecency
items in final free recall
Supports STS-LTS distinction
Effects of Marijuana on Memory
Darley et al. (1973)
start
•presentation and immediate free recall of
ten lists
•administration of drug or placebo
after one hour Phase 1
Final free recall of ten lists
drug
Recall .4 .2 - placebo
serial position
Phase 2
Recall .8 .6 .4 .2 -
No difference
Drug doesn’t
effect retrieval
from LTS
Presentation and immediate free
recall of ten new lists
Drug is
worse on
nonrecency
only
placebo
drug
serial position
Conclude: Drug hurts transfer from STS to LTS
Problems with Classic Theory
(1) You can get recency even when STS isn’t
involved
•Baddeley & Hitch (1977) “Rugby” experiment
•Roediger & Crowder (1976) “President”
experiment
Washington
Ford
Lincoln
100%
Nixon
Recency
serial position of presidents
(2) The Classic Theory’s view of rehearsal is wrong
Craik & Watkins (1973)
Phase 1
keep track of most recent “b” word
“basket” “spoon” “telephone” “baby”
report “baby”
Rehearsal time
basket - 2 items
Phase 2
recall all “b” words
20—
%
recall
10 —
Actual Data
|
0
|
1
|
|
|
2
3
4
Rehearsal Time
|
5
|
6
(2) The Classic Theory’s view of rehearsal is wrong
(sheer amount of rehearsal doesn’t matter always)
Craik & Watkins (1973)
Phase 1
keep track of most recent “b” word
“basket” “spoon” “telephone” “baby”
report “baby”
Rehearsal time
basket - 2 items
Phase 2
recall all “b” words
If classic theory
is right
20—
%
recall
10 —
Actual Data
|
0
|
1
|
|
|
2
3
4
Rehearsal Time
|
5
|
6
Levels of Processing
Theory of Memory
A modification of the Classic Theory
Craik & Lockhart (1972)
•memory is determined by encoding
•the “deeper” the processing during encoding, the
better the memory
•amount of rehearsal is not important
•distinction between STS (primary memory) and
LTS (secondary memory) is still present
Craik & Lockhart (1972)
2. Kinds of rehearsal
maintenance - keep repeating word
at shallow (phonemic) level
KEEPS WORD AVAILABLE, but
DOES NOT improve long-term
memory for word
elaborative - think about the
meaning of the word in lots
of different ways
(semantic)
does improve long-term memory
Relation Between Rehearsal and Perception
What happens when you perceive a word?
DOG
Stage 1
(visual)
detect parts of letters
recognize letters
“D” “O” “G”
Stage 2
(phonemic)
convert letters to “phonemes”
D=d
Stage 3
(semantic)
O=o
G = hard g
convert phonemes to meaning
“d o g”
=
pet that barks
Event:
DOG
shallow
letters
(visual)
concentrate
on letters
concentrate
on sound
concentrate
on meaning
poor
memory
sound
(phonemic)
meaning
(semantic)
deep
fairly
short
memory
long
lasting
memory
1. Personally significant?
2. 2 syllables?
3. Animate?
4. Personally significant?
5. 2 syllables?
6. Animate?
7. Personally significant?
8. 2 syllables?
9. Animate?
10. Personally significant?
11. 2 syllables?
12. Animate?
13. Personally significant?
14. 2 syllables?
15. Animate?
16. Personally significant?
17. 2 syllables?
18. Animate?
19. Personally significant?
20. 2 syllables?
21. Animate?
Watkins & Watkins, 1974
list
length
known
immediate recall
list
length
unknown
final recall
serial position
Supports the idea that negative recency in final
recall is due to shallow processing when list
length is known (maintenance rehearsal)
Support for Levels of Processing
Craik & Tulving (1975)
Orienting questions
Case
word in capital letters
subject responds
Rhyme
word rhymes with weight?
subject responds
Sentence
TABLE
yes
MARKET
no
word fits in sentence
“He met a _______”
subject responds
FRIEND
yes
.8
Probability
of “Hit” in
Recognition
.6
yes
no
.4
yes
no
yes
.2
sentence
rhyme
no
case
Deeper means better memory
“yes” better than “no” (elaboration)
Some Criticisms
1. Theory fails to emphasize retrieval cues.
2. Theory fails to say exactly what is “deeper”
processing
Fisher & Craik (1977)
Shows that retrieval cues are as important
as depth of processing during encoding.
STUDY
TEST
context - item
pail - HAIL
milk - COW
mat - CAT
leaf - TREE
rhyme context
semantic context
rhyme context
semantic context
pail? (HAIL)
dog? (CAT)
milk? (COW)
free? (TREE)
cue
encoding
same cue - rhyme
semantic - rhyme
same cue - semantic
rhyme - semantic