Gazzaniga • Heatherton • Halpern
Psychological Science
FOURTH EDITION
Chapter 6
Learning
©2013 W. W. Norton & Company, Inc.
“Old Brain, New Tricks”
Brain researchers have new hope for people who struggle with
words. As this ScienCentral News video explains, they’ve
shown that treatment to improve the reading skills of adult
dyslexics actually changes how the dyslexic brain processes
information—teaching old trains new tricks.
Learning
• B. F. Skinner, who was inspired by the work of
Watson and Pavlov, has been one of the most
influential people in contemporary psychology
• Skinner believed that, to be scientists,
psychologists had to study observable actions
and focus on the behaviors people and
nonhuman animals display
“Bird Brain Gene”
Human speech and bird song may have more in common than
we know, according to scientists at Duke University. As this
ScienCentral News video reports, the research could lead to a
new progress for people with genetic speech disorders.
6.1 What Ideas Guide
the Study of Learning?
• Define classical conditioning.
• Differentiate between US, UR, CS, and CR.
• Describe the role of learning in the
development and treatment of phobias and
drug addiction.
• Discuss the evolutionary significance of
classical conditioning.
• Describe the Rescorla-Wagner model of
classical conditioning.
6.1 What Ideas Guide
the Study of Learning?
• Skinner and other behaviorists dismissed the
importance of introspection and mental states
in favor of basic learning principles and
scientific approaches to psychology.
• Learning theories have been used to improve
quality of life and to train humans and
nonhuman animals to learn new tasks.
Learning Results from Experience
• Learning: a relatively enduring change in
behavior, resulting from experience
• Associations develop through conditioning, a
process in which environmental stimuli and
behavioral responses become connected
– classical (Pavlovian) conditioning: learning that
two types of events occur together
– operant (instrumental) conditioning: learning that
a behavior leads to a particular outcome
“Expert Nose”
You may be skeptical when someone sniffs a glass of wine and
says it has an “oaky bouquet” or “overtones of cherry and
cinnamon.” But new research suggests that becoming an
expert smeller is in reach for all of us. This ScienCentral News
video explains.
Learning Results from Experience
• Learning Theory arose in the early twentieth century
in response to Freudian and introspective
approaches
• John B. Watson argued that only observable behavior
was a valid indicator of psychological activity, and
that the infant mind was a tabula rasa, or blank slate
• He stated that the environment and its effects were
the sole determinants of learning
• Behaviorism was the dominant paradigm into the
1960s, and had a huge influence on every area of
psychology
“Live Learning”
Have you ever wondered how babies learn to talk? As this
ScienCentral News video reports, some researchers now think
it’s more than just hearing the people around them.
Behavioral Responses Are Conditioned
• Watson was influenced by Ivan Pavlov’s research on
the salivary reflex, an automatic response when food
stimulus is presented to a hungry animal
• Pavlov noticed the dogs salivated as soon as they
saw the bowls that usually contained food,
suggesting a learnedresponse
• Twitmyer made a similar observation of the kneejerk reflex in humans: when paired with a bell,
subjects can be conditioned to demonstrate the
knee-jerk response without other triggers
Pavlov’s Experiments
• Classical (Pavlovian) conditioning: A neutral object
comes to elicit a response when it is associated with
a stimulus that already produces that response
• A typical Pavlovian experiment involves:
– Conditioning trials: neutral stimulus AND unconditioned
stimulus are paired to produce reflex, e.g. salivation
• Neutral stimulus: anything the animal can see or hear as long as it
is NOT associated with the reflex being tested, e.g. ringing bell
• Unconditioned stimulus (US): a stimulus that elicits a response,
such as a reflex, without any prior learning, e.g. food
– Critical trials: neutral stimulus alone is tested, and effect
on the reflex is measured
Terminology of Pavlov’s Experiments
• Unconditioned response (UR): a response that does not have
to be learned, such as a reflex
• Unconditioned stimulus (US): a stimulus that elicits a
response, such as a reflex, without any prior learning
• Conditioned stimulus (CS): a stimulus that elicits a response
only after learning has taken place
• Conditioned response (CR): a response to a conditioned
stimulus; a response that has been learned
Can you think of any learned associations that have classically
conditioned you?
Acquisition, Extinction, and
Spontaneous Recovery
• Pavlov was influenced by Darwinand believed that
conditioning is the basis of adaptive behaviors
• Acquisition: the gradual formation of an association
between the CS and US
• The critical element in the acquisition of a learned
association is time, or contiguity
• The CR is stronger when there is a very brief delay
between the CS and the US
– For example, scary music begins to play right before a
frightening scene in a movie—not during or after
Acquisition, Extinction, and
Spontaneous Recovery
• How long do learned behaviors persist?
• Animals must learn when associations are no longer
adaptive
– extinction: a form of learning that the prior association no
longer holds. The CR is weakened when the CS is repeated
without the US, and eventually extinguishes
• Spontaneous recovery: a previously extinguished
response reemerges after the presentation of the CS
• The recovery will fade unless the CS is again paired
with the US
• Extinction inhibits the associative bond, but does
not eliminate it
Generalization, Discrimination, and
Second Order Conditioning
• In a learning situation, how does the brain determine
which stimulus is relevant?
• stimulus generalization: responding to stimuli that
are similar but not identical to the CS produce the CR
• stimulus discrimination: a differentiation between
two similar stimuli when only one of them is
consistently associated with the US
• Second-order conditioning: a CS becomes associated
with other stimuli associated with the US. This
phenomenon helps account for the complexity of
learned associations
Phobias and Addictions Have
Learned Components
• Classical conditioning helps explain many
behavioral phenomena. Among the examples
are phobias and addictions.
Phobias and Their Treatment
• Phobia: an acquired fear out of proportion to
the real threat of an object or of a situation
• Fear conditioning: the process of classically
conditioning animals to fear neutral objects
• The responses include specific physiological
and behavioral reactions
– freezing: may be a hardwired response to fear that
helps animals deal with predators
Phobias and their Treatment
• In 1919, John B. Watson became one of the first
researchers to demonstrate the role of classical
conditioning in the development of phobias by
devising the “Little Albert” experiment
• At the time, the prominent theory of phobias
wasbased on Freudian ideas about unconscious
repressed sexual desires
• Watson proposed that phobias could be explained by
simple learning principles, such as classical
conditioning
Phobias and their Treatment
• The “Little Albert” Research Method:
– Little Albert was presented with neutral objects (a white
rat and costume masks) that provoked a neutral response
– During conditioning trials, when Albert reached for the
white rat (CS) a loud clanging sound (US) scared him (UR)
– Results: Eventually, the pairing of the rat (CS) and the
clanging sound (US) led to the rat’s producing fear (CR) on
its own. The fear response generalized to other stimuli
presented with the rat initially, such as the costume masks
– Conclusion: Classical conditioning can cause people to fear
neutral objects
Phobias and their Treatment
• Watson planned to conduct extinction trials to
remove the learned phobias but Albert’s mother
removed him from the study
– Do you think this type of research is ethical?
• Watson’s colleague, Mary Cover Jones, used classic
conditioning techniques to develop effective
behavioral therapies to treat phobias
– Counterconditioning –exposing a patient to small doses of
the feared stimulus while they engage in an enjoyable task
Phobias and their Treatment
• Systematic desensitization:a formal treatment
based on counterconditioning
– Developed by behavioral therapist Joseph Wolpe
in 1997
– CS → CR1 (fear) connection can be broken by
developing a CS → CR2 (relaxation) connection
• Psychologists now believe that exposure to
the feared stimulus is more important than
relaxation
Drug Addiction
• Classical conditioning also plays an important role in
drug addiction.
• Environmental cues associated with drug use can
induce conditioned cravings
• Unsatisfied cravings may result in withdrawal, an
unpleasant state of tension and anxiety, coupled with
changes in heart rate and blood pressure
• The sight of drug cues leads to activation of the
prefrontal cortex and various regions of the limbic
system and produces an expectation that the drug
high will follow
Drug Addiction
• Psychologist Shepard Siegel (2005) believed exposing addicts
to drug cues was an important part of treating addiction
– Exposure helps extinguish responses to the cues and prevents them
from triggering cravings
• Siegel and his colleagues conducted research into the
relationship between drug tolerance and situation
– The body has learned to expect the drug in that location and
compensates by altering neurochemistry or physiology to metabolize
it
– Conversely, if addicts take their usual large doses in novel settings,
they are more likely to overdose because their bodies will not respond
sufficiently to compensate
Classical Conditioning
Involves More Than Events
Occurring at the Same Time
• Pavlov’s original explanation for classical conditioning was
that any two events presented in contiguity would produce a
learned association
• Pavlov and his followers believed that the association’s
strength was determined by factors such as the intensity of
the conditioned and unconditioned stimuli
• However, in the mid-1960s, a number of challenges to
Pavlov’s theory suggested that some conditioned stimuli were
more likely than others to produce learning
• Contiguity was not sufficient to create CS-US associations
Evolutionary Significance
• Psychologist John Garcia and colleagues showed that certain
pairings of stimuli are more likely to become associated than
others
• conditioned food aversion:the association between eating a
food and getting sick
– Response occurs even if the illness was caused by a virus or some
other condition
– Especially likely to occur if the food was not part of the person’s usual
diet. A food aversion can be formed in one trial
• Animals that associate a certain flavor with illness, and
therefore avoid that flavor, are more likely to survive and pass
along their genes
Evolutionary Significance
• Learned adaptive responses may reflect the survival
value that different auditory and visual stimuli have
based on potential dangers associated with the
stimuli
• What evolutionary value do you see in this learned
behavior?
• Biological preparedness: Psychologist Martin
Seligman (1970) argued that animals are genetically
programmed to fear specific objects
• People are predisposed to wariness of outgroup
members (Olsson, Ebert, Banaji, & Phelps, 2005)
Learning Involves Cognition
• Classical conditioning is a way that animals come to
predict the occurrence of events—which prompted
psychologists to take a cognitive perspective on
learning
• Robert Rescorla argued that for learning to take
place, the conditioned stimulus must accurately
predict the unconditioned stimulus
• Rescorla-Wagner model: states that the strength of
the CS-US association is determined by the extent to
which the unconditioned stimulus is unexpected or
surprising
Learning Involves Cognition
• Other aspects of classical conditioning
consistent with the Rescorla-Wagner model:
– Orienting response: occurs when an animal
encounters a novel stimulus
– Blocking effect: once a conditioned stimulus is
learned, it can prevent the acquisition of a new
conditioned stimulus
– Occasion setter: a stimulus associated with a CS
that acts as a trigger for the CS
Critical Thinking Skill:
Avoiding the Association of Events
with Other Events That Occur
at the Same Time
• People, and apparently other animals, have a
strong need to understand what causes or
predicts events. Their resulting associations
can lead people to cling to superstitions
• What misfortunes could actually occur in the
situations shown on the following slide?
6.2 How Does Operant Conditioning
Differ from Classical Conditioning?
• Define operant conditioning.
• Distinguish between positive reinforcement,
negative reinforcement, positive punishment,
and negative punishment.
• Distinguish between schedules of
reinforcement.
• Identify biological and cognitive factors that
influence operant conditioning.
6.2 How Does Operant Conditioning
Differ from Classical Conditioning?
• Operant (instrumental) conditioning: a learning process in
which the consequences of an action determine the likelihood
that it will be performed in the future
• B. F. Skinner chose the term operant to express the idea that
animals operate on their environments to produce effects.
• Edward Thorndike performed the first reported carefully
controlled experiments in comparative animal psychology
using a puzzle box.
– Law of Effect: Any behavior that leads to a “satisfying state of affairs”
is likely to occur again, and any behavior that leads to an “annoying
state of affairs” is less likely to occur again.
Reinforcement Increases Behavior
• Thirty years after Thorndike, Skinner developed a
more formal learning theory based on the law of
effect
• He objected to the subjective aspects of Thorndike’s
law of effect: States of “satisfaction” are not
observable empirically
• Skinner believed that behavior occurs because it has
been reinforced
– reinforcer: a stimulus that follows a response and
increases the likelihood that the response will be repeated
The Skinner Box
• An operant chamber that allowed repeated
conditioning trials without requiring
interaction from the experimenter
• Contained one lever connected to a food
supply and another connected to a water
supply
Shaping
• Sometimes animals take a long time to perform the
precise desired action. What can be done?
• Shaping: an operant-conditioning technique that
consists of reinforcing behaviors that are increasingly
similar to the desired behavior
– successive approximations: anybehavior that even slightly
resembles the desired behavior
• Suppose you wanted to teach yourself to do
something. Which behavior would you choose, and
how would you go about shaping it?
Reinforcers Can Be Conditioned
• primary reinforcers: satisfy biological needs
such as food or water
• secondary reinforcers: events or objects
established through classical conditioning that
serve as reinforcers but do not satisfy
biological needs, e.g. money or compliments
Reinforcer Potency
• David Premack theorized about how a reinforcer’s
value could be determined
• The key is the amount of time an organism, when
free to do anything, engages in a specific behavior
associated with the reinforcer
• Premack principle: Using a more valued activity can
reinforce the performance of a less valued activity
– How do you think you could use this principle on yourself?
Both Reinforcement and Punishment
Can Be Positive or Negative
• Reinforcement and punishment have the opposite
effects on behavior
– Reinforcement increases a behavior’s probability
– Punishment decreases its probability
• Both reinforcement and punishment can be positive
or negative
• This designation depends on whether something is
given or removed, not on whether any part of the
process is good or bad
Positive and Negative Reinforcement
• Reinforcement — positive or negative —
increases the likelihood of a behavior
– positive reinforcement: the administration of a
stimulus to increase the probability of a behavior’s
being repeated, e.g. a reward
– negative reinforcement: the removal of a stimulus
to increase the probability of a behavior’s being
repeated, e.g. requiring a rat to press a lever to
turn off a shock
Positive and Negative Punishment
• Punishment reduces the probability that a
behavior will recur
– positive punishment: the administration of a
stimulus to decrease the probability of a
behavior’s recurring, e.g. receiving a ticket for
speeding
– negative punishment: the removal of a stimulus
to decrease the probability of a behavior’s
recurring, e.g. taking away driving privileges for
bad behavior
Effectiveness of Parental Punishment
• For punishment to be effective, it must be reasonable,
unpleasant, and applied immediately so that the relationship
between the unwanted behavior and the punishment is clear
– How might this go wrong?
• Punishment often fails to offset the reinforcing aspects of the
undesired behavior
• Research indicates that physical punishment is often
ineffective, compared with grounding and time-outs
• Many psychologists believe that positive reinforcement is the
most effective way of increasing desired behaviors while
encouraging positive parent/child bonding
Operant Conditioning is Influenced by
Schedules of Reinforcement
• How often should reinforcers be given?
• continuous reinforcement: a type of learning
in which behavior is reinforced each time it
occurs
• partial reinforcement: a type of learning
inwhich behavior is reinforced intermittently
• Partial reinforcement’s effect on conditioning
depends on the reinforcement schedule
Ratio and Interval Schedules
• Partial reinforcement can be administered
according to either the number of behavioral
responses or the passage of time
– ratio schedule: Reinforcement is based on the
number of times the behavior occurs
– interval schedule: Reinforcement is provided after
a specific unit of time
• Ratio reinforcement generally leads to greater
responding than does interval reinforcement
Fixed and Variable Schedules
• Partial reinforcement can also be given on a
fixed schedule or a variable schedule
– fixed schedule: Reinforcement is provided after a
specific number of occurrences or after a specific
amount of time
– variable schedule: Reinforcement is provided at
different rates or at different times
Behavioral Persistence
• Continuous reinforcement is highly effective for
teaching a behavior. If the reinforcement is stopped,
however, the behavior extinguishes quickly
• variable-ratio schedule: persistent behavior thatonly
sometimesresults in reward
• partial-reinforcement extinction effect: Behavior is
more persistent under partial reinforcement than
under continuous reinforcement
– Can this explain why gambling is so addictive?
Psychology: Knowledge You Can Use—
Can Behavior Modification Help Me
Stick with an Exercise Program?
• Consider these steps:
– Identify a behavior you wish to change
– Set goals
– Monitor your behavior
– Select a reinforcer and decide on a reinforcement
schedule
– Reinforce the desired behavior
– Modify your goals, reinforcements, or
reinforcement schedules, as needed
Behavior Modification
• Behavior modification: the use of operantconditioning techniques to eliminate
unwanted behaviors and replace them
withdesirable ones
• Token economies operate on the principle of
secondary reinforcement. Tokens are earned
for completing tasks and lost for bad behavior.
Tokens can later be traded for objects or
privileges
Biology and Cognition Influence
Operant Conditioning
• Behaviorists such as Skinner believed that all
behavior could be explained by
straightforward conditioning principles
• However, a great deal about behavior remains
unexplained
• Biology constrains learning, and reinforcement
does not always have to be present for
learning to take place
Biological Constraints
• Animals have a hard time learning behaviors that run
counter to their evolutionary adaptation
• Marian and Keller Breland used operant-conditioning
techniques to train animals but ran into difficulty
when the tasks were incompatible with innate
adaptive behaviors
• Conditioning is most effective when the association
between the response and the reinforcement is
similar to the animal’s built-in predispositions
– For example, Bolles argued that animals have built-in
defense reactions to threatening stimuli
“Talent vs. Practice”
Is talent something you’re born with, or can practice really
make perfect? Experts on expertise—who’ve studied the
minds of experts in fields from sports to medicine—have the
answer. As this ScienCentral News video explains, they’re
applying it to life or death situations.
Acquisition/Performance Distinction
• Tolman argued that learning can take place without
reinforcement
– latent learning: takes place in the absence of reinforcement
– insight learning: A solution suddenly emerges after either a period of
inaction or of contemplation
• Tolman’s studies involved rats running through mazes
– cognitive map: a visual/spatial mental representation of an
environment
• The presence of reinforcement does not adequately explain
insight learning, but it helps determine whether the behavior
is subsequently repeated
6.3 Does Watching Others
Affect Learning?
• Describe the concept of the meme.
• Define observational learning.
• Generate examples of observational learning,
modeling, and vicarious learning.
• Discuss contemporary evidence regarding the
role of mirror neurons in learning.
6.3 Does Watching Others
Affect Learning?
• Teaching someone to perform a complex task
requires more than reinforcing arbitrary
correct behaviors.
• We learn many behaviors, including attitudes,
through observation.
Learning Can Be Passed On
through Cultural Transmission
• Meme: a unit of knowledge transmitted
within a culture
• Memes can be conditioned through
association or reinforcement, but are often
learned by watching the behavior of other
people
• Through social learning, some behaviors are
passed along from one generation to the next
Learning Can Occur
through Observation and Imitation
• Observational learning: the acquisition or
modification of a behavior after exposure to at
least one performance of that behavior
• Observational learning is a powerful adaptive
tool for humans and other animals
– Can you think of some examples of observational
learning in animals?
Bandura’s Observational Studies
• Bandura’s studies suggest that exposing
children to violence may encourage them to
act aggressively
Media and Violence
• The extent to which media violence impacts
aggressive behavior in children is debated
• Some studies demonstrate desensitization to
violence after exposure to violent video games
• However, it is difficult to draw the line between
“playful” and “aggressive” behaviors in children
• There may be extraneous variables that affect both
TV habits AND violent tendencies
• Based on what you have just learned, how might
media impact behavior?
“Violent Games”
As the content of video games becomes more and more violent,
researchers are debating whether virtual violence can lead kids
to the real thing. This ScienCentral News video has more.
Social Learning of Fear
• Susan Mineka noticed that lab-reared
monkeys were not afraid of snakes the way
monkeys in the wild are
• Her research demonstrated that animals’ fears
can be learned through observation
• Social forces play a role in fear-learning in
humans too
Demonstration and Imitation
• modeling: the imitation of behavior through
observational learning
• Modeling is effective only if the observer is
physically capable of imitating the behavior
• Imitation is much less common in nonhuman
animals than in humans
• Adolescents who associate smoking with
admirable figures are more likely to begin
smoking
Vicarious Reinforcement
• vicarious learning: learning the consequences
of an action by watching others being
rewarded or punished for performing the
action
• A key distinction in learning is between the
acquisition of a behavior and its performance
• In other words, learning a behavior does not
necessarily lead to performing that behavior
Mirror Neurons
• What happens in the brain during imitation learning?
• mirror neurons: neurons that are activated when one
observes another individual engaging in an action and when
one performs the action that was observed
• May serve as the basis of imitation learning, but the firing of
mirror neurons does not always lead to imitative behavior
• May be the neural basis for empathy and play a role in
humans’ ability to communicate through language
• Debatable if brain activity reflects prior learning rather than
imitation
6.4 What Is the Biological
Basis of Learning?
• Discuss the role of dopamine and the nucleus
accumbens in the experience of
reinforcement.
• Define habituation, sensitization, and longterm potentiation.
• Describe the neural basis of habituation,
sensitization, long-term potentiation, and fear
conditioning.
6.4 What Is the Biological
Basis of Learning?
• When animals and people learn, what
changes in the brain?
• Researchers are rapidly identifying the
neurophysiological basis of learning.
• Similar brain activity occurs for most
rewarding experiences.
Dopamine Activity
Underlies Reinforcement
• Positive reinforcement works in two ways:
– provides the subjective experience of pleasure
– increases wanting for the object or event that
produced the reward
• The neurotransmitter dopamine is involved in
addictive behavior and plays an important role
in reinforcement
Pleasure Centers
• intracranial self-stimulation: self-administered shock to
pleasure centers of the brain
• Starving rats prefer ICSS to food over 80 percent of the time
• The neural mechanisms underlying both ICSS and natural
reward appear to use the same neurotransmitter: dopamine
• This suggests dopamine serves as the neurochemical basis of
positive reinforcement in operant conditioning
• Interfering with dopamine eliminates self-stimulation as well
as naturally motivated behaviors
Dopamine and Reward
• The nucleus accumbens is a subcortical brain region that is part of the
limbic system
• More dopamine is released under deprived conditions than under
nondeprivedconditions
– Do you have the intuition that food tastes better when you are
hungry?
• In operant conditioning, dopamine release sets the value of a reinforcer,
and blocking dopamine decreases reinforcement
– Dopamine blockers are can also help people with Tourette’s
syndrome regulate their involuntary body movements
• Robinson and Berridge (1993) introduced an important distinction
between the wanting and liking aspects of reward
– For example, a smoker may want a cigarette but not especially
enjoy it
• Dopamine appears to be especially important in wanting a reward
Secondary Reinforcers
Also Rely on Dopamine
• Natural reinforcers appear to signal dopamine
reward directly
• Secondary reinforcers at first fail to trigger
dopamine release but may do so readily after
they are paired with unconditioned stimuli
• Money is a secondary reinforcer that activates
dopamine systems
Habituation and Sensitization
Are Simple Models of Learning
• Kandel’s work on the aplysia has shown that
habituation and sensitization, two simple
forms of learning, occur through alteration in
neurotransmitter release
– habituation: a decrease in behavioral response
after repeated exposure to a nonthreatening
stimulus
– sensitization: an increase in behavioral response
after exposure to a threatening stimulus
Long Term Potential Is a Candidate
for the Neural Basis of Learning
• long-term potentiation (LTP): the strengthening of a
synaptic connection, making the postsynaptic
neurons more easily activated
• Through LTP, intense stimulation of neurons
strengthens synapses, increasing the likelihood that
one neuron’s activation will increase the firing of
other neurons
• LTP effects are most easily observed in brain sites
known to be involved in learning and memory, such
as the hippocampus
• Research has also supported Hebb’s rule that
neurons that fire together wire together
“Smart Mice”
They didn’t mean to create smart mice, but that’s what
happened when neurologists genetically altered mice to lack a
certain brain protein. As this ScienCentral News story explains,
the chance discovery could lead to new drugs to treat learning
and memory disorders.
LTP and the NMDA Receptor
• LTP occurs when NMDA receptors are
stimulated by nearby neurons
• Joseph Tsien modified genes in mice to make
the genes’ NMDA receptors more efficient
• Tsein’s “Doogie Mice” learned novel tasks
quicker and showed increased fear
conditioning
Fear Conditioning
• LTP in the amygdala appears to play a role in fear
conditioning
• Joseph LeDoux’s research suggests that fear
conditioning might produce long-lasting learning
through the induction of LTP
• Heightened activity in the amygdala, when subjects
watched another person’s distress, suggests that
similar mechanisms are involved in conditioned and
observational fear learning
“Wiring the Brain”
Interested in lifelong learning? Here’s some good news. As this
ScienCentral News video reports, brain researchers have
uncovered one mechanism that controls how out brains make
new connections.