Operant Conditioning
• Action results in consequence
– Decision
• B.F. Skinner
• Three term contingency
– Stimulus - response - outcome
• Outcomes
– Positive/negative; reinforcement/punishment
Differences from Classical
• Usually assumed to be under “conscious”
control
• Operant conditioned after the behaviour
– Outcome feeds back to alter response
Probabilities
• Hierarchy of behaviours (actions)
– Probability
• Operant conditioning changes events and/or
consequences
• Results in adjustment of probability
hierarchy
Shaping
• Directed learning
• Behavioural outcome more certain
– Select a specific response to occur in a specific
way
• Gradual process
• Chaining
– Forward and backward
Reinforcement Schedules
•
•
•
•
Fixed ratio
Fixed interval
Variable ratio
Variable interval
Reinforcement Consistency
• Continuous schedules
• Intermittent schedules
• Response-reinforcer?
– Technically, only FR-1 is continuous
• Systematic reinforcer
– Any fixed schedule (FR or FI)
Extinction
• Response - outcome pattern disrupted
• Easiest for:
– Continuous reinforcement/punishment
– Low schedules
• Variable ratio schedules hardest to
extinguish
Reinforcers
•
•
•
•
•
Primary
Secondary
Speed of learning
Extinction
Money
Delay
• Immediate reinforcement
• Delayed reinforcement
• Generally, delayed harder to condition
– Difference with well conditioned system
Changing Schedules
• Cost of response
• Contingency
– Rate of reinforcement
• Modification
– Decrease
– Increase
• Delay
Applications
• Discriminative stimulus
– “Blue-light special,” coloured sale tags, logos
(if previous positive experience with product)
– Christmas music in October
• Positive reinforcement
– Give-aways, purchase points, Canadian Tire
money, parking lot barbeques
• Negative reinforcement
– Purchase to avoid pushy sales pitch
• VR schedule
– Lotteries, door prizes, etc.
• Shaping
– Free trial periods, leading signs/displays (get
person into store area)
• Punishment
– Unusual in advertising/marketing
– Commercials showing customer who suffers
from not using product (observational learning)
Ecological Design
• Structuring the environment
• Facilitation of particular behaviour(s)
– Increase/decrease probability of response
• Store layout, purchase locations, noises,
odors, lighting
• A type of shaping of a response
Behaviour Modification
• Application of operant theory to change
behaviour
• Primary application of operant principles
• Skinner’s behaviour analyst techniques
Behav. Mod. in Marketing
• Role of marketing as influencing,
modifying, and controlling consumer
behaviour to achieve purchasing objectives
• An applied field
– Not aimed at developing theory, but applying
theory
• Observable behaviour
– No inferred behavioural constructs
Economic Psychology
• Integration of psychology and economic
analysis
• Marketing
– Not a discipline
– An application area for the social sciences and
other disciplines
• EcPsyc offers detailed analysis of
consumer-firm interactions
Behavioural Perspective Model
• Gordon R. Foxall
• Operant behaviourist paradigm
• Modern marketing firms
– Embedded in networks of marketing relationships
– Extra-firm environment (e.g., consumers) drive
marketing behaviour
– Reinforcement/punishment shift firm’s behaviour
• Applies also to behaviour of individuals comprising the
firm (e.g., employees, owners, shareholders, etc.)
A Firm’s Purpose
• To make marketing relationships more
economic
– Production and selling are independent of firms
• Don’t need firms to do these
– Creation and maintenance of marketing is what
firms do
Uhm… So?
• Operant conditioning theory and firms
• Economic behaviour is instrumentally
conditioned
• Behaviour that operates on the environment
to produce consequences changes the future
rates of behaviour
• Reinforcement/punishment shifts economic
(market) factors
Consumer Behaviour
• Economic purchasing and consumption
activities
• Basic three-term contingency applies
– Stimulus - response - outcome
• Plus, consumer behavioural setting and
learning history
Behav. Persp. Model and
Consumer’s Choice
• Consumer choice reduces aversive
consequences of facing multiple equivalent
options
Model
Utilitarian
reinforcement
Consumer
behaviour
setting
Consumer
behaviour
Consumer’s
learning
history
Aversive
consequences
Informational
reinforcement
Marketing Management in BPM
• Influence two factors
• Consumer behaviour settings
– Social, physical, temporal, and regulatory
discriminative stimuli
• Utilitarian and information reinforcers
– Actual outcome and knowledge gained
Managing Reinforcers
• Three ways
• Enhancing effectiveness of reinforcers
• Controlling the schedules of reinforcer
delivery
• Increasing the quantity or quality of
reinforcers
Complexities
• Multiple systems operating simultaneously
• Is operant conditioning separable from
classical?
• Do stimuli fulfill role of CS, SD, or both?
Role of Operant Reinforcer in
Classical Conditioning
• In classical conditioning
– US presented regardless of CR
– Defining feature
– But, operant reinforcement can slip in
• Operant reinforcement via
– 1. Reinforcing CR directly
• e.g., food (US) coming after CR
– 2. CR increases “value” of US
• e.g., salivation (CR) makes swallowing food (US)
easier
Omission Control Procedure
• US presentation depends on occurrence of
CR
– CS presented; if no CR, US follows
– CS presented; if CR, no US follows
• Therefore, US can’t operantly reinforce CR
Omission Control
Trial with a CR
% of CRs
CS
US
CR
Trial without a CR
CS
US
CR
Blocks of sessions
Omission control
Standard classical conditioning
Conclusion
• Can have classical conditioning without
operant reinforcement
• But what about classical conditioning in
operant conditioning?
Associative Structure in Operant
Conditioning
• Basic form of association
– S-R
– S-O
• Pavlovian processes
Outcome
Stimulus
Instrumental
response
• Can keep instrumental reinforcement
out of classical conditioning, but not
vise versa
S-R, S-O, rg-sg
• Thorndike’s Law of Effect
– Focus on S-R association
• Hull and Spence
– Law of Effect plus a classical conditioning
process
• rg-sg
– Fractional anticipatory goal response; sensory
feedback
Fractional Anticipatory Goal
Response
• SD influences rg-sg (expectancy of reward
from classical conditioning) through
sensory substitution-like process
• Motivation
Timeline
Stimulus
rg
sg
Response
Outcome
Prediction
• According to rg-sg
–
–
–
–
CR occurs before operant response
But, not always true
e.g., lever pressing and salivation
CR should occur before operant, but it doesn’t
Central Emotional State
• Classical conditioning in operant
conditioning
• Not for learning response
• For CES (Central Emotional State)
• CES --> motivation, “mood”
Modern Two-Process Theory
• Classical in operant conditioning
• Neutral stimulus --> elicit motivation (CES)
• CES elicited by CS corresponds to US
– CES a characteristic of CNS = “mood”
• CES doesn’t produce only one response
– e.g., anger --> multiple responses
• CES conditioned during ordinary operant training
– CES conditioned to situational cues or discriminative stimulus
– CES motivates operant behaviour
Prediction
• Rate of instrumental response will be
modified by presentation of CS
• Consider
– In operant conditioning, CES develops to
motivate operant response
– CS from classical conditioning also elicits CES
– Therefore, giving CS during operant
conditioning will alter CES that
motivates/maintains operant response
Conditioned Emotional Response
• Suppression ratio
• CES elicited by CS --> decrease response
“Explicit” Predictions
• Emotional states
US
CS
CS+
CS-
Appetitive
(e.g., food)
Hope
Disappointment
Aversive
(e.g., shock)
Fear
Relief
• Behavioural predictions
Instrumental schedule
Aversive US
CS+(fear)
CS-(relief)
Positive reinforcement
Negative reinforcement
decrease
increase
increase
decrease