Reflex, Taxis, and Instinct Lecture 7 1 Cognitive Functions of the Mind • Mediate Adaptive Behavior – Interactions Between Person, World • Form Internal Representations of the World – Perception, Memory • Reflect on this Knowledge – Reasoning, Problem-Solving • Use Knowledge to Guide Behavior – , Decision, Choice, Action • Communicate Knowledge to Others – Language 2 Traditional Associationist View of Learning after Aristotle, De Memoria (250 BCE) and Hume, Treatise of Human Nature (1739) The Mind Forms Associations Between... • Stimuli – Events in the Environment – Their Correlates and Consequences • Responses – Organism’s Behaviors – Their Correlates and Consequences 3 Innate Responses to Stimulation • Reflexes – Involve Individual Muscles • Taxes – Involve Entire Body • Instincts (Fixed Action Patterns) – Very Discriminating 4 Reflexes after Descartes, De Homine (c. 1633/1662) • Automatic Response to Adequate Stimulus – Patellar Reflex – Eyeblink Reflex – All “Spinal” Reflexes www.osceskills.com • Involve Single Muscles • Mediated by Spinal, Cranial Nerves – No Involvement of “Higher” Cortical Centers 5 The Decorticate Pigeon as Reflex Machine Flourens (19th c.) • • • • • Preserved Righting Stepping Flying Swallowing Irritation • • • • Abolished Flight from Irritation Avoidance Voluntary Action Emotionality 6 Reflexes in the Human Infant Teitelbaum (1967) • Reflexes of Approach – Rooting – Grasping – Plantarflexion • Reflexes of Avoidance – Eye Closure – Mouth Closure – Dorsiflexion • Babinski Reflex • Stepping 7 Advantages and Limitations of Reflexes • Respond to External Stimulation – No Need for Learning • Limited Repertoire • No Control by Goals, Intentions 8 Taxes • Gross Orientation Responses – Entire Skeletal Musculature • Positive vs. Negative • Phototaxes • Geotaxes www.reddit.com/r/biology Kangaroos.org www.geog.ucsb.edu 9 Instincts (Fixed Action Patterns) Tinbergen (1951) • Complex, Stereotyped Action Patterns – Rigidly Organized • Respond to Releasing Stimulus • Innate – Not Modified by Learning • Species-Specific – Universal within Species Ethology 10 Imprinting in Greylag Geese Lorenz (1935) • “First” Moving Object • Indiscriminate • Critical Period 11 Alarm Reaction in Birds Tinbergen (1951) • Distress Call 12 Food-Begging in Herring-Gull Chicks Tinbergen (1951) 13 Releasing Stimulus • Imprinting – Moving Object • Alarm Reaction – “Hawklike” Silhouette • Food-Begging – Contrasting Spot – Chick’s Peck 14 The Zig-Zag Dance of the Stickleback Tinbergen (1951) • Red-Bellied Male – Head-Down Threat Posture • Female with Swollen Abdomen – Head-Up Receptive Posture • Zig-Zag Dance • Female Lays Eggs – Stimulated in Hindquarters • Male Tends Nest 15 Evolution of Behavior • Evolution Shapes Behavior as well as Body Morphology • Adaptation to Environmental Niche • Change in Species Characteristics Over Evolutionary Time • How Can Individuals and Species Adjust to New Circumstances? 16 Positive Phototaxis in Sea Turtles 17 Limitations on Innate Responses to Stimulation • Stimulus Must be Physically Present in Current Environment • Cannot Respond to New Stimulus • Cannot Acquire New Responses • Little Opportunity for Trial-and-Error Learning How Can Individuals Modify Their 18 Responses to Environmental Stimulation? Learning • Relatively Permanent... • Change in Behavior... • Resulting from Experience. – Not Drugs – Not Injury – Not Maturation 19 Pavlov’s Apparatus Classical Conditioning of “Psychic” Reflexes 20 Classical Conditioning Phase 1: Bell, Food Phase 2: Bell ==> Food Phase 3: Bell Alone 21 Basic Vocabulary of Classical Conditioning • • • • Unconditioned Stimulus Unconditioned Response Conditioned Stimulus Conditioned Response US UR CS CR 22 Classical and Instrumental Conditioning Lecture 8 1 Basic Procedure for Classical Conditioning CS (Bell) CR (Salivation) US (Meat Powder) UR (Salivation) 2 Major Phenomena of Classical Conditioning • Acquisition – CR to CS Reinforced by US – Response Gains Strength • Magnitude of CR • Probability of CR 3 Acquisition: Trial 1 CS US CR Bell Food Drops of Saliva Time 4 Acquisition: Trial 5 CS US CR Bell Food Drops of Saliva Time 5 Acquisition: Trial 10 CS US CR Bell Food Drops of Saliva Time 6 Acquisition of a Conditioned Response CS ==> US Probability of CR 1 0.8 Positive Acceleration 0.6 Negative Acceleration 0.4 0.2 The Learning Curve Forms an Ogive 0 1 2 3 4 5 6 7 8 9 10 11 12 Trials 7 Major Phenomena of Classical Conditioning • Acquisition • Extinction – No Reinforcement – Response Loses Strength 8 Extinction: Trial 1 CS US CR Bell Food Drops of Saliva Time 9 Extinction: Trial 5 CS US CR Bell Food Drops of Saliva Time 10 Extinction: Trial 10 CS US CR Bell Food Drops of Saliva Time 11 Extinction of a Conditioned Response CS ==> No US Probability of CR 1 0.8 0.6 0.4 0.2 0 1 2 3 4 5 6 7 8 9 10 11 12 Trials 12 Major Phenomena of Classical Conditioning • Acquisition • Extinction • Spontaneous Recovery – Rest After Extinction – Retest CS alone 13 Spontaneous Recovery After Extinction CS ==> No US Probability of CR 1 Acquisition Extinction 0.8 Spontaneous Recovery 0.6 0.4 Further Extinction 0.2 (Rest) 0 Trials 14 Re-Acquisition of Extinguished Response CS ==> US Acquisition Extinction Re-Acquisition Probability of CR 1 0.8 Savings in Relearning 0.6 0.4 Spontaneous Recovery 0.2 (Rest) 0 Trials 15 Extinction Below Zero • Extinction Trials Continued After CR Disappears Entirely • Reduced Spontaneous Recovery • Less Savings in Relearning – Slower Reacquisition 16 Major Phenomena of Classical Conditioning • Acquisition • Extinction – Extinction Below Zero • Spontaneous Recovery • Re-Acquisition – CS Reinforced by US – Savings in Relearning 17 Implications of Savings and Spontaneous Recovery • Conditioned Response Not “Lost” • Rather, Inhibited or Suppressed – In Line with Changing Circumstances • Conditioned Stimulus No Longer Reinforced • CR Can be Disinhibited – In Line with Changing Circumstances • CS Reinforced Once Again 18 Major Phenomena of Classical Conditioning • • • • • Acquisition Extinction Spontaneous Recovery Re-Acquisition Generalization – CS0 vs. CS1…CSn – Generalization Gradient 19 Probability of CR The Generalization Gradient 250cps 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 200cps 300cps Lower Higher 150cps -5 -4 350cps Original CS -3 -2 -1 0 1 2 3 4 5 Test Stimulus 20 Major Phenomena of Classical Conditioning • • • • • • Acquisition Extinction Spontaneous Recovery Re-Acquisition Generalization Discrimination – CS+ (Reinforced) – CS- (Unreinforced) 21 Discrimination Learning CS+ ==> US CS- ==> No US Probability of CR 1 250 cps 0.8 0.6 CS+ 0.4 CS- 200 cps 0.2 0 1 2 3 4 5 6 7 8 9 10 11 12 Trials 22 Review of Classical Conditioning • Vocabulary – – – – Unconditioned Stimulus Unconditioned Response Conditioned Stimulus Conditioned Response • Phenomena – Acquisition • Reinforcement – – – – Extinction Spontaneous Recovery Savings in Relearning Generalization • Generalization Gradient – Discrimination 23 Sensory Preconditioning 1 2 3 CS1 Light CS2 Bell CS2 Bell US Food CR Saliva CS1 Light CR Saliva24 Higher-Order Conditioning 1 CS1 Bell US Food CR Saliva CS2 Light CS1 Bell 2 3 CS2 Light CR Saliva25 Significance of Classical Conditioning • Extends Control of Reflexes to Other Environmental Events – Associations between Events • Ubiquitous (Nervous System) • Pavlov: All Learning is Classical Conditioning (?) • Laws of Classical Conditioning are the Laws of Emotional Life 26 Thorndike’s Puzzle Box 27 Thorndike’s Results 28 Thorndike’s Laws of Learning Law of Readiness Law of Effect Law of Exercise 29 Instrumental Conditioning (Operant Conditioning) • Learn Adaptive Behavior – Through Experience of Success, Failure • Organism Operates on Environment – Behavior Changes Environment • Behavior Instrumental – Obtains Desired State of Affairs • Associations between Behaviors and Outcomes 30 B.F. Skinner’s Operant Chamber 31 Instrumental Conditioning Procedure • • • • Phase 1: Baseline Behavior Phase 2: Acquisition Phase Phase 3: Discrimination Learning Phase 4: Extinction 32 Vocabulary of Instrumental Conditioning • Conditioned Response (No URs) • Conditioned Stimulus (No USs) • Reinforcement – Positive – Negative (Not Punishment) • • • • Acquisition Extinction Generalization Discrimination 33 Schedules of Reinforcement • Continuous • Partial • Intermittent – Fixed Ratio (FR) – Variable Ratio (VR) – Fixed Interval (FI) – Variable Interval (VI) • Differential Reinforcement – Of Low Rates (DRL) – Of High Rates (DRH) 34 Intermittent Reinforcement 35 The Matching Law Herrnstein (1970) • Concurrent VI Schedules – Give Organism a Choice • Key A: VI3 • Key B: VI1 • Response Rate is Proportional to the Frequency of Reinforcement – Also Magnitude, Delay of Reinforcement • Basic Principle of Microeconomics – Supply and Demand • Relative Value of Reinforcers 36 Significance of Instrumental Conditioning • Voluntary Behaviors Come Under Control of Environmental Events – Behavior-Outcome Associations • Ubiquitous (Vertebrates) • Thorndike, Skinner: All Learning is Instrumental/Operant Conditioning (?) • Laws of Instrumental Conditioning Are the Laws of Adaptive Behavior – Habits – Incentives 37 What is Learned? Lecture 9 1 Classical and Instrumental Conditioning Compared Classical Instrumental • Reinforcement Not • Reinforcement Contingent on Behavior Contingent on Behavior • Behavior Elicited by US • Behavior Emitted by Organism • Voluntary Responses • Involuntary Response (“Spontaneous”) (Reflex) • Many Conditionable • Few Conditionable Behaviors Behaviors 2 Avoidance Learning Solomon & Wynne (1953) • Dog Placed in One Side of Apparatus • Overhead Lights Deliver CS • Floor Grid Delivers US – After CS-US Interval • Vault Barrier – Escape After US Onset – Avoidance Before US Onset 3 Two-Factor Theory of Avoidance Learning Mowrer (1947); critiqued by Seligman & Johnston (1973) • Light ==> Shock – Respond During Shock ==> Escape – Respond Prior to Shock ==> Avoidance • Classical Conditioning – Anticipatory Fear Conditioned to Light • Instrumental Conditioning – Reinforce Escape/Avoidance • Cessation of Shock US • Cessation of Light CS 4 The Stimulus-Response Theory of Learning • Association between Stimulus and Response – Pavlov: CS = Bell; CR = Salivation – Thorndike: CS = Puzzle Box; CR = Paddle Press – Skinner: CS = (Illuminated) Key; CR = Keypeck • Reinforcement – Pavlov: US = Meat Powder – Thorndike: Reward = Escape – Skinner: Reinforcement = Food Pellet 5 Assumptions of S-R Learning Theory • Association by Contiguity – Co-Occurrence in Space, Time • Arbitrariness (Equipotentiality) – Any Stimulus, Any Response • Empty Organism – Organism as “Black Box” Collecting Ss, Rs • Passive Organism – Metaphor of “Conditioning” 6 Taste-Aversion Learning (Bait Shyness) Garcia & Koelling (1966) • Compound CS – “Bright, Noisy, Sweet” Water • US – Foot Shock (Immediate Pain) – X-Rays (Delayed Nausea) • Avoidance Test of Conditioning – Choose Water Source • Bright, Noisy Water • Sweet Water 7 Taste-Aversion Learning Garcia & Koelling (1966) All Subjects CS1 Light CS2 CS3 Noise Taste Group 1 US1 Shock UR1 Group 2 US2 UR2 X-Ray 8 Preference Test of Conditioned Fear (Avoidance of Water Source) Garcia & Koelling (1966) CR (Licks per Mnute) 350 300 250 CS 200 Sweet 150 Bright/Noisy 100 50 0 Shock X-Ray US 9 Implications for S-R Learning Theory • Arbitrariness – Taste-Nausea, Sight/Sound-Shock • Empty Organism – Internal Structure Shaped by Evolution • Association by Contiguity – CS, CR Distant in Space, Time • Law of Exercise – One-Trial Taste-Aversion Learning 10 Species-Specific Defense Reactions Bolles (1970) • Escape/Avoidance Learning in Pigeons – Easy: Flap Wings, Stretch necks – Impossible: Key Peck • Escape/Avoidance Learning in Rats – Easy: Jump Up, Run – Hard: Lever Press • Avoidance Learning Capitalizes on Species-Specific Repertoire of Defensive Reactions – Built In by Evolution 11 The Preparedness Principle (Belongingness) Seligman (1970); Rozin & Kalat (1971) By Virtue of Its Evolutionary History, Each Species is Predisposed to Learn Certain Associations • Prepared • Unprepared • Contraprepared 12 Constraints on Learning • Biological – Evolutionary History • Cognitive – Internal Representation of CS CR 13 “Standard Paradigm” for Classical Conditioning CS US CR Bell Food Drops of Saliva Time 14 Delay Conditioning CS US CR Bell Food Drops of Saliva Time 15 Trace Conditioning CS US CR Bell Food Drops of Saliva Time 16 Simultaneous Conditioning CS US CR Bell Food Drops of Saliva Time 17 Backwards Conditioning CS US CR Bell Food Drops of Saliva Time 18 Conditioned Inhibition in Backwards Conditioning • Fear Conditioning – Tone ==> Shock – CR = Heart Rate Acceleration • Physiological Index of Fear • Standard Paradigm – HR Acceleration During Tone • Conditioned Fear Response • Backward Paradigm – HR Deceleration During Tone • Inhibition of Fear Response 19 Contiguity vs. Contingency in Conditioning Rescorla (1967, 1988) • Contiguity – CS Co-Occurs with US • Contingency – CS Predicts US • Standard Paradigm – CS, US both Contingent and Contiguous • Delay, Trace Conditioning – CS, US Contingent but Not Contiguous • Simultaneous Conditioning – CS, US Contiguous but not Contingent • Backwards Conditioning, Extinction (Below Zero) – CS Predicts Absence of US 20 Informational Value of the CS • When the US is Contingent on the CS, then the CS Provides Information About the US • Conditioning Occurs Because the CS Provides Information about US • Conditioning Does Not Occur When the CS is Not Informative What Happens When the Information in the CS is Redundant? 21 Conditioned Emotional Responses • Tone CS Footshock US • CR: Heart Rate Acceleration • Paradigmatic Variations – Standard Paradigm – Delay, Trace Paradigms – Simultaneous Paradigm – Backwards Paradigm • Safety Signal 22 The Blocking Experiment (1) Kamin (1969) Initial Conditioning with Compound CS CS1 Noise CS2 Light US Shock Test Conditioning with Light CS Alone CS2 Light CR Fear Test 23 The Blocking Experiment (2) Kamin (1969) Phase 1: Conditioning with Noise CS1 (1) Phase 2: Add Light CS2 Simultaneous with CS1 (2) CS1 Noise US Shock CS1 Noise CS2 Light US Shock 1 2 Test CS2 CR Light Fear24 The Blocking Effect Kamin (1969) CR Magnitude Test Response to Light 100 90 80 70 60 50 40 30 20 10 0 Noise Alone Compound Alone Noise then Compound Sequence of Conditioning Trials 25 “Unblocking” the Blocking Experiment Phase 1: Conditioning with Simple CS Phase 2: Add Light CS2 Preceding CS1 CS1 Noise US Shock CS1 Noise CS2 Light US Shock 1 2 Test CS2 CR Light Fear26 Predictability, Surprise, and Conditioning Kamin (1969) • Conditioning Only Occurs When the US Surprises the Organism • Organism Searches Environment for Predictors of US • Irrelevant, Redundant Stimuli are Ignored • Classical Conditioning Involves the Formation of Expectations – CS predicts US 27 Implications for S-R Learning Theory • Association by Contiguity – Association by Contingency • Empty Organism – Expectations, Surprise • Passive Organism – Actively Engaged in Predicting Events 28 A Cognitive View of Learning Lecture 10 1 Cognition and Conditioning • Conditioning Occurs When the US Surprises the Organism • Conditioning Accrues to CSs that Provide Information About the US • In Classical Conditioning, the Organism Learns to Predict Environmental Events • What about Instrumental Conditioning? 2 Two-Factor Theory of Avoidance Learning Mowrer (1947) • Tone CS ==> Shock US – Respond During Shock ==> Escape – Respond Prior to Shock ==> Avoidance • Classical Conditioning – Fear Conditioned to Tone • Instrumental Conditioning – Escape Reinforced by Offset of Shock • Reduction in Pain – Avoidance Reinforced by Offset of Tone • Reduction in Fear 3 Learned Helplessness e.g., Seligman & Maier (1967); Seligman, Maier, & Solomon (1971) • Prior Fear Conditioning Should Facilitate Avoidance Learning – Organism Already Fears CS – Only Has to Learn Avoidance Response • Phase 1: Condition Fear to Tone • Phase 2: Avoidance Learning • But Prior Fear Conditioning Retards Acquisition of Avoidance Response 4 Escapable vs. Inescapable Shock Seligman & Maier (1967) • Avoidance Learning in Shuttlebox • Pretreatment: 64 Signalled Shocks – Escape Shock by Pressing Paddle with Snout – Yoked Controls • Receive Same Amount of Shock as Escape Group • No Experience of Control Over Shock – Standard Controls • Receive No Shocks 5 Learned Helplessness Response Latency (secs) Seligman & Maier (1967) 60 72.5% 50 Escape Failures 40 30 26.3% 22.5% Controls Escape 20 10 0 Yoked Pretreatment 6 Controllability and Instrumental Conditioning • Learned Helplessness – Shock is Inescapable, Unavoidable – Negative Expectations of Control • Generalize to New Learning Environment • Instrumental Conditioning – Organism Learning to Control Environment • Develops Expectations Concerning Control 7 Prediction, Control, and Conditioning • Classical Conditioning – Learning to Predict Environmental Events – Conditional Probability: p(Event | Signal) • Instrumental Conditioning – Learning to Control Environmental Events – Conditional Probability: p(Event | Behavior) 8 Role of Reinforcement in Learning • Corollary to S-R Learning Theory – Conditioned Response – Reinforced in Presence of Conditioned Stimulus • Classical Conditioning – CS Reinforced by Experience of US • Instrumental Conditioning – CR to CS Reinforced by Reward • Yields Thorndike’s Law of Effect • Skinner: Reinforcement is Any Stimulus that Increases the Probability of the Conditioned Response to the CS 9 Classical Conditioning in the Absence of Reinforcement • Sensory Preconditioning – CS1 CS2 – CS1 Predicts US by Transitivity • Higher-Order Conditioning – CS2 CS1 – CS2 Predicts US by Transitivity 10 Maze Learning in Rats Tolman & Honzik (1930); Tolman (1932) 11 Latent Learning Tolman & Honzik (1930) 2 1 3 Hilgard & Bower 12 Curiosity in Rhesus Monkeys Harlow (1953); Harlow et al. (1956) • • • • “Puzzle Lock” Food Reward Hunger Intrinsic vs. Extrinsic Motivation 13 What is Learned in Learning? • Not Associations between Stimuli, Responses • Expectations – Outcomes of Events – Outcomes of Behaviors • Classical Conditioning – Predictability of Environmental Events • Instrumental Conditioning – Controllability of Environmental Events 14 Cognitive Reinterpretation of Learning • Relatively Permanent... • Change in Behavior Knowledge... – Predictability – Controllability • Resulting from Experience. – Not Drugs – Not Injury – Not Maturation 15 Observational Learning • Direct versus Vicarious Experience – Conditioned Stimuli – Unconditioned Stimuli – Reinforcements 16 Observational Fear Conditioning Mineka et al. (1984) • Snake Fear in Rhesus Monkeys – It’s Adaptive: But Is It Innate? • Wild-Reared vs. Lab-Reared – Snake Fear Not Innate • What Role for Direct Experience? – Snake Fear Not Acquired Directly • Adaptive Value of Vicarious Learning – Vicarious Learning of Human Fears, Phobias 17 Test of Snake Fear • Wisconsin General Test Apparatus • Obstacle Between Animal and Food Reward – Must Reach Past Object to Obtain Food • Measure of Fear – Latency to Reach Past Object 18 Pretest of Snake Fear in Rhesus Monkeys Latency of Food-Reaching (secs) Mineka et al. (1984) 70 Snake 60 50 40 Wild 30 Lab 20 Cord 10 0 Real Toy Model Black Yellow Neutral Obstacle 19 Vicarious Conditioning of Snake Fear Mineka et al. (1984) • Wild-Reared Adults – Already Afraid of Snakes • Lab-Reared Adolescent Offspring – No Snake Fear • Posttest After Observing Parent 20 Snake Fear in Rhesus Monkeys Prior to Vicarious Exposure Time Spent With Object (secs) Mineka et al. (1984) 250 200 Snake 150 Parents Offspring 100 50 0 Real Toy Model Neutral Object 21 Snake Fear in Rhesus Monkeys After Vicarious Exposure Time Spent With Object (secs) Mineka et al. (1984) 250 200 150 Parents 100 Offspring Snake 50 0 Real Toy Model Neutral Object 22 Variants on Observational Learning • Unrelated Adult as Model • Immunization through Extensive Prior Exposure to Snakes • Snake vs. Flower – Mirrors, Video to Control Exposure – Model Sees Snake – Observer Sees Snake or Flower 23 Examples of Observational Learning by Animals in the Wild • • • • Chickadees and Milk Bottles Red Squirrels and Hickory Nuts Israeli Roof Rats and Pine Cones Chimpanzees and Panda Nuts Scientific American 24 The “Bobo Doll” Experiment Bandura et al. (1961) • Nursery-School Children • Adult Model – Aggressive – Non-Aggressive – Control • Aggression During Free Play – Imitative: Physical and Verbal – Punching, Hitting with Mallet – Nonimitative Aggresion, Gun-Play 25 Aggression Scores Model Aggressive Non-Aggressive 18 Aggression Score 16 14 12 10 8 6 4 2 0 Physical Verbal Aggression Category Other 26 Observational Learning and Language Acquisition • Native or Secondary Language • Exposure to Language – Aural or Gestural – Critical Period before Puberty • Detect Sounds of Language – Separate into Words • Recognize Words • Grammatical Rules • Prosody, Nonverbal Communication 27 Statistical Learning by Infants Saffran, Aslin, & Newport (1996) • Stimulus Materials • Test Items, Exp. 1 – Tupiro Golabu Dapiku Tilado • Test Items, Exp. 2 – Tupiro Golabu Tibida Kupado Listening Time (secs) – Tupiro Golabu Bidaku Padoti Familiar Novel 10 9 8 7 6 5 4 3 2 1 0 1 2 Experiment 28 Social Learning Theory Miller & Dollard (1941) • Imitation as Acquired (Secondary) Drive – Necessary Component of Socialization • Conformity • Discipline • Two Forms of Imitation – Matched-Dependent Behavior • No Awareness of Cues – Copying • Awareness of Cues 29 Cognitive Social Learning Theory Bandura & Walters (1963); Bandura (1977) • Expectations, not Behavior • Learning by Response Consequences – Direct Experience • Trial and Error • Reward and Punishment • Vicarious (Observational) Learning – Example • Modeling, Imitation (Informal) – Precept • Sponsored Teaching (Formal) 30 Social Learning and Culture • Culture: Customary Beliefs, Social Forms, and Material Traits of a Racial, Ethnic, or Social group – Transmitted through Informal Learning and Formal Training from One Generation to the Next • Observational Learning is the Cognitive Basis of Culture and Cultural Transmission • Thinking is the Cognitive Basis of Cultural Evolution 31 Micronesia Project